Computing Facilities
Computing Clusters
The computing facilities for the UCSC Genomics Institute include two computer clusters available to our researchers. The clusters are built using Supermicro quad-node units. Each node has dual 16-core AMD processors and 256 GB of memory. The public network cluster, Ku, provides a total of 1024 processing cores available using the Parasol batch processing system. The cluster runs the Linux operating system and is managed using the Rocks toolset. The private network cluster is configured as an Openstack cloud. It provides 1792 processing cores configurable in a variety of ways using the Openstack software tools. Both systems were designed to provide an exceptional amount of inexpensive computing power in minimal space. For other memory-intensive jobs, we also have 5 large memory development machines, two of them with 1 TB of RAM each, two with 1.5 TB RAM and one with 3 TB. The computational facilities are supported by 4 clusters providing data storage and backups; together they manage over 10 petabytes of raw disk storage to provide replicated network storage using IBM GPFS and CEPH filesystems. The clusters connect to the network infrastructure using 10-gigabit ethernet.
The Genomics Institute also employs a high-availability connected computer setup for virtual machine hosting. These redundant machines have 64 computing cores, 500 gigabytes of memory, and access to 75 terabytes of local disk space.
UC Santa Cruz Genomics Institute’s System Administration Team:
- Jorge Garcia
- Erich Weiler
Web Servers
The web servers for the UCSC Genome Browser are housed in a data center designed to function 24/7, 365 days a year. They consist of 2 dual 24-core Intel processors; each offers 256 gigabytes of internal solid state storage, 120 terabytes of local disk data storage and 384 gigabytes of memory. Five additional servers, each with 256 gigabytes of memory, provide web access to BLAT (a BLAST-like alignment tool) software and its memory-intensive calculations. Servers available for public use include a genome preview server for access to raw data before it has gone through QA, a server that hosts all the browser MySQL data, a server to store user-generated custom tracks, and a wiki server that holds public information and can keep track of named sessions. A local download server that provides access to our raw data serves nearly 2 terabytes of data every day. We also provide a web server available to European users (genome-euro, hosted in Germany) and another available to Asian users (genome-asia, hosted in Japan).
Why Parallel Processors?
Computer clusters such as these are a cost effective way to process large amounts of data. Since many bioinformatics problems are “embarrassingly parallel,” they do not require high speed inter-process communication to perform calculations. This eliminates the need for high-priced networking equipment. Taking advantage of this fact by employing parallel but separate computation by many processors, we have pioneered the development of “supercomputing on-the-cheap” for the specific needs of genome presentation, annotation, and analysis.
Our current clusters are the fifth generation of clusters available at UCSC. The first generation was a cluster of 100 Pentium III processors that was built to assemble the first working draft of the human genome in June of 2000, using a 10,000-line program written by Jim Kent called GigAssembler. We have come a long way since then.
These computing systems are funded through the Howard Hughes Medical Institute, the National Human Genome Research Institute (NHGRI), the California Institute for Quantitative Biosciences (QB3), and the National Cancer Institute.