GESTIÓN EN SALUD PÚBLICA: GotCloud

lunes, 15 de diciembre de 2014

GotCloud - Genome Analysis Wiki

[hide]

Genomes on the Cloud (GotCloud) Introduction

To handle the increasing volume of next generation sequencing and genotyping data, we created and developed software pipelines called Genomes on the Cloud (GotCloud).

GotCloud contains Mapping & Variant Calling Pipelines.

Key Features:

Connects sequence analysis tools together in automated pipeline
- Alignment, quality control, variant calling
Robust against unexpected system failure using GNU make
- easy restart after failure
Massively parallel, can run hundreds of jobs
- Splits large jobs into many pieces
- Simplifies running on clusters
Scalable to tens of thousands of samples
Easy to use - Automates series of configurable steps
- user doesn't have to understand/configure/know the many tools required to create high quality results
Available on Amazon Web Services (AWS) Elastic Compute Cloud (EC2)
Run on local machines/clusters

GotCloud incorporates the alignment and variant calling pipelines that we have been using at UM into one easy to use, publicly available tool. GotCloud can run on a user's computer, on an instance in a compute cloud, and/or can split the work up onto a cluster of machines or instances.

Getting Help with GotCloud

Please join in the GotCloud Google Group to ask / discuss / comment about these pipelines.

Currently the "join" button appears to be missing. Click "NEW TOPIC", then select "Join this group". You can then cancel the message post (or post a message).

See GotCloud: FAQs if you have any questions. If your questions are not answered there, join the google group and ask, or email Mary Kate Wing (mktrost@umich.edu).

Sequence Analysis Background Information

There are many essential steps in the analysis of next generation sequence data.

Next generation sequence data analysis starts with FASTQ files, the typical format provided from your sequencing center containing the sequence & base quality information for your data.

The fastq files are processed using the alignment pipeline which finds the most likely genomic location for each read and stores that information in a BAM (Binary Sequence Alignment/Map format) file. In addition to the sequence and base quality information contained in FASTQ files, a BAM file also contains the genomic location and some additional information about the mapping. As part of the alignment pipeline, the base qualities are adjusted to more accurately reflect the likelihood that the base is correct.

The alignment pipeline can be skipped if you already have Deduped and Recalibrated BAM files.

The variant calling pipeline processes the deduped and recalibrated BAM files produced by the alignment pipeline or that you provide it, generating an initial list of polymorphic sites and genotypes stored in a VCF (Variant Call Format) file. The variant calling pipeline then filters the variants using both hard filters and a Support Vector Machine (SVM). It then uses haplotype information to refine these genotypes in an updated VCF file.

After completing the GotCloud Variant Calling Pipeline, EPACTS (Efficient and Parallelizable Association Container Toolbox) can be used to perform statistical tests to identify genome-wide association from sequence data.

GotCloud Setup

You may run the GotCloud software in several modes:

On your own hardware running Ubuntu or Redhat/CentOS. See the instructions about installing the software below.
On an Amazon Elastic Compute Cloud using Ubuntu or Redhat/CentOS if you do not have your own set of machines.
- See GotCloud: Amazon for more information.
- You can run on an EC2 cluster instance created by StarCluster.

GotCloud has been developed and tested on Linux Ubuntu 12.10 and 12.04.2 LTS. While it should work on other Linux systems, they have not yet been tested.

GotCloud on Amazon

You can take advantage of GotCloud AMI's when running on Amazon. The GotCloud already includes GotCloud and default reference files.

See GotCloud: Amazon for instructions on using GotCloud on Amazon.

GotCloud Setup on Any Linux Machine

GotCloud Dependencies

GotCloud requires certain things to be installed in order to run:

perl - gotcloud is a perl script and it calls many other perl scripts
- Zlib.pm - required for perl scripts to read compressed files.
make - GNU make is used to run the pipelines
java - required to run the beagle step of the ld-aware genotype refinement
curses/ncurses (required for samtools)
- On Ubuntu: sudo apt-get install libncurses5 libncurses5-dev
cmake (required for premo)
- On Ubuntu: sudo apt-get install cmake

You can check if your system has the proper software installed by invoking the command

[gotcloud_path]/scripts/check_requirements.sh

Install GotCloud Software

You can install gotCloud on your system as (follow the links for the appropriate instructions):

source release - contains the scripts and uncompiled source
binary release - contains the scripts and pre-compiled binaries (no source)

GotCloud: Versions describes the changes added to each version.

Alternatively if you are using Amazon EC2, you can use one of the following sets of instructions:

Create a machine instance based on the AMI we provide: Amazon Single Node
Create an EC2 cluster instance using StarCluster

For more information on using GotCloud on Amazon, see: GotCloud: Amazon

For more information on Amazon Web Services, see: https://aws.amazon.com/

GotCloud Reference/Resource Files

In order to run gotCloud, you need to provide Genetic Reference and Resource Files.

These include information about the reference sequence and dbnsp positions.

See: GotCloud: Genetic Reference and Resource Files for information about the required files. It contains a description of the required files, information about generating your own versions, as well as a downloadable set of files.