This walkthrough provides a simple example of how to set-up and run the CloVR DigiNorm pipeline using the web-browser accessible CloVR dashboard.Â The pipeline uses the DigiNorm algorithm to normalize the dataset, substantially reducing the size without any significant impact on the assemblies that will be generate. For a detailed description about the DigiNormÂ algorithm please click here.
For this walkthrough, we shall utilize a sample datasetÂ from the HMP Â Illumina WGS ReadsÂ – Sample SRS018671.
This pipeline requires as much as 15GB of memory or more.Â Thus in most cases, additional computational resources may be needed. This walkthrough uses the cloud for computational support. Specifically, we demonstrate how to run the pipeline using the academic cloud DIAG, which is free for researchers. Alternatively, you could run the pipeline on Amazon EC2 or using other cloud computing providers.
Note: This pipeline is not yet available on the lastest CloVR release (clovr-1.0-RC5, Nov. 2012). For now, you can access this pipeline by requesting an account atÂ www.diagcomputing.org. After you login, select â€œStart CloVRâ€ from the â€œMy Accountâ€ drop-down. This will launch a new CloVR VM. Continue with the walkthrough from theÂ Add input datasets to the pipelineÂ step.
Getting started with CloVR
Installing and setting up CloVR is a one-time process. If you have done this before, you may skip to the next step â€“Â Setting up input dataset.
Alternatively, you could run this pipeline through DIAG. Simply request an account at www.diagcomputing.org. Once you log-in, click on “My Account” then “Start CloVR”. A CloVR Â VM will start up, and you can skip to the next step – Â Setting up input datasetÂ to continue.
CloVR is run using a local desktop client.Â Visit theÂ Getting started with CloVRÂ page to download and install the client. Once the CloVR virtual machine is set up and launched, you should see a screen similar to Figure 1.
Figure 1. CloVR desktop client
Start the CloVR web interface
First check the CloVR desktop window for the IP address of your virtual machine (VM). Then enter this IP address in a web browser as shown in Figure 2.
Figure 2. Accessing the CloVR web interface
Add cloud credentials to the pipeline
Visit theÂ Adding CredentialsÂ page for steps on how to add DIAG credentials. Once the your DIAG credentials are setup, you should see it listed within the credentials tab as shown in Figure 3.
Figure 3. Credentials
Setting up input dataset
Prepare input datasets
The input to this pipeline is a set of paired-end reads files -Â FASTQ format. The first step is to move your input data files into the user_data folder located within the clovr-standard-* image directory. This will enable you to easily access the data through the CloVR dashboard. You could skip this step if you’re running this pipeline through www.diagcomputing.org or if plan to upload the input data directly from your computer (more on this below).
Before starting a pipeline, you must add your datasets to the CloVR VM as â€œTagsâ€.Â To add tags, click â€œAddâ€ on the web interface.
Figure 5. Adding new tags
Then click on â€œSelect file from imageâ€, which will open a sub-window where you can select files for upload to the VM. Select the first paired end file to tag.
Alternatively, you can use â€œBrowseâ€ in the â€œUpload a fileâ€ window to find and select files from anywhere on your local computer. If you’re running this pipeline through www.diagcomputing.org, use the “Upload a file” option.
Select â€œNucleotide FASTQâ€ from the â€œFile Typeâ€ drop-down menu and name your dataset, e.g. as â€œSRS018671_1â€. Add an optional description of your dataset. Click â€œTagâ€ to upload the data to CloVR. A â€œCompleted Successfullyâ€ window should appear to indicate that your dataset was added to the CloVR VM. Repeat this step to tag the second paired-end file.
Now that the data we want to analyze has been tagged, we can setup and run the CloVR-diginorm pipeline.
Pipeline setup and execution
To initialize a new pipeline run, click on the â€œOther Protocolsâ€ drop-down as shown in the figure below. Then select â€œclovr_diginorm”.
This will open the pipeline configuration window. Select the Tag corresponding to the input dataset file(s). Select
Select the appropriate credentials from the â€œAccountâ€ drop-down menu. In this case, we’re using DIAG. If you’re running this pipeline through www.diagcomputing.org, select “local”.
Provide a name to recognize your pipeline in the web interface home page as â€œPipeline Descriptionâ€, e.g. â€œdiginorm_testâ€³.
Check your input by clicking â€œvalidateâ€. If the validation is successful, start the pipeline by clicking â€œRunâ€.
After a successful pipeline submission, the web interface will change to the â€œHomeâ€ page where the new pipeline will be listed as â€œStatus: running.â€
Monitoring the pipeline
Your pipeline should now appear in theÂ PipelinesÂ window in the CloVR dashboard along with its status (Figure 9). Occasionally, the pipeline may idle for a minute or two before running. You can click on the pipeline to get a description, input parameters, and hyperlinks to more advanced workflow interfaces like Ergatis (Figure 10). Additionally, once the pipeline completes, the results can be downloaded from this window by clicking on theÂ OutputsÂ tab. (Figure 11). The outputs consists of two files representing the normalized dataset.
Figure 9. Â Running, failed, idle, and complete pipelines are shown in the major panel.