Processing and analysis of Illumina sequencing data

• Illumina Sequencing Analysis Pipeline
• Read processing with non-Illumina software
• Application Specific Files
  • ChIP-Seq and whole-genome sequencing
  • miRNA sequencing
• File Formats
• Data Storage

Illumina Sequencing Analysis Pipeline

Illumina provides a standard pipeline used to conduct image analysis and base calling for all types of samples. This process converts the images taken by the Illumina GA sequencer to digital intensity levels, and subsequently uses these intensity levels for base calling.
Data were processed with the following Illumina pipeline versions:

• 2008-01-07 to 2008-03-12: version 0.2.6
• 2008-04-25 to 2008-11-20: version 0.3.0
• 2008-12-23: version 1.0
• 2008-11-25 to 2009-05-28: version 1.3.2
• 2009-05-28 to 2009-11-19: version 1.4.0 (within SCS 2.3)
• 2009-11-19 to 2010-04-07: version 1.5.0 (within SCS 2.5)
• 2010-04-07 to 2010-07-30: version 1.6.0 (within SCS 2.6)
• 2010-07-30 to 2010-08-05: version 1.7.0 (within SCS 2.6)
• 2010-08-05 to present: version 1.7.0 (within SCS 2.8)

Two types of files are created containing the read sequences. The seq.txt file is a file containing only the sequence information. The qseq file contains both sequence and base by base quality information. Please consult us for directions on how to interpret the quality information. A summary of standard files is provided below.

In general only the seq.txt files are provided to users, but qseq files are available on request.

NOTE: For applications (e.g. miRNA sequencing) where the fragment length is lower than the read length, the adapter sequence must be located and trimmed. For this reason we recommend using the sequence files we provide with the adapter sequence removed, as described below.

Read data are passed through the Illumina alignment and sequence analysis program, GERALD, where they are aligned to a reference sequence using the Eland software. Depending on the sample, the reference sequence will be a genome sequence, RefSeq dataset, or mirbase sequences, etc.

NOTE: For the Illumina Sequencing Analysis Software and MIRO, the most recent version of genomes has been used from May 2008 to May 7, 2009. The only exception to this is human, which was updated in our references to hg_19 on April 30th, 2009. As genomes are update and changed, a record of when which version was used will appear here.

This software creates files containing alignment locations and information, which can be provided to the user at their request. One file of interest may be the s_[lane#]_sequence.txt file, which contains only reads passing quality filtering, in FastQ format, and is available upon request. Reads pass filtering when the base-calling software is confident in the first X bases (12 by default) of the read.

Before providing any data, we review the results of the GERALD analysis to be sure the reads are of high quality and sufficient in number. We do so by reviewing intensity level and sequencing error reports, as well as reviewing the number of reads which match the reference unambiguously, ambiguously, and not at all. If all these standards are passed, we then pass the sequence data on to the user. Eland is a convenient tool to assess the quality of read datasets. However, it has limitations (e.g. it tolerates only two mismatches, does not allow indel errors, has read length limitation, and trims all reads to the same length). We therefore recommend to users to consider the analysis of their data employing further mapping algorithms without these restrictions (e.g. SeqMap, Gem, or others).

Details regarding the Illumina Sequencing Analysis Software were provided by the Illumina Sequencing Analysis Software User Guide For Pipeline Version 1.3 and CASAVA Version 1.0 by Illumina, Inc.

Read processing with non-Illumina software

We generally use the output from the MIRO package (Kofler et al. manuscript in prep.) which uses Eland, GEM, or SeqMap as alignment programs.

The first step involved is to trim the adapter for samples with small insert sizes (e.g. miRNA and small RNA samples, NlaIII or DpnII Digital Gene Expression). No sequence in the resulting file contains the adapter sequence, apart from rare cases where the adapter sequence was not detected. In this file, only one copy of identical reads is kept (along with the the read count).

After preprocessing, the reads are aligned using the most appropriate alignment software (Eland, GEM, SeqMap). Three separate files are created containing reads which mapped unambiguously (1 place in the reference), ambiguously (2 or more places), or not at all, as well as a statistics file describing the results. MIRO also possesses an option to map reads to multiple references, and a set of these 3 files is created for each reference used. All of these files are provided to users.

A URL from which the aforementioned files can be downloaded is emailed to the user once available. If such an email is not received, please contact Debayan Datta.

Application Specific Files

Below is a description of standard files available for a given application. Any additional analysis should be discussed with Heinz Himmelbauer.

ChIP-Seq and whole-genome sequencing

These reads are generally aligned using the GEM alignment program (Ribeca, Guigo et al., in preparation), called from within MIRO. Bed and wig files are then generated for all ChIP-Seq samples. These files can be loaded into various genome browsers to view the location of the reads sequenced. They can also be used in conjunction with other software, such as that of Genomatix, and various programs designed for peak calling.

miRNA sequencing

Several different files are created by MIRO during analysis of miRNA samples. By default, the following files are created for quantile normalization, linear-scaling normalization and for non-normalized data:

1. A heatmap pdf with a colour coded representation of miRNA abundance, and a file containing a legend for the heatmap.
2. A lineplot pdf. This is primarily intended for viewing miRNA abundance for samples which represent progressive time points.
3. Scatterplot pdfs showing the correlation between samples.
4. Tables containing a list of miRNAs, and the number of reads found for each miRNA.

The following files may be created upon request:

1. A text file containing clusters of overlapping reads.
2. A text file containing clustered reads using a sliding window approach
3. Logos may be created for selected miRNAs (see WebLogo 3.0)
4. miRNA profiles which show the position of the reads on the reference sequence. The RNA folding prediction is also displayed.
5. 'bed' and 'wig' files for display with the UCSC Genome Browser
6. Files which are compatible with miRDeep for de novo miRNA identification

In depth descriptions of all these files, and the columns in each table, can be found in the MIRO documentation

File Formats

  4 1 23 1566 ACCGCTCTCGTGCTCGTCGCTGCGTTGAGGCTTGCG

  SOLEXA 90403 4 1 23 1566 0 1 ACCGCTCTCGTGCTCGTCGCTGCGTTGAGGCTTGCG `aaaaa```aZa^`]a``a``a]a^`a\Y^`^^]V` 1

	@SOLEXA:4:1:22:769#0/1 ATTTCCTGAGCTTAATGCTTGGGAGCGTGCTGGTGC +SOLEXA:4:1:22:769#0/1

	aaba_^``^^\_`_`Z_^_^Z^_^[^\WZW`^UW_Z

  3070||Count=45 GCCAAGCCCGTTCCCTTGGCTGTGGTTTCGCTTGATAGTA chr8 1 2 F 70764945

  track name=region 1 description="unknown" useScore=1  chr22  23901283        23901298        id:715          600     -

  track type=wiggle_0 name="unknown" description="variableStep format" visibility=full  variableStep chrom=chr7     span=1000  156001         30