Scaling read aligners to hundreds of threads on general-purpose processors

Abstract

Abstract Motivation General-purpose processors can now contain many dozens of processor cores and support hundreds of simultaneous threads of execution. To make best use of these threads, genomics software must contend with new and subtle computer architecture issues. We discuss some of these and propose methods for improving thread scaling in tools that analyze each read independently, such as read aligners. Results We implement these methods in new versions of Bowtie, Bowtie 2 and HISAT. We greatly improve thread scaling in many scenarios, including on the recent Intel Xeon Phi architecture. We also highlight how bottlenecks are exacerbated by variable-record-length file formats like FASTQ and suggest changes that enable superior scaling. Availability and implementation Experiments for this study: https://github.com/BenLangmead/bowtie-scaling . Bowtie http://bowtie-bio.sourceforge.net . Bowtie 2 http://bowtie-bio.sourceforge.net/bowtie2 . HISAT http://www.ccb.jhu.edu/software/hisat Supplementary information Supplementary data are available at Bioinformatics online.