Porting the Variant Calling Pipeline for NGS data in cloud-HPC environment

Mulone, Alberto; Awad, Sherine; Chiarugi, Davide; Aldinucci, Marco

doi:10.1109/COMPSAC57700.2023.00288

Item

ITEM ACTIONSEXPORT

Add to Basket

Local TagsRelease HistoryDetailsSummary

Released

Conference Paper

Porting the Variant Calling Pipeline for NGS data in cloud-HPC environment

MPS-Authors

/persons/resource/persons141828

Chiarugi, Davide
Methods and Development Group Computing and Databases Services, MPI for Human Cognitive and Brain Sciences, Max Planck Society;

External Resource

No external resources are shared

Fulltext (restricted access)

There are currently no full texts shared for your IP range.

Fulltext (public)

There are no public fulltexts stored in PuRe

Supplementary Material (public)

There is no public supplementary material available

Citation

Mulone, A., Awad, S., Chiarugi, D., & Aldinucci, M. (2023). Porting the Variant Calling Pipeline for NGS data in cloud-HPC environment. In Proceedings of the 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC). doi:10.1109/COMPSAC57700.2023.00288.

Cite as: https://hdl.handle.net/21.11116/0000-000D-B44E-1

Abstract

In recent years we have understood the importance of analyzing and sequencing human genetic variation. A relevant aspect that emerged from the Covid-19 pandemic was the need to obtain results very quickly; this involved using High-Performance Computing (HPC) environments to execute the Next Generation Sequencing (NGS) pipeline. However, HPC is not always the most suitable environment for the entire execution of a pipeline, especially when it involves many heterogeneous tools. The ability to execute parts of the pipeline on different environments can lead to higher performance but also cheaper executions. This work shows the design and optimization process that led us to a state-of-the-art Variant Calling hybrid workflow based on the StreamFlow Workflow Management System (WfMS). We also compare StreamFlow with Snakemake, an established WfMS targeting HPC facilities, observing comparable performance on single environments and satisfactory improvements with a hybrid cloud-HPC configuration.