Lesson · 40 min · Free
Reproducible Workflows with WDL and Containers
Reproducible Workflows with WDL and Containers In computational biomedicine, an analysis is only as valuable as it is reproducible. A result you cannot reproduce — on another machine, next semester, or in a collaborator'
Reproducible Workflows with WDL and Containers
In computational biomedicine, an analysis is only as valuable as it is reproducible. A result you cannot reproduce — on another machine, next semester, or in a collaborator's lab — is difficult to trust, publish, or translate. This lesson introduces two technologies that together solve most reproducibility problems: Workflow Description Language (WDL) , a language for describing multi-step analyses, and containers (using Docker or Singularity/Apptainer), which package software together with its exact dependencies. Think of the problem first. A typical single-cell RNA-seq analysis involves many tools: a read aligner, a quantification tool, quality-control scripts, normalization libraries, and plotting packages. Each of these depends on specific versions of libraries, compilers, and even the operating system. If your collaborator has a different version of one library, their numbers may differ subtly from yours — a phenomenon sometimes called the dependency hell problem. Containers solve this by freezing the entire software environment into a portable image, while workflow languages solve the complementary problem: ensuring the sequence of steps , their inputs, and their parameters are recorded explicitly rather than in a researcher's memory or a notebook. WDL is one of several workflow languages (others include Snakemake, Nextflow, and CWL) and is widely used in genomics, for example by the GATK and the Broad Institute. A WDL script declares tasks — single commands wrapped with their inputs, outputs, and runtime requirements — and a workflow that chains tasks together. Crucially, each task can specify a Docker image, so every step of the pipeline runs inside a pinned, versioned container. When you execute the workflow on a different platform (a local laptop, an HPC cluster, or a cloud service such as Terra or Cromwell), the same container images are pulled, guaranteeing that the software behaves identically everywhere. The first example below shows a minimal WDL workflow with two tasks: fast quality trimming of reads with fastp , followed by quantification with . Note how each task declares its own container image, its inputs, and its command block. version 1.0 task trim_reads { input { File read1 File read2 } command >> output { File trimmed_r1 = "trimmed_R1.fastq.gz" File trimmed_r2 = "trimmed_R2.fastq.gz" File report = "trim_report.html" } runtime { docker: "biocontainers/fastp:v0.23.4" memory: "4 GB" cpu: 2 } } task quantify { input { File trimmed_r1 File trimmed_r2 File transcriptome_index } command >> output { File quant = "salmon_quant/quant.sf" } runtime { docker: "combinelab/salmon:1.10.0" memory: "16 GB" cpu: 8 } } workflow rnaseq_quant { input { File sample_r1 File sample_r2 File transcriptome_index } call trim_reads { input: read1 = sample_r1, read2 = sample_r2 } call quantify { input: trimmed_r1 = trim_reads.trimmed_r1, trimmed_r2 = trim_reads.trimmed_r2, transcriptome_index = transcriptome_index } output { File quant_matrix = quantify.quant } } The second example shows the container side of the workflow. Before a container image can be referenced in WDL, someone must build it from a Dockerfile — a short recipe that starts from a base image, installs the required software at pinned versions, and defines how the container runs. Pinning versions (rather than writing apt-get install fastp with no version) is what makes the image reproducible; without pins, rebuilding the image a year later may silently install different software. # Dockerfile for a pinned fastp environment FROM ubuntu:22.04 LABEL maintainer="course.example@university.edu" LABEL description="fastp 0.23.4 for read QC/trimming" ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update && apt-get install -y --no-install-recommends \ fastp=0.23.4-1 \ libncurses6 \ && rm -rf /var/lib/apt/lists/* CMD ["fastp", "--help"] Once built, the image is tagged (e.g., docker build -t myorg/fastp:0.23.4 . ), pushed to a registry such as Docker Hub or Quay.io, and referenced by that exact tag in the WDL runtime block. On systems where Docker is unavailable (many HPC clusters), Apptainer can run the same images directly from the registry, which is why container-based workflows port so well between laptops and clusters. Together, WDL plus containers give you a complete, machine-readable record of what was run, with what software , and in what order — the three questions any reviewer or collaborator will ask.
Key Takeaways
Reproducibility requires recording both the analysis steps (the workflow) and the software environment (the container). WDL organizes analyses into tasks (single commands with inputs, outputs, and runtime settings) and workflows that chain tasks together. Each WDL task should pin a specific container image tag ; unpinned tags like latest undermine reproducibility. Dockerfiles should install software at pinned versions so that rebuilding the image produces the same environment. Container-based workflows run identically on laptops, HPC clusters (via Apptainer), and cloud platforms, making collaboration and publication far easier. Tools like Cromwell execute WDL workflows and automatically track provenance — which inputs, containers, and commands produced each output file.
Practice Exercise
Using the two-task WDL workflow above as a template, extend it with a third task called qc_summary that runs MultiQC (container: quay.io/biocontainers/multiqc:1.21--pyhdfd78af_0 ) on the trimming report produced by fastp , and wire it into the workflow so it runs after trim_reads . Then write a minimal Dockerfile that pins MultiQC at version 1.21 on an Ubuntu 22.04 base. Finally, in two or three sentences, explain why pinning the container tag in the runtime block — rather than relying on latest — matters for a colleague who will rerun your pipeline six months from now.
Watch the full lesson — free
This topic is part of Computational Biomedicine: From Command Line to Single-Cell, a complete AI-narrated video course. Press play once and watch the entire lecture like a movie.
Start the course free →