variant-calling/joint-calling/SKILL.md
Joint genotype a cohort of per-sample gVCFs with GATK (HaplotypeCaller -ERC GVCF -> GenomicsDBImport or CombineGVCFs -> GenotypeGVCFs) or GLnexus for DeepVariant gVCFs, producing a squared-off sample-by-site genotype matrix. Use when deciding between joint genotyping and merging single-sample callsets (never bcftools merge as absent==hom-ref), choosing GenomicsDBImport vs CombineGVCFs by cohort size and memory, solving the N+1 problem so a new sample does not force re-calling everyone, understanding cohort rescue of low-coverage het sites, handling the spanning-deletion star allele and GQ/PL recomputation at the joint step, scaling to biobank cohorts by interval sharding, or picking DeepVariant+GLnexus over the GATK path on throughput. Not for single-sample calling (see variant-calling/gatk-variant-calling) or VQSR/hard-filter mechanism (see variant-calling/filtering-best-practices).
npx skillsauth add GPTomics/bioSkills bio-variant-calling-joint-callingInstall this skill globally with one command. Works with Claude Code, Cursor, and Windsurf.
3 of 9 scanners reported clean
Some scanners were skipped, did not run, or reported a non-clean status. Review each row below.
Reference examples tested with: GATK 4.5+, GLnexus 1.4+, bcftools 1.19+
Before using code patterns, verify installed versions match. If versions differ:
<tool> --version then <tool> --help to confirm flagsIf code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.
"Joint genotype my cohort samples" -> Combine per-sample gVCFs into a single cohort callset with consistent genotyping across all sites, enabling cohort filtering and population-level analysis.
gatk HaplotypeCaller -ERC GVCF -> gatk GenomicsDBImport (or CombineGVCFs) -> gatk GenotypeGVCFsdeepvariant --output_gvcf per sample -> glnexus_cli --config DeepVariantWGSJoint genotyping is not the same operation as merging single-sample VCFs, and confusing the two silently corrupts a cohort callset. Two facts drive every decision below:
<NON_REF> allele in each gVCF lets GenotypeGVCFs distinguish confident homozygous reference (0/0) from no-data/no-call (./.) at a site another sample carries.The decision this forces: never bcftools merge single-sample callsets as if an absent record means hom-ref. A single-sample VCF omits sites where that sample looked reference, so a naive merge fills those cells with ./. (missing), NOT 0/0 - a sample genuinely hom-ref and a sample never assessed become indistinguishable, and downstream allele frequencies and association tests are wrong. ./. != 0/0 is the load-bearing distinction (see variant-calling/vcf-manipulation for merge semantics and variant-calling/vcf-basics for the genotype-field grammar). Genotype from gVCFs so every cell is filled from evidence, not assumption.
Single-sample calling discards cross-sample evidence that is critical for accurate genotyping:
Naive joint calling re-visits every BAM whenever the cohort changes: adding one genome forces re-calling all N. The gVCF workflow decouples expensive per-sample discovery (local assembly + PairHMM likelihoods, captured once per sample in the gVCF) from cheap cohort-wide genotyping. Adding sample N+1 then requires only generating that one gVCF plus re-running the cheap consolidation and GenotypeGVCFs - the assembly work for the existing N is never repeated. The gVCF is the reusable intermediate; GenomicsDB workspaces can even be updated in place (--genomicsdb-update-workspace-path). GATK frames this as decoupling "the initial identification of potential variant sites from the genotyping step, which is the only part that really needs to be done jointly" (see variant-calling/gatk-variant-calling for per-sample gVCF generation).
| Cohort Size | Approach | Notes | |---|---|---| | <100 | CombineGVCFs or GenomicsDB | Either works; CombineGVCFs is simpler to manage | | 100-10,000 | GenomicsDB + GenotypeGVCFs | Standard GATK Best Practices; shard by chromosome | | 10,000-100,000 | GATK Biggest Practices | Heavily sharded and parallelized across intervals | | >100,000 | DeepVariant + GLnexus, or Hail VDS | GATK becomes unwieldy at this scale; purpose-built tools required |
Both produce a combined object that GenotypeGVCFs consumes; they differ in how they store it and how they scale.
| | GenomicsDBImport | CombineGVCFs |
|---|---|---|
| Storage | GenomicsDB workspace on a TileDB array backend; transposes sample-centric gVCFs into a locus-centric sparse 2-D array (samples x positions) | Pure-Java hierarchical merge into a single combined gVCF |
| Scaling | Best when N is large; the locus-centric transpose is what makes per-locus genotyping fast at scale | Fails when N grows - memory-hungry and slow; recommended only as a small-cohort fallback |
| Portability | Workspace is not a plain gVCF; genotype via gendb:// | Output is a plain gVCF, portable and inspectable |
| Incremental | Add new samples with --genomicsdb-update-workspace-path (the N+1 win in practice) | No incremental mode; re-run over all samples |
| Best when | >100 samples, sharded by interval, biobank scale | <100 samples, or a small family/trio where simplicity wins |
Memory landmine specific to GenomicsDBImport: the heavy lifting runs in native C/C++ (TileDB), so cap the JVM heap (--java-options -Xmx) at ~80-90% of RAM. Over-allocating the JVM starves the native layer and causes a native out-of-memory failure that looks unrelated to heap size.
Sample BAMs
│
├── HaplotypeCaller (per-sample, -ERC GVCF)
│ └── sample1.g.vcf.gz, sample2.g.vcf.gz, ...
│
├── CombineGVCFs or GenomicsDBImport
│ └── Combine into cohort database
│
├── GenotypeGVCFs
│ └── Joint genotyping
│
└── VQSR or Hard Filtering
└── Final VCF
# Generate gVCF for each sample
gatk HaplotypeCaller \
-R reference.fa \
-I sample1.bam \
-O sample1.g.vcf.gz \
-ERC GVCF
# With intervals (faster)
gatk HaplotypeCaller \
-R reference.fa \
-I sample1.bam \
-O sample1.g.vcf.gz \
-ERC GVCF \
-L intervals.bed
# Process all samples
for bam in *.bam; do
sample=$(basename $bam .bam)
gatk HaplotypeCaller \
-R reference.fa \
-I $bam \
-O ${sample}.g.vcf.gz \
-ERC GVCF &
done
wait
For <100 samples:
gatk CombineGVCFs \
-R reference.fa \
-V sample1.g.vcf.gz \
-V sample2.g.vcf.gz \
-V sample3.g.vcf.gz \
-O cohort.g.vcf.gz
# Create sample map file
# sample1 /path/to/sample1.g.vcf.gz
# sample2 /path/to/sample2.g.vcf.gz
ls *.g.vcf.gz | while read f; do
echo -e "$(basename $f .g.vcf.gz)\t$f"
done > sample_map.txt
# Combine with -V for each
gatk CombineGVCFs \
-R reference.fa \
$(cat sample_map.txt | cut -f2 | sed 's/^/-V /') \
-O cohort.g.vcf.gz
For >100 samples, use GenomicsDB:
# Create sample map
ls *.g.vcf.gz | while read f; do
echo -e "$(basename $f .g.vcf.gz)\t$f"
done > sample_map.txt
# Import to GenomicsDB (per chromosome for parallelism)
gatk GenomicsDBImport \
--sample-name-map sample_map.txt \
--genomicsdb-workspace-path genomicsdb_chr1 \
-L chr1 \
--reader-threads 4
# Or all chromosomes
for chr in {1..22} X Y; do
gatk GenomicsDBImport \
--sample-name-map sample_map.txt \
--genomicsdb-workspace-path genomicsdb_chr${chr} \
-L chr${chr} &
done
wait
gatk GenomicsDBImport \
--genomicsdb-update-workspace-path genomicsdb_chr1 \
--sample-name-map new_samples.txt \
-L chr1
GenomicsDB is powerful but has sharp edges that can cause data loss or silent failures:
--consolidate periodically to merge fragments.--genomicsdb-update-workspace-path.--batch-size 50 to control memory consumption. The default is 0, which loads ALL samples in a single batch (maximum memory); a finite batch size trades a little speed for a bounded heap, so set it explicitly for large cohorts. Larger batches load more gVCFs simultaneously and can exhaust heap space.gatk GenotypeGVCFs \
-R reference.fa \
-V cohort.g.vcf.gz \
-O cohort.vcf.gz
gatk GenotypeGVCFs \
-R reference.fa \
-V gendb://genomicsdb_chr1 \
-O chr1.vcf.gz
# All chromosomes
for chr in {1..22} X Y; do
gatk GenotypeGVCFs \
-R reference.fa \
-V gendb://genomicsdb_chr${chr} \
-O chr${chr}.vcf.gz &
done
wait
# Merge chromosomes
bcftools concat chr{1..22}.vcf.gz chrX.vcf.gz chrY.vcf.gz \
-Oz -o cohort.vcf.gz
GenotypeGVCFs re-derives genotypes jointly from the stored per-sample PL vectors (including the <NON_REF> likelihood) under a Bayesian model; it does not simply copy per-sample genotypes into a wider file. Two consequences matter when reading the output:
<NON_REF> likelihood is redistributed onto them and PLs are recomputed; GQ is then the difference of the two smallest PLs. A sample's genotype/GQ in the joint VCF can therefore differ from what its single-sample gVCF implied - this is the rescue mechanism working, not a bug.--heterozygosity (expected theta, ~0.001 for humans; verify in-tool) and --indel-heterozygosity, folding the cohort allele count into each sample's posterior. --stand-call-conf (~30; verify in-tool) drops sites below that QUAL.* AlleleJoint genotyping across a cohort surfaces two representation issues absent from single-sample calling:
--max-alternate-alleles caps the ALT alleles genotyped per site (most-supported kept; confirm the default with gatk GenotypeGVCFs --help for the installed version). Genotyping cost scales roughly exponentially in ALT count, so GATK caps it and discourages raising it; --max-genotype-count similarly bounds genotype configurations. Highly multiallelic sites are also where GenotypeGVCFs can blow past very large RAM at scale.* spanning/overlapping-deletion allele (VCF 4.3 reserved) appears at a variant position that falls inside an upstream deletion carried by some samples. It means "for a sample carrying the upstream deletion, these bases are deleted/absent" - not reference, not the local ALT. Such a sample genotypes as */A or */*, which correctly keeps deletion-carriers from being called spuriously hom-ref at the interior site. Downstream tools must special-case it: VEP/SnpEff have no ref/alt sequence to predict a consequence on, and bcftools norm decomposition often splits or filters it, so it is a frequent source of annotation surprises after joint genotyping.For larger cohorts where multiallelic sites are common, allele-specific annotations allow VQSR to evaluate each allele independently rather than penalizing a good allele because a co-occurring allele is poor:
gatk GenotypeGVCFs \
-R reference.fa \
-V gendb://genomicsdb \
-O cohort.vcf.gz \
-G StandardAnnotation \
-G AS_StandardAnnotation
When allele-specific annotations are present, use -AS mode in VariantRecalibrator and ApplyVQSR for allele-level filtering.
Filtering the joint VCF is a whole-cohort step, not a per-sample one, and it must run after joint genotyping. VQSR (and its GATK successor VETS) fit a model to the cohort-wide distribution of site annotations - VQSR's Gaussian mixture needs enough variants and enough overlap with the truth resources to converge, which is why it is unreliable on a single exome or a small panel. This is the decision:
| Cohort | Filter | Why |
|---|---|---|
| Single deep WGS, or a joint cohort (~30+ exomes) | VQSR, or VETS (isolation forest, VQSR's successor) | Enough variants to fit a stable multivariate model -- one WGS genome supplies millions of sites, but exomes are variant-poor so the ~30-sample floor applies to exomes/panels, not WGS; pass -AS when allele-specific annotations were emitted so each allele at a multiallelic site is filtered independently |
| Single exome, gene panel, or too few variants | Hard filters | GMM will not converge on too few variants; use fixed per-annotation thresholds instead |
The full VariantRecalibrator/ApplyVQSR/VETS invocations, training resources, tranche levels, and hard-filter thresholds live in variant-calling/filtering-best-practices - this skill does not duplicate that mechanism. A minimal hard-filter fallback for a small cohort:
gatk VariantFiltration \
-R reference.fa \
-V cohort.vcf.gz \
--filter-expression "QD < 2.0" --filter-name "QD2" \
--filter-expression "FS > 60.0" --filter-name "FS60" \
--filter-expression "MQ < 40.0" --filter-name "MQ40" \
-O cohort.filtered.vcf.gz
Joint genotyping mitigates most batch effects because it re-evaluates genotype likelihoods across all samples simultaneously, recalibrating quality scores against the full cohort distribution. However, certain batch effects persist through joint calling because they affect the underlying read data, not the genotyping model:
Mitigation: process all samples through an identical upstream pipeline (same aligner, same duplicate marking, same BQSR resources). If batches are unavoidable, include batch as a covariate in downstream association or differential analyses.
| Scenario | Action | Rationale | |---|---|---| | Adding new samples | Re-genotype (GenomicsDB incremental add + GenotypeGVCFs on full database) | New samples change cohort allele frequencies, improving all genotype calls | | Changing reference genome | Full reprocess from alignment | gVCF coordinates are reference-specific | | Updating caller version | Optional but recommended for consistency | Different caller versions may produce different quality scores; mixing versions adds noise | | Adding new genomic intervals | Reimport from scratch | GenomicsDB intervals are locked at initial import; incremental update cannot expand them |
GLnexus is a scalable gVCF-merging/joint-genotyping engine (originally rocksdb-backed) that grows a cohort incrementally as samples are added, avoiding the full-cohort reprocessing the GenomicsDBImport + GenotypeGVCFs path requires. Yun et al. tuned its quality thresholds for DeepVariant output; the optimized presets ship in GLnexus v1.2.2+ as DeepVariantWGS (whole-genome) and DeepVariantWES (whole-exome). The original method was validated at cohort scale on ~50,000 exomes (Lin 2018 bioRxiv 343970). Prefer this path when DeepVariant is the caller (see variant-calling/deepvariant).
# Step 1: Run DeepVariant per sample to produce gVCFs
run_deepvariant --model_type=WGS \
--ref=reference.fa --reads=sample.bam \
--output_vcf=sample.vcf.gz --output_gvcf=sample.g.vcf.gz
# Step 2: Joint call with GLnexus (pre-tuned configs encode DeepVariant-tuned GQ + multiallelic handling)
# GLnexus emits BCF on stdout; pipe through bcftools to bgzip a VCF
glnexus_cli --config DeepVariantWGS --bed intervals.bed \
sample1.g.vcf.gz sample2.g.vcf.gz ... | bcftools view - | bgzip -c > cohort.vcf.gz
From the GLnexus benchmark (Yun et al. 2020 Bioinformatics 36:5582, GIAB 40x WGS and the 2,504-sample 1000 Genomes cohort). These are one study's figures at specific versions/coverage - treat as representative, not universal:
| Metric | DeepVariant + GLnexus | GATK (VQSR) | |---|---|---| | SNP F1 error | 0.07% | 1.23% | | Indel F1 error | 1.14% | 2.92% | | Cohort Mendelian violation rate | 1.7% | 5.0% | | Cohort merge time, chr22 (2,504 samples) | 0.84 h | 6.83 h (GenomicsDBImport + GenotypeGVCFs) | | Cohort gVCF footprint | 2.20 TB | 15.16 TB |
The throughput gap (GLnexus merge ~8x faster, DeepVariant gVCFs ~7x smaller on disk) is the practical reason large DeepVariant cohorts use GLnexus rather than routing DeepVariant gVCFs through GenotypeGVCFs.
Naive GenotypeGVCFs does not scale to tens of thousands of samples: I/O and per-site QUAL computation dominate, GenotypeGVCFs can exceed very large RAM at highly multiallelic sites, and single-interval GenomicsDB workspaces plus fragment proliferation and open-file-descriptor limits become the recurring failures. The scaling levers:
bcftools concat. A --sample-name-map file (sample<TAB>path, one per line) is mandatory at this scale - passing thousands of -V arguments is unmanageable and slow.QUALapprox INFO field without iterating over all genotypes, the dominant cost saver above ~tens of thousands of samples. Broad switches production to reblocking around ~2,000 samples for cost. gnomAD v2.1 aggregated its callset in Hail and filtered with a custom random-forest model rather than VQSR (Karczewski 2020 Nature 581:434); later releases (v3+) ingest gVCFs directly via the Hail sparse combiner.Goal: Run the full joint calling workflow from BAMs to filtered cohort VCF.
Approach: Generate per-sample gVCFs, import into GenomicsDB, joint genotype, then index and compute statistics.
#!/bin/bash
set -euo pipefail
REFERENCE=$1
OUTPUT_DIR=$2
THREADS=16
mkdir -p $OUTPUT_DIR/{gvcfs,genomicsdb,vcfs}
echo "=== Step 1: Generate gVCFs ==="
for bam in data/*.bam; do
sample=$(basename $bam .bam)
gatk HaplotypeCaller \
-R $REFERENCE \
-I $bam \
-O $OUTPUT_DIR/gvcfs/${sample}.g.vcf.gz \
-ERC GVCF &
# Limit parallelism
while [ $(jobs -r | wc -l) -ge $THREADS ]; do sleep 1; done
done
wait
echo "=== Step 2: Create sample map ==="
ls $OUTPUT_DIR/gvcfs/*.g.vcf.gz | while read f; do
echo -e "$(basename $f .g.vcf.gz)\t$(realpath $f)"
done > $OUTPUT_DIR/sample_map.txt
echo "=== Step 3: GenomicsDBImport ==="
gatk GenomicsDBImport \
--sample-name-map $OUTPUT_DIR/sample_map.txt \
--genomicsdb-workspace-path $OUTPUT_DIR/genomicsdb \
-L intervals.bed \
--reader-threads 4
echo "=== Step 4: Joint genotyping ==="
gatk GenotypeGVCFs \
-R $REFERENCE \
-V gendb://$OUTPUT_DIR/genomicsdb \
-O $OUTPUT_DIR/vcfs/cohort.vcf.gz
echo "=== Step 5: Index ==="
bcftools index -t $OUTPUT_DIR/vcfs/cohort.vcf.gz
echo "=== Statistics ==="
bcftools stats $OUTPUT_DIR/vcfs/cohort.vcf.gz > $OUTPUT_DIR/vcfs/cohort_stats.txt
echo "=== Complete ==="
echo "Joint VCF: $OUTPUT_DIR/vcfs/cohort.vcf.gz"
# Increase Java heap for GenotypeGVCFs (default 4g is often insufficient for >500 samples)
gatk --java-options "-Xmx64g" GenotypeGVCFs ...
# For GenomicsDBImport, --batch-size controls how many gVCFs are loaded simultaneously
gatk GenomicsDBImport --batch-size 50 ...
| Symptom | Cause | Fix |
|---|---|---|
| Merged cohort has ./. where samples are truly hom-ref; allele frequencies look wrong | bcftools merge of single-sample VCFs treats absent records as missing, not 0/0 | Genotype from gVCFs (GenotypeGVCFs/GLnexus) so every cell is filled from evidence; never merge single-sample callsets for a cohort matrix |
| GenomicsDBImport dies with a native/out-of-memory error despite a large -Xmx | Over-allocated JVM heap starves the native TileDB layer | Cap --java-options -Xmx at ~80-90% of RAM; the heavy lifting is native C/C++ |
| Cannot update an existing sample in GenomicsDB | GenomicsDB has no sample replacement; only new sample names can be added | Recreate the workspace to fix a sample; use --genomicsdb-update-workspace-path only to ADD |
| --genomicsdb-update-workspace-path cannot expand to new regions | Intervals are locked at initial import | Reimport from scratch to add genomic intervals |
| * alleles / genotypes like */A break VEP/SnpEff or vanish after bcftools norm | Spanning-deletion symbolic allele has no ref/alt sequence to annotate | Expected after joint genotyping; special-case or split * records before annotation |
| VariantRecalibrator fails to converge or errors | Too few variants for the Gaussian mixture (a single exome/panel, not a single WGS) | Fall back to hard filters for exomes/panels below ~30 samples (see filtering-best-practices) |
| Fewer ALT alleles than expected at a multiallelic site | --max-alternate-alleles dropped the least-supported alts | Raise cautiously (cost scales ~exponentially); confirm the default with --help |
./. vs 0/0, the * allele)tools
End-to-end CLIP-seq pipeline from FASTQ to ENCODE-compliant binding sites, single-nucleotide crosslink maps, annotation, motifs, and (optionally) differential binding. Use when running the full Yeo lab eCLIP / iCLIP / iCLIP2 / iCLIP3 / irCLIP / PAR-CLIP analysis with SMInput control, protocol-specific UMI extraction, ENCODE STAR parameters, CLIPper or Skipper peak calling with stringent log2 FC and -log10 p thresholds, IDR rescue and self-consistency QC, and downstream motif registration with mCross or PEKA.
development
Detect, date, and contextualize whole-genome duplication (WGD / paleopolyploidy) events using wgd v2 (Chen et al 2024), KsRates (Sensalari 2022 substitution-rate-corrected Ks dating), DupGen_finder (Qiao 2019), MAPS (Li 2018 phylogenomic), POInT (Conant 2008 ordered-block), SLEDGe (2024 ML-based), Whale.jl (Bayesian DL+WGD), and synteny-anchored paranome construction. Use when identifying ancient polyploidy from Ks distributions and synteny block analysis, positioning WGD events relative to speciation, distinguishing tandem from segmental from WGD duplications, dating the 2R/3R vertebrate / fish / salmonid WGDs, building paranome and Ks-age mixture models, applying KsRates substitution-rate correction across lineages, or testing alternative biased-fractionation / dosage-balance models post-WGD.
tools
Build whole-genome alignments using Progressive Cactus (Armstrong 2020 reference-free clade-level WGA), Minigraph-Cactus (Hickey 2024 pangenome-aware), LASTZ chain/net (UCSC pipeline), MUMmer4 (Marçais 2018 pairwise), minimap2 -x asm5/10/20 (Li 2018 fast pairwise), AnchorWave (Song 2022 WGD-aware), and Mauve / progressiveMauve (bacterial). Operates the HAL toolkit (Hickey 2013) for downstream extraction including halSynteny, halLiftover, halBranchMutations, and hal2maf. Use when constructing multi-species alignments for comparative-annotation projection (TOGA), synteny detection, conservation analyses (phyloP / PhastCons), or pangenome graph construction; selecting between reference-free (Cactus) and reference-anchored (LASTZ chains/nets) approaches; tuning sensitivity for closely vs distantly related genomes; or producing HAL files for genome-wide downstream tools.
development
Detect syntenic blocks and structural rearrangements between genomes using MCScanX (Wang 2012), JCVI/MCScan (Tang 2008 Python), GENESPACE (Lovell 2022) for orthology-anchored riparian visualization, SyRI for structural variation, AnchorWave for sequence-level synteny, i-ADHoRe 3.0 for highly diverged species, SynNet for synteny networks, and ntSynt for multi-genome macrosynteny. Use when identifying collinear gene blocks across species, distinguishing macrosynteny from microsynteny, detecting inversions/translocations/duplications, anchoring orthology in WGD lineages, producing publication riparian plots, computing synteny block age via Ks (cross-references whole-genome-duplication), or running synteny-aware ortholog inference in polyploids.