GPTomics/bio-flow-cytometry-bead-normalization

Version Compatibility

Reference examples tested with: CATALYST 1.26+, CytoNorm 2.0+, flowCore 2.14+.

Before using code patterns, verify installed versions match. If versions differ:

• R: packageVersion('<pkg>') then ?function_name to verify parameters

normCytof() returns a LIST ($data, $beads, $removed, ...), not a flowFrame; beads="dvs" encodes EQ masses 140,151,153,165,175. Confirm with ?normCytof before relying on slot names.

Bead Normalization

"Normalize my CyTOF data" -> Correct instrument sensitivity drift with EQ beads (within/across runs), then harmonize batches with a reference anchor.

• R (drift): CATALYST::normCytof() (EQ-bead-based) or premessa
• R (batch): CytoNorm::CytoNorm.train() + CytoNorm.normalize() (per-cluster quantile splines)

The Single Most Important Modern Insight -- Two Different Layers; Anchor Controls Are the Guarantee

Bead normalization and batch normalization correct DIFFERENT things and are NOT interchangeable. (1) EQ-BEAD normalization (Finck 2013 Cytometry A 83:483) corrects within-run and run-to-run instrument SENSITIVITY DRIFT using the four-element beads as a physical internal standard - applied first, on raw counts. (2) CROSS-BATCH normalization (CytoNorm, Van Gassen 2020 Cytometry A 97:268) corrects staining/acquisition batch effects using a shared ANCHOR/reference sample present in EVERY batch, learning per-FlowSOM-cluster quantile-spline transforms. Beads cannot fix staining-batch or reagent-lot effects; CytoNorm cannot fix intra-run detector drift. The anchor control is the load-bearing design element: because it is biologically identical across batches, any cross-batch difference in it is technical BY CONSTRUCTION. Dropping the anchor (CytoNorm 2.0) is convenient but reintroduces the over-correction risk the anchor was designed to eliminate - so the safest stance for inference is to MODEL batch in the diffcyt design and reserve normalization for visualization/clustering display.

Why Per-Cluster and Many (99) Quantiles

Batch effects are cell-type-specific - a marker can drift in monocytes but not in T cells - so a single global channel transform over-corrects one population while under-correcting another and can erase real abundance differences. CytoNorm therefore learns the transform PER FlowSOM cluster. And it uses ~99 quantiles + a spline because the drift is non-linear and intensity-dependent (the negative and positive peaks move by different amounts); a single median shift or linear rescale reintroduces the distortion it is trying to remove.

EQ-Bead Normalization (drift)

Goal: Correct sensitivity drift and remove bead events.

Approach: normCytof() gates beads, computes the correction on the linear scale, and returns a list - the cleaned SCE is in $data.

library(CATALYST)

sce <- prepData(fs, panel, md)                         # no by_time arg - normalization is normCytof's job
res <- normCytof(sce, beads = 'dvs',                   # EQ masses 140,151,153,165,175
                 k = 500, remove_beads = TRUE, overwrite = FALSE)   # k = smoothing window (default; affects bead-trace viz, not correction magnitude)
sce_norm <- res$data                                   # normalized SCE; res$beads / res$removed available

Cross-Batch Normalization (CytoNorm)

Goal: Harmonize batches using a shared reference sample.

Approach: Train on the anchor (present in every batch) -> learn per-cluster quantile splines -> apply to the real samples. testCV() first: if cluster CV is high, the FlowSOM model is batch-unstable and per-cluster splines will distort (fall back to nClus=1).

library(CytoNorm)

model <- CytoNorm.train(files = ref_files, labels = batch_labels, channels = marker_channels,
                        transformList = tl,
                        FlowSOM.params = list(nCells = 6000, xdim = 10, ydim = 10, nClus = 10),
                        normMethod.train = QuantileNorm.train,
                        normParams = list(nQ = 99), seed = 42)
CytoNorm.normalize(model = model, files = sample_files, labels = batch_labels,
                   transformList = tl, transformList.reverse = tl_rev,   # BOTH required
                   outputDir = 'normalized/')

Per-Method Failure Modes

Treating bead and batch normalization as the same

Trigger: expecting beads to fix staining-batch effects. Mechanism: different layers. Symptom: residual batch structure after bead norm. Fix: bead norm for drift; CytoNorm for batch.

No anchor in a batch

Trigger: a batch lacking the reference sample. Mechanism: nothing biologically-identical to learn from. Symptom: that batch can't be normalized / is over-corrected. Fix: run the anchor in every batch (or model batch instead).

Over-correction

Trigger: CytoNorm with groups confounded with batch, or anchor-free on variable samples. Mechanism: splines absorb real biology. Symptom: attenuated group differences. Fix: testCV() check; model batch in diffcyt for inference; normalize for display only.

Using normCytof return as a flowFrame

Trigger: sce_norm <- normCytof(...). Mechanism: it returns a list. Symptom: downstream type error. Fix: res$data.

Quantitative Thresholds

| Threshold | Source | Rationale | |-----------|--------|-----------| | bead drift reduced ~4.9x -> 1.3x | Finck 2013 Cytometry A 83:483 | EQ-bead correction over a month of runs | | 99 quantiles, per-cluster | Van Gassen 2020 Cytometry A 97:268 | non-linear intensity-dependent, cell-type-specific drift | | EQ masses 140,151,153,165,175 (dvs) | CATALYST | DVS/Fluidigm EQ four-element bead set |

Common Errors

| Error / symptom | Cause | Solution | |-----------------|-------|----------| | normCytof output not usable | it returns a list | use res$data | | prepData(by_time=TRUE) errors | no such argument | use normCytof() for bead/drift correction | | CytoNorm distorts populations | unstable FlowSOM clustering | run testCV(); reduce nClus (or 1) | | batch effect remains | only bead-normalized | add CytoNorm with anchor samples |

References

• Finck 2013 Cytometry A 83(5):483-494 — EQ-bead normalization of CyTOF drift.
• Van Gassen 2020 Cytometry A 97(3):268-278 — CytoNorm per-cluster quantile normalization.
• Quintelier 2025 Cytometry A 107(2):69-87 — CytoNorm 2.0 (anchor-free; over-correction caveat).
• Chevrier 2018 Cell Syst 6(5):612-620 — CyTOF spillover (CATALYST normalization context).

Related Skills

Workflow order (CyTOF): EQ-bead drift normalization (raw counts, FIRST) -> cytometry-qc -> doublet-detection -> clustering -> CytoNorm cross-batch (LAST). The two normalization layers sit at opposite ends.

• cytometry-qc - EQ-bead-median-vs-Time is the primary CyTOF drift readout
• doublet-detection - Remove doublets before normalization
• compensation-transformation - Transform scale used by CytoNorm
• clustering-phenotyping - Cluster across normalized batches
• differential-analysis - Model batch in the design rather than over-cleaning
• experimental-design/batch-design - Anchor/reference-sample design; differential-expression/batch-correction for execution