wenmin-wu/cv-multichannel-to-rgb-adaptation

Multi-Channel to RGB Adaptation

Overview

Pretrained ImageNet backbones expect 3-channel RGB input. When working with multi-channel data (fluorescence microscopy, satellite multispectral, medical imaging), composite or select channels into a 3-channel image. Handles bit-depth normalization (16-bit to 8-bit) and channel selection strategy.

Quick Start

import cv2
import numpy as np

def multichannel_to_rgb(channels, bit_depth=16):
    """Composite multi-channel images to 3-channel RGB.

    Args:
        channels: dict of {"red": array, "green": array, "blue": array, ...}
        bit_depth: source bit depth (8 or 16)
    """
    r = channels["red"]
    g = channels["green"]
    b = channels["blue"]

    if bit_depth == 16:
        r = (r / 256).astype(np.uint8)
        g = (g / 256).astype(np.uint8)
        b = (b / 256).astype(np.uint8)

    return np.dstack([r, g, b])


def load_hpa_rgby(image_dir, image_id):
    """Load HPA-style RGBY channels into RGB."""
    colors = ["red", "green", "blue", "yellow"]
    channels = {}
    for c in colors:
        path = f"{image_dir}/{image_id}_{c}.png"
        channels[c] = cv2.imread(path, cv2.IMREAD_UNCHANGED)
    # Option: blend yellow into red+green
    return multichannel_to_rgb(channels, bit_depth=16)

Workflow

1. Load each channel as a single-channel array (preserve original bit depth)
2. Normalize to 8-bit (divide by 256 for 16-bit sources)
3. Select or blend channels into 3-channel RGB
4. Feed into pretrained backbone (ResNet, EfficientNet, etc.)

Key Decisions

• Channel selection: Pick 3 most informative channels, or blend extras into existing ones
• Normalization: Per-channel percentile clipping often outperforms linear scaling
• Alternative: Modify first conv layer to accept N channels (requires unfreezing + retraining)
• Domain: Applies to microscopy, satellite (Sentinel-2), and medical (MRI sequences)

References

• HPA Single Cell Classification competition (Kaggle)
• Source: mmdetection-for-segmentation-training