wenmin-wu/cv-ordinal-multilabel-encoding

Ordinal Multilabel Encoding

Overview

Standard classification treats ordinal classes (severity grades 0-4) as unrelated categories, ignoring their natural order. Ordinal multilabel encoding converts class K into K+1 binary labels where all labels up to K are activated. For example, class 3 becomes [1,1,1,1,0]. Training with sigmoid + BCE per label teaches the model that higher classes imply all lower classes. Decoding is simply summing active labels minus one.

Quick Start

import numpy as np

def ordinal_encode(labels, n_classes=5):
    """Convert ordinal labels to cumulative binary encoding.

    Args:
        labels: (N,) integer labels in [0, n_classes-1]
        n_classes: number of ordinal classes
    Returns:
        (N, n_classes) binary array
    """
    encoded = np.zeros((len(labels), n_classes), dtype=np.float32)
    for i, label in enumerate(labels):
        encoded[i, :label + 1] = 1.0
    return encoded

def ordinal_decode(preds, threshold=0.5):
    """Decode sigmoid predictions back to ordinal labels."""
    return (preds > threshold).astype(int).sum(axis=1) - 1

# Training
y_encoded = ordinal_encode(y_train, n_classes=5)
model.compile(loss='binary_crossentropy', optimizer='adam')  # sigmoid output
model.fit(X_train, y_encoded)

# Inference
y_pred = ordinal_decode(model.predict(X_test))

Workflow

1. Encode ordinal labels as cumulative binary vectors
2. Use sigmoid activation (not softmax) on the output layer
3. Train with binary cross-entropy loss per label
4. At inference, threshold each sigmoid output and sum active labels

Key Decisions

• vs softmax: Softmax ignores ordinal structure; this encoding enforces monotonicity
• Threshold: 0.5 is default; optimize on validation with QWK or other ordinal metric
• n_classes: Equals the number of ordinal levels (e.g., 5 for grades 0-4)
• Alternative: Frank & Hall method trains K-1 binary classifiers independently

References

• APTOS 2019 DenseNet Keras Starter