wenmin-wu/nlp-minmax-normalized-ensemble-blend

Min-Max Normalized Ensemble Blend

Overview

Different models produce predictions on different scales — one might output 0.01-0.99, another 0.3-0.7. Simple averaging lets the wider-range model dominate. Min-max normalizing each model's predictions to [0, 1] before blending ensures each contributes proportionally to the final ensemble regardless of its raw score range.

Quick Start

import pandas as pd

def minmax_normalize(series):
    return (series - series.min()) / (series.max() - series.min())

# Load predictions from different models
pred1 = pd.read_csv("submission_xlmr.csv")
pred2 = pd.read_csv("submission_bert.csv")

# Normalize then blend
submission = pred1.copy()
submission["toxic"] = (
    minmax_normalize(pred1["toxic"]) * 0.5 +
    minmax_normalize(pred2["toxic"]) * 0.5
)
submission.to_csv("submission.csv", index=False)

Workflow

1. Generate submission files from each model independently
2. Inspect score distributions (histogram) to confirm different scales
3. Apply min-max normalization to each model's prediction column
4. Blend with equal or tuned weights
5. Optionally re-calibrate the final blend if competition metric requires it

Key Decisions

• When to use: When models have visibly different score ranges; skip if ranges are similar
• Weights: Start with equal weights; tune via validation AUC if a holdout exists
• Rank-based alternative: scipy.stats.rankdata followed by dividing by N gives a rank-based normalization that is more robust to outliers
• Clipping: After blending, clip to [0, 1] if the metric expects probabilities
• N models: Normalize each independently, then apply weighted sum

References

• [TPU-Inference] Super Fast XLMRoberta