wenmin-wu/nlp-multi-temperature-candidate-sampling

Multi-Temperature Candidate Sampling

Overview

A single temperature produces homogeneous candidates. Run nucleus sampling at 3+ temperatures (e.g. 0.55, 0.75, 1.05) to generate candidates with varying diversity levels, then pool all for MBR selection. Combines with beam search candidates for maximum coverage.

Quick Start

def generate_diverse_candidates(model, tokenizer, input_ids, attn_mask,
                                 temperatures=[0.55, 0.75, 1.05],
                                 samples_per_temp=4, top_p=0.9):
    all_candidates = []
    # Beam search candidates
    beam_out = model.generate(input_ids=input_ids, attention_mask=attn_mask,
        num_beams=8, num_return_sequences=4, early_stopping=True, max_new_tokens=256)
    all_candidates.extend(tokenizer.batch_decode(beam_out, skip_special_tokens=True))
    # Multi-temperature sampling
    for temp in temperatures:
        samp_out = model.generate(input_ids=input_ids, attention_mask=attn_mask,
            do_sample=True, temperature=temp, top_p=top_p,
            num_return_sequences=samples_per_temp, max_new_tokens=256)
        all_candidates.extend(tokenizer.batch_decode(samp_out, skip_special_tokens=True))
    return all_candidates

Key Decisions

• 3 temperatures: low (0.55) for safe variants, mid (0.75) for balanced, high (1.05) for creative
• Beam + sampling: beam gives high-probability candidates, sampling adds diversity
• top_p=0.9: nucleus sampling avoids degenerate low-probability tokens
• Pool size: ~16-20 candidates per input (4 beam + 4×3 sampling) is a good default

References

• Source: lb-35-9-with-regex-corrections-public-model
• Competition: Deep Past Challenge - Translate Akkadian to English