0.8B distillation
All benchmark scores of the 0.8B runs. New KD: teacher = final 4B after RL (rl_best_ckpt), v6 data, warm-up 25 steps then constant LR 2e-4 (the anneal is a separate run later). Old KD: teacher = 4B SFT (r10b_819_r9f), constant to 4,000, anneal to 5,000. SFT: the 0.8B trained on the v6 data with no teacher (LR 1e-4, anneal to 6,500). Benchmarks every 100 steps. OmniDoc = parse with headers, overall (higher is better); olmOCR average (higher); Indic printed / handwritten WER, house scorer wer_clean macro (lower); cat = catastrophic crops; cut = cut-off pages. Green = at or better than the old 0.8B final.
loading…
Reference checkpoints
OmniDocBench (higher is better)
olmOCR-Bench (higher is better)
Indic printed WER (lower is better)
Indic handwritten WER (lower is better)
Validation loss (page set)
Every 50 steps; new KD vs old KD on the same held-out set.