2B distillation
Progress of the 2B student runs in ocr_training_distill_new. Warm-up = a 2 × 2 test of how the
33,784 new token rows start ("mean" = average of their old pieces, "mapinit" = the teacher's trained rows mapped into
the 2B) and the learning rate (round 1: 2e-4, round 2: 5e-4). All runs: new rows only, teacher r2_a4,
loss = 0.5 × match-the-teacher + 0.5 × match-the-text, 16 packs per step on 2 nodes, compared at equal steps up to
1,500. The run with the lowest validation loss continues as the warm-up.
Runs
Benchmarks of the main 2B run (vs the 4B teacher)
Headline metrics as in decisions/02-sft.md: OmniDocBench overall (parse with headers, higher is better), olmOCR-Bench average (higher), Indic printed / handwritten word error rate, house scorer wer_clean macro (lower). Dashed line = the final 4B teacher r10b_819_r9f (OmniDoc headline = 3-run mean 92.48). Red line = the 4B's whole SFT path in total steps, batch 256 like the students: stage 1 (0–3,600), then stage 2 to its step 3,100 and stage 3 with the anneal (3,600 + 3,100 … 5,500, anneal in the last 1,000). OmniDoc: only its parse-with-headers points. Green in the tables = at or better than the teacher.
Every column as in the SFT ablations tab: OmniDoc overall and its parts (text edit, formula CDM, table TEDS, read order edit), olmOCR overall and its categories, Indic WER and CER (clean, macro over languages). ↓ = lower is better. Teacher OmniDoc = 3-run mean. The 2B merges are hidden; only the chosen one (best_2b_ckpt) is shown. Benchmarks every 100 steps.
2B: best_2b_ckpt + main run (constant LR, anneal 2,300 → 3,300)
0.8B: main run (LR 2e-4, constant to 4,000, anneal to 5,000) + best_0p8b_ckpt
Indic per language
OmniDocBench (higher is better)
olmOCR-Bench (higher is better)
Indic printed WER (lower is better)
Indic handwritten WER (lower is better)
Validation loss
Measured every 50 steps on the same held-out set for all runs.
Training loss
Total loss per step, smoothed (moving average over 20 steps).
Match-the-teacher part (KL)
Smoothed over 20 steps.
Match-the-text part (CE)
Smoothed over 20 steps.