RL training
About this page
Benchmarks of every scored RL checkpoint (node052, 4-benchmark chain: OmniDocBench parse_headers prompt, olmOCR-Bench parse prompt, Indic printed + handwritten block prompt). Step 0 = sft_best_ckpt scored the same way; Δ against step 0, or for a run that continues another (the anneal) against its start checkpoint; green = better; admissible always against SFT. Admissible = the selection rule in rl/STATUS.md. Refreshes every minute.
loading…
admissible = the selection rule in rl/STATUS.md: Indic printed and handwritten WER ≤ SFT, both catastrophic counts below SFT, OmniDoc ≥ SFT, olmOCR ≥ SFT − 1.1 (its noise).
0.8B RL: paper arms CA / CB (2026-11-13)
CA = RL on A (0.8B SFT, sft_best_ckpt_0p8b), pool probed with A; CB = RL on B (A + OPD, opd_best_ckpt_0p8b), pool probed with B. 4 nodes each. GRPO + DAPO, reward v1.4, 256 prompts × 16, mini 32, warm-up 25 then constant LR; validation (rl_val_08b, 963) every 10 steps, benchmarks every 20. Step 0 = the run's start model; Δ against it.
0.8B OPD (student: 0.8B SFT, teacher: final 4B after RL)
On-policy distillation (reverse KL, per-token clamp 5, no task reward), 256 prompts × 1 answer per step, 6 student + 2 teacher nodes. LR test 5e-6 → 2e-6 → 1e-6 (warm-up 10, about 1 h each), then the winner from step 0 with warm-up 50. Step 0 = the 0.8B SFT (sft_best_ckpt_0p8b); Δ against it. Benchmarks every 10 steps.
Latest scored checkpoint of each run
Mean of the last 3 scored checkpoints (smooths the checkpoint-to-checkpoint noise of about ±0.3 OmniDoc)
Per checkpoint
Headline
OmniDocBench parts
olmOCR-Bench categories
Indic per language
WER per language at each scored step (catastrophic count in brackets). Lower is better.