1 Training data← Back to the data viewer2 TokenizerCompareAuditExplainer slidesCoordinate tokens3 Image budget4 MP vs 8 MP4 NotesData processingAssamese ra checkSynthetic checkSmall held-out blocksData EDAJaigyan latticeJaigyan v4 re-readEmpty pagesManipuri: 4B vs pipeline5 TrainingEmbedding warm-upSFT ablationsSFT merges2B distillation0.8B distillationDistillation explainedRL planRL ablationsRL training

Distillation, explained step by step

How we turn our big 4B OCR model into a small 0.8B one. This page is for a reader who is new to distillation. Every example below is a real crop from our RL validation set, and every probability on this page was measured on 2026-11-12 with our real models: the teacher is the final 4B after RL (rl_best_ckpt) and the student is the 0.8B after SFT (sft_best_ckpt_0p8b). Training numbers come from the logs of our runs.

The plan on one picture

We have two branches. Branch A is the product we want. Branch B is the baseline with no teacher, so the paper can show how much the teacher helps.

  1. Words you need first
  2. The teacher and the student
  3. Step 1 — warm-up of the new token rows
  4. Step 2 — KD: learn from the teacher's probabilities
  5. Step 2 during training — what the logs show
  6. Step 3 — KD anneal
  7. Step 4 — on-policy distillation (OPD)
  8. Step 5 — RL (GRPO + DAPO)
  9. Branch B and the decision points
  10. All steps in one table

Words you need first

Token. The model does not write letters. It writes tokens: pieces of text from a fixed list of 281,920 entries (our extended tokenizer). The English label 15) Draw the human ear. is 8 tokens plus an end token:

Indic text is cut into pieces too: ഓഗസ്റ്റ് (Malayalam "August") is the tokens ഓ ഗ സ്റ്റ്.

Next-token probabilities. At every position, the model reads the image, the prompt and all earlier tokens, and gives a probability to each of the 281,920 tokens. The probabilities add up to 1. Then one token is picked (the most probable one when we benchmark; a random draw when we sample), and the model goes to the next position.

Loss. One number that says how wrong the model is. Training changes the weights a little in the direction that makes the loss smaller. The size of one change is set by the learning rate (LR).

Cross-entropy (CE). The normal loss of SFT: CE = −log p(correct token). If the model gives the correct token p = 0.9, CE = 0.105; at p = 0.5, CE = 0.69; at p = 0.01, CE = 4.6. Only the probability of the one correct token counts.

KL divergence. A measure of how different two probability lists are. KL = 0 means identical lists. It uses all tokens, not only the correct one. This is the key tool of distillation.

The teacher and the student

Teacher: 4B, final after RL

Qwen3.5-4B → our SFT (stages 1–3, merge r10b_819_r9f) → RL (GRPO + DAPO, step 160 anneal merged with step 130). Strong but slow and expensive to serve.

Student: 0.8B

Qwen3.5-0.8B with the same tokenizer. Five times fewer weights: fast and cheap, but it learns less from the same data on its own.

OmniDoc = OmniDocBench overall (parse with headers), olmOCR = olmOCR-Bench average, WER = Indic word error rate (house scorer, clean, macro), cat. = catastrophic crops (printed / handwritten). ↓ = lower is better.

Why distil? Trained alone on the same data (Branch B's start), the 0.8B gets 89.67 on OmniDoc. Our first distilled 0.8B (teacher = the 4B before RL) got 90.84, with better Indic WER too. The teacher gives the small model more information per training example than the label alone. The rest of this page shows how.

1Warm-up of the new token rows done

The stock 0.8B only knows the stock Qwen tokens (IDs 0–248,076). Our tokenizer adds 33,784 rows: table tags, the continued-BPE Indic pieces, <|loc_0..1000|> coordinate tokens and block tokens. A new row starts with no meaning.

Initialisation ("mapinit"). We copied the 4B teacher's trained rows for these tokens into the 0.8B (mapped to its smaller width). In the 2B test this start was clearly better than the "mean" start (the average of the old pieces of each token): validation loss 0.975 vs 1.087 at step 500.

Warm-up training. Then only these new rows train (all other weights frozen), with the same loss as step 2 (half teacher, half label), LR 2e-4, until the validation loss is flat. The 0.8B stopped at step 1,300: Qwen3.5-0.8B-kdwarm-mapinit-s1300. This is the start of the KD run.

Why first? If all weights trained from the start, the large errors of the random new rows would push big, noisy changes into the whole network. Warming up only the new rows lets them "catch up" first.

2KD: learn from the teacher's probabilities running now

Teacher forcing. We give both models the crop, the prompt and the correct answer. At every answer position, each model gives its probability list for the next token. Nothing is generated; it is one forward pass per model.

Both halves of the loss below (KL and CE) use this same input; only the target differs. So KD does correct the student's wrong guesses for the next token: on the Malayalam crop the student gives the label token ഗ only 15%, so CE = −log 0.153 = 1.88 and the KL both push it up. What KD never shows the student is a prefix that contains its own earlier mistake: the text before each position is always the correct label. This is why it is called "off-policy" (the input does not come from the student), and it is the gap that OPD and RL close (step 4).

The loss (per answer token, then averaged):

loss = 0.5 × KL(teacher ‖ student) + 0.5 × CE(label)

The CE half says: "the correct token is this one". The KL half says: "make your whole list look like the teacher's list". The KL here is forward KL: Σ pT(x) · log(pT(x) / pS(x)) over all 281,920 tokens x. It is large when the student gives a small probability to a token that the teacher thinks is likely, so the student must cover every option the teacher considers.

Try it on real crops

Pick a crop, then click a token. Bars show the real probabilities of the teacher (blue) and the student (orange) at that position. The small number under each token is the teacher's probability of the label token.

What the examples teach

The teacher knows more than the label. On the Malayalam crop at ഗ the student says ക്സ് 47% and ഗ only 15%; the teacher says ഗ 83%. The KL pulls the student's whole list towards the teacher's, so it also learns how sure to be, not only the right answer.
Sometimes the label is wrong, and the teacher is right. The Hindi crop starts with a handwritten "7", but the label किसने किससे कहा leaves it out. The teacher puts 100% on 7 at the first position. On दुर्बल क्षार = the label ends in "9", but there is no 9 in the image: the teacher puts 100% on ending the answer. With 0.5 / 0.5, the student gets half of its pull from each. This is one reason to keep the CE half (the label is the ground truth most of the time) and one reason to keep the KL half (it softens label noise).
Sometimes the image is truly unclear. On "Draw the human ear", the last word is cut at the crop edge. The teacher spreads its probability ( ai 21%, heart 12%, ear 1.4%). A label alone would say "ear 100%"; the teacher says "this is uncertain". This uncertainty is the extra information ("dark knowledge") of distillation.

Temperature T. Some setups divide the scores by T > 1 to flatten the teacher's list and show more small probabilities. We use T = 1: most OCR tokens are near-certain (see the English printed crop: almost all 1.0), and a flatter teacher would teach the student to doubt clear text.

Step 2 during training — what the logs show

Run kd8_0p8b_rlteach: 8 nodes (64 H100), 256 packs per step, each pack up to 24,576 tokens. One step is about 5.8 M tokens of which about 1.2 M are answer tokens (the loss is only on answer tokens; the image and prompt tokens are input). About 2,800 samples per step, about 12 s per step. LR: warm-up for 25 steps to 2e-4, then constant. A checkpoint and a validation every 50 steps; benchmarks every 100 steps from step 500.

kd = the KL part, ce = the label part, loss = 0.5 × kd + 0.5 × ce. Values of single steps (noisy).

How to read it

Validation loss: new KD vs old KD

new KD (teacher = 4B after RL)old KD (teacher = 4B SFT)

Page validation set, every 50 steps (the old KD up to 5,000; the last 1,000 steps were its anneal). The new KD is 0.03–0.12 higher at the same steps (0.09 at step 500). Possible reasons: the RL teacher has sharper lists that are harder to copy, or the change in data. This is still open; the benchmarks decide. Live scores: the "0.8B distillation" page.

3KD anneal next — separate run

During the constant-LR phase, each step moves the weights by a size set by the LR. With a constant LR, the model keeps "jumping around" the best point. Annealing brings the LR down to 0 so the steps get small and the model settles. We use the 1−√t shape (fast at the start, gentle at the end), the same as in our SFT.

The old KD shows the effect: validation loss 0.400 at step 4,000 (end of constant LR) → 0.372 at step 5,000 (end of anneal), and the train kd part 0.080 → 0.055. We start the anneal from a checkpoint of the running KD; the step is chosen when the benchmarks flatten.

We do not decay the LR in the main KD run. Then we can start the anneal from any checkpoint and keep training at constant LR if the scores still rise.

4On-policy distillation (OPD) next — smoke test passed

The problem KD cannot see

KD fixes wrong next-token guesses (step 2), but in KD the student always reads the correct earlier tokens. At inference it reads its own earlier tokens. After one wrong token it is in a situation it never trained on, and errors grow. Here are 4 real answers that the 0.8B wrote itself (sampled at temperature 1) for the Malayalam crop (label 1982ൽ ഓഗസ്റ്റ് 6 ന് ഇദ്ദേഹം മരിച്ചു.):

The first error is the same each time (ഓക്സ് for ഓഗസ്റ്റ്). After it, the answers go different ways and invent words. KD trained on the correct prefix only, so it never taught "what to do after ഓക്സ്".

The fix: the student writes, the teacher grades every token

  1. The student writes a full answer for a crop or page (its own "policy" — hence "on-policy").
  2. The teacher reads the same image and the student's answer, and gives its probability for each token the student wrote.
  3. For each token: score = log pS(token) − log pT(token). A large positive score means "the student was sure, the teacher thinks it is wrong". This is a one-sample estimate of the reverse KL, KL(student ‖ teacher).
  4. The student is trained to make these scores smaller, like a reward per token (−score) in RL.

Pick a crop and one of the student's 4 answers. The colour shows the score per token: darker = the teacher disagrees more. Click a token to see the teacher's top choices there.

Reverse vs forward KL. Forward KL (step 2) is weighted by the teacher, so the student must cover all of the teacher's options. Reverse KL is weighted by the student's own outputs, so the student is punished only for what it actually writes. A small model cannot copy everything a big model knows; reverse KL lets it focus on getting its own choices right. This is the "mode-seeking" property.

Why KD first and not OPD from the start

OPD needs a student whose answers are already close. If the answers are bad, the teacher's grades are uninformative (everything is wrong), and the generation of long answers is slow. KD is cheap and gets the student close; OPD then fixes the student's own errors. The same order is used by Qwen3-VL (off-policy, then on-policy distillation) and studied in "Rethinking OPD".

How OPD looks in training (our smoke test)

Framework verl. Student on 4 GPUs (rollouts with vLLM + FSDP training), teacher on 4 more GPUs (vLLM, which returns the log-probabilities of the given tokens: prompt_logprobs). 10 steps, 16 prompts per step, LR 1e-6. It runs end to end.

5RL: GRPO + DAPO after OPD

RL does not need a teacher. The student writes several answers per prompt, each answer gets a reward from a program that compares it with the label (for crops: 1 − character error rate; for pages: text, layout, tables and formulas), and the student is pushed towards its better-than-average answers. Same setup as our 4B RL.

GRPO in one example

For each prompt the student writes a group of answers (16 in training, 4 here). The advantage of an answer is its reward minus the group mean, divided by the group's standard deviation. Positive advantage → make this answer more likely; negative → less likely.

Reward here = max(0, 1 − CER) of the real student samples. The real crop reward also gives 0 to answers without an end token and subtracts a repetition penalty. The two run-on answers on the English crop were cut at 200 tokens in this probe, so they would also get 0.

DAPO dynamic sampling. On the Hindi crop all 4 answers write the "7" that the label lacks, so all rewards are equal (0.87). The advantages are all 0: this group teaches nothing and is dropped and replaced. In our 4B prompt selection, 72.6% of candidate prompts had such flat rewards.

RL vs OPD. OPD gives a signal at every token from the teacher. RL gives one number per answer, from the label. RL can go beyond the teacher (it optimises the benchmark-like reward directly); OPD cannot, but its signal is much denser. That is why A does OPD first, then RL.

How RL looks in training (our 4B run, the template for the 0.8B)

Branch B and the decision points

Branch B: 0.8B SFT → RL. The same RL recipe, started from the 0.8B trained without a teacher (sft_best_ckpt_0p8b). It answers the reviewer's question "what does the teacher add?".

Decision points in A:

All steps in one table

StepStudent inputSignalLossTeacher at train timeStatus
1 Warm-upcorrect textteacher lists + label0.5 fwd KL + 0.5 CE, new rows only4B SFT, forward passdone (s1300)
2 KDcorrect textteacher lists + label0.5 fwd KL + 0.5 CE, all weights4B RL, forward passrunning
3 KD annealcorrect textsamesame, LR → 0samenext
4 OPDits own answersteacher grade per tokenreverse KL (log pS − log pT), clip 54B RL, vLLM scoringsmoke passed
5 RLits own answersreward per answerGRPO + DAPOnoneafter OPD
B SFT → RLcorrect text, then own answerslabel, then rewardCE, then GRPO + DAPOnoneSFT done