1 Training data← Back to the data viewer2 TokenizerCompareAuditExplainer slidesCoordinate tokens3 Image budget4 MP vs 8 MP4 NotesData processingAssamese ra checkSynthetic checkSmall held-out blocksData EDAJaigyan latticeJaigyan v4 re-readEmpty pagesManipuri: 4B vs pipeline5 TrainingEmbedding warm-upSFT ablationsSFT merges2B distillation0.8B distillationDistillation explainedRL planRL ablationsRL training
Unified OCR · Qwen3.5-4B · RL

RL ablations

About this page

4B RL from sft_best_ckpt (verl, GRPO + DAPO fixes / GDPO): training curves, per-component rewards and validation per run (registry rl/runs/registry.json; metrics from the verl file logger and rollout dumps), plus the algorithms explained with real probe groups. Validation reward is weighted by the number of validation prompts per kind. Refreshes every minute.

loading…

Runs

Training curves

One line per run of the selected group; x = step. Hover a point for its value.

Reward components

Mean score of each component per step (verl: from the rollout dumps; TRL: its per-reward logs). Choose a run.

Benchmarks per checkpoint

Scored on node052 with the SFT scripts (OmniDocBench parse_headers prompt, olmOCR-Bench, Indic printed / handwritten WER, lower is better). The dashed line is sft_best_ckpt.

The loss is not cross-entropy

For each token of each sampled answer, PPO's clipped policy-gradient loss:

loss = − min( ρ · A , clip(ρ, 1 − εlow, 1 + εhigh) · A ), ρ = πnew(token) / πold(token)

A is the answer's advantage (below). ρ is how much more likely the token is now than when the answer was sampled. When ρ ≈ 1 the gradient is −A·∇log π(token): cross-entropy on the model's own answer, multiplied by A. A positive A trains the answer like an SFT target; a negative A pushes it away. The reference transcript is never a target: it is used only inside the reward.