1 Training data← Back to the data viewer2 TokenizerCompareAuditExplainer slidesCoordinate tokens3 Image budget4 MP vs 8 MP4 NotesData processingAssamese ra checkSynthetic checkSmall held-out blocksData EDAJaigyan latticeJaigyan v4 re-readEmpty pagesManipuri: 4B vs pipeline5 TrainingEmbedding warm-upSFT ablationsSFT merges2B distillation0.8B distillationDistillation explainedRL planRL ablationsRL training

Coordinate tokens: what we add, how they start, how they learn

Unified Qwen3.5-4B OCR · model base_models/Qwen3.5-4B-cbpe32k-v3-loc · 2026-10-24

Every block in the answer carries a box: <div bbox="x1 y1 x2 y2" label="…">…</div>. We will test two ways to write the four numbers:

  • Digits (Qwen's own format): 152 138 957 258, about 16 tokens per box.
  • Coordinate tokens: <|loc_152|><|loc_138|><|loc_957|><|loc_258|>, exactly 4 tokens per box.

We added 1,001 tokens (<|loc_0|> … <|loc_1000|>) to the tokenizer. A matched pilot will decide which format the main run uses.

This deck explains: (1) the 0–1000 grid, (2) why new rows need a careful start, (3) the start we chose, (4) how training makes them precise.

1 · The 0–1000 grid: relative to the page, not pixels

The image is not made square. Only the numbers are normalized: 0 is the left/top edge, 1000 the right/bottom edge.

x_grid = round(1000 · x_px / W)    y_grid = round(1000 · y_px / H)
x_px = x_grid · W / 1000         y_px = y_grid · H / 1000

Page W H · box in pixels x1 y1 x2 y2

The 6.29 MP cap shrinks width and height by the same factor, so the grid numbers are identical for the original scan and for what the model sees. Precision: 0.1% of a side (≈3 px on a 3,000 px page).

2 · Digits vs coordinate tokens, as real tokens

Real encodings of one block header by our tokenizer (each box = one token; blue outline = coordinate token).

Digits
Coordinate tokens
For digits
  • Qwen already writes boxes this way (our probe: stock Qwen3.5-4B outputs bbox_2d digits on 0–1000).
  • Used by Chandra 2, LightOnOCR-3, Infinity-Parser2, MinerU, HunyuanOCR.
For coordinate tokens
  • ≈12 fewer tokens per block: ≈15–20% shorter answers, faster decoding.
  • Cannot produce malformed numbers. Used by Surya 2, Qianfan, Nemotron Parse.
  • Risk: 1,001 new rows that start without meaning.

3 · Why the start value of a new row matters

Each token has one row in the embedding table (2,560 numbers). Qwen3.5 ties its embeddings: the same row is used twice.

  • Input: when the model reads <|loc_152|>, it receives this row.
  • Output: to choose the next token, the model compares its final state with every row. The most similar row wins.

So a new row's start value decides both what the model "sees" and what it tends to predict, before any training. Two rows that start identical are always predicted with identical probability.

4 · The trap: averaging the digit rows

For BPE tokens we averaged the rows of the pieces (e.g. प्रश्न = mean of its 3 Qwen pieces). For numbers that fails: an average ignores order.

Type a coordinate 0–1000:

Numbers whose digit-average row would be identical to it:

Over all 1,001 values, only distinct rows would exist. The model could not tell 152 from 521 at the start.

5 · The start we chose: digit mean + small random noise

row(k) = m + noise_k

m = (row("0") + row("1") + … + row("9")) / 10    // "this is a number"
noise_k = random direction, length 0.3 × typical row length, fixed seed

What it gives
  • ✔ Every row starts as "a number" (Qwen's prior kept).
  • ✔ All 1,001 rows are different: no 152 = 521.
  • ✔ Small noise: no coordinate token dominates the output at the start.
What it leaves to training
  • Neighbouring values (151, 152, 153) do not start similar.
  • That order is learned in the warm-up, driven by the Gaussian targets (next slides).

Considered and dropped: a smooth sine-wave position signal (built first; it had ripples: values 500 apart started more similar than values 50 apart, and it needed tuning), a straight line from row(0) to row(1000) (with tied rows the model could then only prefer 0 or 1000 at the start), and hidden states from Qwen (most complex, small expected gain).

6 · How training makes them precise: Gaussian targets

Plain cross-entropy puts all target probability on the exact token. Predicting 153 for 152 is then as wrong as predicting 900. Gaussian targets spread a little probability over near values, so near misses cost less.

true value · σ

Only coordinate positions use these targets; text tokens keep plain cross-entropy. σ ≈ 2–3 grid units; the curve is cut at 0 and 1000 and renormalized.

7 · Training plan for the new rows, and the decision

Warm-up (before full SFT)
  • Train only the new rows: 32,768 BPE tokens, 12 table tags, 1,001 coordinate tokens.
  • Embeddings are one tied matrix: a gradient mask keeps old rows fixed, and weight decay 0 on embeddings (otherwise AdamW shrinks frozen rows anyway).
  • Vision encoder and backbone frozen. Stop when loss at new-token positions stops falling (≈ a few hundred to 1,000 steps).
Pilot A/B (same data, resolution, steps)
  • A: digits + plain cross-entropy.
  • B: coordinate tokens + Gaussian targets.
  • Optional C: coordinate tokens + plain cross-entropy (separates format from loss).
  • Measure: box F1 / IoU, reading order, CER, answer length, decode speed.
  • B wins if boxes are equal or better: shorter and faster.