1 Training data← Back to the data viewer2 TokenizerCompareAuditExplainer slidesCoordinate tokens3 Image budget4 MP vs 8 MP4 NotesData processingAssamese ra checkSynthetic checkSmall held-out blocksData EDAJaigyan latticeJaigyan v4 re-readEmpty pagesManipuri: 4B vs pipeline5 TrainingEmbedding warm-upSFT ablationsSFT merges2B distillation0.8B distillationDistillation explainedRL planRL ablationsRL training

Assamese ra check (v6 fix)

Decision 2026-11-01 (user, with an Assamese reader): the v6 fix is reverted for IndicDLP. The IndicDLP Assamese pages are old prints that use the Bengali র. OCR text must keep the letter as printed, so data-clean, indicdlp-v3-final(-np), their shards and the synthetic IndicDLP set now keep the text as the annotators typed it (the row "typed" below). Modern NCERT and hw-vs Assamese text (model OCR) keeps the ra fix. This page stays as the record of the check.
Decision (user, 2026-11-01, checked with an Assamese reader): the v6 fix is reverted for IndicDLP. The IndicDLP Assamese pages are old prints that use the Bengali র. OCR text keeps the letter as printed, so the training text now has র where the annotators typed it (data-clean, indicdlp-v3-final, v3-final-np, their shards, and syn-pr-indicdlp). Each card shows the crop and the training text. hw-vs and NCERT (modern prints, model OCR text) still use ৰ.

The annotators typed the Bengali letter র (U+09B0). The v6 fix changed every one of them to the Assamese ra ৰ (U+09F0). This page shows whether the printed letter is ra ৰ or wa ৱ (U+09F1). The letter ৱ that the annotators typed correctly is not changed by the fix.

Every Bengali র in the transcript is red; the word for this card is marked yellow. Compare the marked word with the crop.

ৰAssamese ra: stroke inside the letter
ৱAssamese wa: stroke below the letter
রBengali ra: dot below (what the annotators typed)