Unified OCR · Training data

What the model sees: image, prompt and target, built by data-processing-training/unifiedResearch archive (tokenizer, image budget) →
The 6 prompts (MODE1–MODE6)
Clause order: BASE · MD · HDR · MATH · FIG · TBL. The language variant writes the page's primary language into BASE ("Parse this Hindi document image …"); the loader uses it on part of the samples. Shares decided 2026-10-28; 20% of samples carry the language hint. A page that cannot give a mode (no table, untrusted text, no header containers) uses another mode.

Source

Split
Each page is read from its WebDataset tar (make_shards.py output), exactly as training reads it; see the Storage tab. The viewer shows a fixed random sample (2,000 train / 500 val per source). Counts are built records after all page exclusions; "layout only" is the share of train pages whose text is not trusted.
page