📖
Beyond Clean Accuracy: The Practical Challenges and Benefits of Pixel-Based Language Models
Haowen Tang, Wei Xia, Raoyuan Zhao, Hinrich Schütze, Yihong Liu

Beyond Clean Accuracy: The Practical Challenges and Benefits of Pixel-Based Language Models

Published 2026 Verification Pending

Overview & Synopsis

Pixel-based language models process rendered text without relying on a fixed subword vocabulary, offering a promising approach to noisy and multilingual inputs. However, their practical value depends on whether they can be reliably adapted to downstream tasks. To this end, we systematically examine this question through fine-tuning behavior, typographical robustness, and zero-shot crosslingual transfer. Under our tested settings, pixel-based models require longer training and broader labelled-data coverage than text-based models, while delayed learning and learning-rate-dependent failures make adaptation less reliable. Our analyses further suggest that task-relevant information is less readily accessible in pretrained pixel representations, placing greater demands on fine-tuning. Once successfully adapted, however, pixel-based models lose substantially less accuracy under typographical noise and achieve higher target-language accuracy than English-pretrained text-based counterparts on two crosslingual benchmarks, despite weaker clean English performance. Together, these findings reveal a central trade-off: replacing tokenization with rendering reduces sensitivity to corrupted and unfamiliar inputs, but shifts a central bottleneck to downstream adaptation.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Beyond Clean Accuracy: The Practical Challenges and Benefits of Pixel-Based Language Models
Author / Organization Haowen Tang, Wei Xia, Raoyuan Zhao, Hinrich Schütze, Yihong Liu
Subject & Discipline cs.CL
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-277781
Original Source Link Visit Source Page on arxiv.org »