๐Ÿ“–
OmniGround: Strengthening Omni-Modal Temporal Grounding via Attention-Consistency Training
Zixuan Chen, Depeng Wang, Hao Lin, Ke Xu, Ya Guo, Huijia Zhu, Tanfeng Sun, Xinghao Jiang

OmniGround: Strengthening Omni-Modal Temporal Grounding via Attention-Consistency Training

Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

Omni-modal large language models can jointly understand video, audio, and text, yet they remain weak at temporal grounding, a capability essential for real-world audio-visual applications. To understand this limitation, we look beyond final predictions and examine the model's internal temporal evidence. We identify a temporal generation gap: the internal attention of the model localizes the target moment more accurately than the timestamps it generates. To close this gap, we propose TACG (Temporal Attention-Consistency Grounding), a post-training method that treats internal attention as a reliable temporal pivot and aligns it with both the model prediction and the ground-truth annotation through attention-consistency training. We further build a 580K-sample training dataset by reconstructing temporal QA from dense audio-visual annotations and organizing general omni-modal instruction data. Using TACG and this training dataset, we train OmniGround, a general-purpose omni-modal model with stronger temporal grounding capability, without injecting any additional temporal cues. OmniGround reaches state-of-the-art omni-modal temporal grounding, including 37.5 mIoU on LongVALE and 35.7 mIoU on UnAV-100, while staying competitive on both general audio-visual understanding and vision-only grounding.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title OmniGround: Strengthening Omni-Modal Temporal Grounding via Attention-Consistency Training
Author / Organization Zixuan Chen, Depeng Wang, Hao Lin, Ke Xu, Ya Guo, Huijia Zhu, Tanfeng Sun, Xinghao Jiang
Subject & Discipline cs.CV
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-274053
Original Source Link Visit Source Page on arxiv.org »