๐Ÿ“–
HORIA: 4D Human and Object Reconstruction from a Monocular Video with Interaction-Aware Re-Rendering
Hyeongjin Nam, Taeryung Lee, Daniel Sungho Jung, Kyoung Mu Lee

HORIA: 4D Human and Object Reconstruction from a Monocular Video with Interaction-Aware Re-Rendering

Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

Joint 4D reconstruction of a human and object from a monocular video is essential for understanding how people interact with their environment. While recent methods have made noticeable progress, capturing such human-object interactions remains difficult due to two main challenges. First, a video typically contains composite semantic context spanning multiple interaction phases, making it difficult to track the temporal changes of the interaction in the reconstruction. Second, under human-object occlusion, a single-view observation contains incomplete visual information about the interaction, resulting in geometrically inconsistent reconstructions. To address these challenges, we introduce HORIA, built upon two core designs. First, we decompose the entire input video into temporally grounded and semantically distinct interaction phases with motion-aware captioning, obtaining a specific text description for each phase. Second, conditioned on these text descriptions of the interaction, we propose an interaction-aware re-rendering that compensates for the limited single-view observation. Specifically, we render the initially reconstructed 4D human and object from novel viewpoints and refine the renderings with a text-to-video diffusion prior, which provides auxiliary novel-view supervision beyond single-view cues. As a result, our framework produces temporally coherent and accurate 4D reconstructions under diverse interaction scenarios, achieving state-of-the-art performance.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title HORIA: 4D Human and Object Reconstruction from a Monocular Video with Interaction-Aware Re-Rendering
Author / Organization Hyeongjin Nam, Taeryung Lee, Daniel Sungho Jung, Kyoung Mu Lee
Subject & Discipline cs.CV
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-274033
Original Source Link Visit Source Page on arxiv.org »