๐Ÿ“–
Read the Route: Attention-Steered Hypergraph Retrieval for Multi-hop RAG
Jiate Liu, Liuyi Chen, Xiaofei Yuan, Yifu Tang, Zihan Yang, Xu Zhou, Zhengyi Yang

Read the Route: Attention-Steered Hypergraph Retrieval for Multi-hop RAG

Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

Graph-based retrieval-augmented generation (GraphRAG) answers multi-hop questions by organizing a corpus into a graph and retrieving along its structure. Existing retrieval methods face two limitations. Diffusion- and similarity-based methods expand retrieval along fixed edge weights or by similarity to the question, without resolving its implicit entity references, allowing relevance errors to accumulate across hops. In contrast, exploration-based methods let an LLM reason about the next hop, but require a separate generation at each hop, leading to high inference costs. We observe that an LLM can resolve implicit entity references from retrieved passages and a multi-hop question before generating any tokens. Further analysis identifies a small set of attention heads, termed addressing heads, that capture this capability across multiple hops in a single forward pass. Building on this finding, we propose HyperAddress, which runs Personalized PageRank on a description-labelled hypergraph, using addressing heads to determine the restart distribution and transition weights, thereby guiding multi-hop retrieval without any generation at query time. On MuSiQue, 2WikiMultiHopQA, and HotpotQA, HyperAddress outperforms the strongest baseline by 3.5 points of Recall@5 and 7.4 points of All@5 on average, with gains that grow with the number of hops. It answers more than twice as fast and builds its index at half the cost.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Read the Route: Attention-Steered Hypergraph Retrieval for Multi-hop RAG
Author / Organization Jiate Liu, Liuyi Chen, Xiaofei Yuan, Yifu Tang, Zihan Yang, Xu Zhou, Zhengyi Yang
Subject & Discipline cs.IR
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-970
Original Source Link Visit Source Page on arxiv.org »