๐Ÿ“–
Geometric knowledge spaces, Attention pairing and Architectures
Shang Xu

Geometric knowledge spaces, Attention pairing and Architectures

Author / Organization:
Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

In this paper, we propose a geometric approach to deriving attention mechanisms and exploring potential new transformer architectures. We begin by formalizing the notions of knowledge worlds and attention pairings, then demonstrate how this framework naturally leads to the architectures of standard attention, causal attention, linear attention, tensor attention, and recurrent neural networks. Finally, we introduce several new geometric spaces that may inspire novel neural network architectures.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Geometric knowledge spaces, Attention pairing and Architectures
Author / Organization Shang Xu
Subject & Discipline cs.LG
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-267175
Original Source Link Visit Source Page on arxiv.org »