๐Ÿ“–
Decodability Is Not Controllability: Local and Anisotropic Speaker Control in Continuous-Latent TTS
Yuanzhong Zheng, Wei Huang, Ming Ma, Kailun Jin, Youhai Jiang

Decodability Is Not Controllability: Local and Anisotropic Speaker Control in Continuous-Latent TTS

Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

High linear decodability does not entail causal controllability in continuous-latent text-to-speech (TTS). In VoxCPM2, position-matched probes and equal-norm interventions across 30 speakers and 60 directed dyads expose this separation: reference-audio states at the final residual acoustic language model layer (RALM L7) retain 0.970 probe accuracy, yet aligned edits yield 0/60 target-dominant dyads. Edits at early reference-audio positions produce the opposite pattern. At L0, pair-aligned updates increase target similarity by 0.326 and achieve target dominance in all 60 dyads, against at most 1/60 under any equal-norm direction or position control. Independent speaker encoders, retrieval against held-out speaker prototypes, and double-blind listening confirm that the shifted voice is heard as the intended target. Cross-architecture interventions in dots.tts distinguish backbone leverage from direct acoustic control: backbone edits displace the source without target attraction, while x-vector ablation reduces source similarity from 0.849 to 0.006. These findings identify two requirements for training-free speaker control: geometric specificity and a computational route that carries the intended speaker condition into acoustic synthesis.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Decodability Is Not Controllability: Local and Anisotropic Speaker Control in Continuous-Latent TTS
Author / Organization Yuanzhong Zheng, Wei Huang, Ming Ma, Kailun Jin, Youhai Jiang
Subject & Discipline cs.SD
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-268038
Original Source Link Visit Source Page on arxiv.org »