Structure-Guided Learning for Audio Analysis under Low Supervision

Overview & Synopsis

For many audio analysis tasks, supervised learning is either inadequate or inefficient. Navigating a low-supervision landscape within modern data-driven frameworks requires carefully integrating expert domain knowledge to guide the learning process and constrain the set of possible solutions. In this dissertation, three contributions that provide structure to learning in the form of signal (or physical) priors or geometric constraints are presented. The first contribution builds on a geometric understanding of the Fourier frequency space through Optimal Transport. Building on this, a frequency-aware training objective for audio tasks, named Spectral Optimal Transport (SOT), is proposed. SOT is evaluated on synthetic data for harmonic parameter estimation using unsupervised analysis and synthesis, and compared to traditional spectral losses based on Lp norms. The second contribution presents an extensive evaluation of a pitch estimator trained using equivariant self-supervised learning across cross-dataset scenarios, along with practical improvements informed by domain knowledge. The improvements are shown to noticeably improve performance, enabling a lightweight model to match or exceed the performance of supervised baselines, while enabling real-time inference.The third and final part addresses the underexplored task of separating individual drum instruments from a drums-only mixture. The event-driven nature of percussive recordings, in which each sound arises from a distinct, discrete strike, is modeled as a prior within a synthesis model. This enables training a modular separation model that matches the performance of larger, fully supervised models while bypassing the need for separated tracks by relying instead on transcription annotations. Together, these three contributions demonstrate that providing signal and geometric structure to data-driven learning models is beneficial, particularly for musical audio analysis tasks under low supervision.