📖
Discrete Latent Structure in Neural Networks
Vlad Niculae, Caio Corro, Nikita Nangia, Tsvetomila Mihaylova, André Martins

Discrete Latent Structure in Neural Networks

Science & Mathematics Published 2025 Verification Pending

Overview & Synopsis

Many types of data from fields including natural language processing, computer vision, and bioinformatics, are well represented by discrete, compositional structures such as trees, sequences, or matchings. Latent structure models are a powerful tool for learning to extract such representations, offering a way to incorporate structural bias, discover insight about the data, and interpret decisions. However, effective training is challenging, as neural networks are typically designed for continuous computation. This text explores three broad strategies for learning with discrete latent structure: continuous relaxation, surrogate gradients, and probabilistic estimation. Our presentation relies on consistent notations for a wide range of models. As such, we reveal many new connections between latent structure learning strategies, showing how most consist of the same small set of fundamental building blocks, but use them differently, leading to substantially different applicability and properties.

Indexed from repository: HAL Open Science (hal.science)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Discrete Latent Structure in Neural Networks
Author / Organization Vlad Niculae, Caio Corro, Nikita Nangia, Tsvetomila Mihaylova, André Martins
Subject & Discipline Multidisciplinary Academic Open Research
Categories Science & Mathematics
Publication Year 2025
License / Rights Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website HAL Open Science (hal.science)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-159639
Original Source Link Visit Source Page on hal.science »