๐Ÿ“–
EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities
Eunice Akani, Romain Gemignani, Rim Abrougui

EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities

Science & Mathematics Published 2023 Verification Pending

Overview & Synopsis

Pre-trained language model based on the Transformers architecture has significantly advanced natural language processing. This allowed for better results on several tasks, such as text classification or named entity recognition. Most existing models are general, and their use in a particular domain is sometimes tedious. Indeed, some domains have terms that are outside of commonly written languages. In these conditions, the implementation of a domain-specific model is necessary. In this paper, we have introduced a language model for the DSO activities of Enedis. This model is based on state-of-the-art language models but has been trained on domain-specific data. We also show that using the model for specific tasks, such as text classification, increases the performance obtained compared to a general language model.

Indexed from repository: HAL Open Science (hal.science)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title EneBERT: A State-of-the-art Language Model Trained on a Corpus of Texts Generated from the Set of DSO Activities
Author / Organization Eunice Akani, Romain Gemignani, Rim Abrougui
Subject & Discipline Multidisciplinary Academic Open Research
Categories Science & Mathematics
Publication Year 2023
License / Rights Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website HAL Open Science (hal.science)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-212573
Original Source Link Visit Source Page on hal.science »