Evaluating semantic classes used for ontology building and learning from texts

Overview & Synopsis

A large effort has been devoted to the development of ontology building tools but it is still difficult to assess their strengths and limitations. Proposed evaluations are hardly reproducible and there is a lack of well- accepted protocols and data. In this paper, we propose to decompose the evaluation of ontology acquisition process into independent functionalities. We focus on the evaluation of semantic class acquisition considered as a main step in the ontology acquisition process. We propose an approach to automatically evaluate semantic classes of ontologies that offer lexical entries for concepts. It is based on the comparative paradigm (to a gold standard). Its main focus is to compare how similar the generated semantic classes are to the gold standard concerning the disposition of concepts frontiers. This comparison relies on the lexical level and on the hierarchical structure of the "gold" concepts. The propositions are implemented, two experiments are settled on different domains and prove that the measures give a more accurate information on quality of systems' performances.