48 research outputs found
The Music Annotation Pattern
Get PDFThe annotation of music content is a complex process to represent due to its inherent multifaceted, subjectivity, and interdisciplinary nature. Numerous systems and conventions for annotating music have been developed as independent standards over the past decades. Little has been done to make them interoperable, which jeopardises cross-corpora studies as it requires users to familiarise with a multitude of conventions. Most of these systems lack the semantic expressiveness needed to represent the complexity of the musical language and cannot model multi-modal annotations originating from audio and symbolic sources. In this article, we introduce the Music Annotation Pattern, an Ontology Design Pattern (ODP) to homogenise different annotation systems and to represent several types of musical objects (e.g. chords, patterns, structures). This ODP preserves the semantics of the object s content at different levels and temporal granularity. Moreover, our ODP accounts for multi-modality upfront, to describe annotations derived from different sources, and it is the first to enable the integration of music datasets at a large scale
RevOnt: Reverse engineering of competency questions from knowledge graphs via language models
Get PDFThe process of developing ontologies – a formal, explicit specification of a shared conceptualisation – is addressed by well-known methodologies. As for any engineering development, its fundamental basis is the collection of requirements, which includes the elicitation of competency questions. Competency questions are defined through interacting with domain and application experts or by investigating existing datasets that may be used to populate the ontology i.e. its knowledge graph. The rise in popularity and accessibility of knowledge graphs provides an opportunity to support this phase with automatic tools. In this work, we explore the possibility of extracting competency questions from a knowledge graph. This reverses the traditional workflow in which knowledge graphs are built from ontologies, which in turn are engineered from competency questions. We describe in detail RevOnt, an approach that extracts and abstracts triples from a knowledge graph, generates questions based on triple verbalisations, and filters the resulting questions to yield a meaningful set of competency questions; the WDV dataset. This approach is implemented utilising the Wikidata knowledge graph as a use case, and contributes a set of core competency questions from 20 domains present in the WDV dataset. To evaluate RevOnt, we contribute a new dataset of manually-annotated high-quality competency questions, and compare the extracted competency questions by calculating their BLEU score against the human references. The results for the abstraction and question generation components of the approach show good to high quality. Meanwhile, the accuracy of the filtering component is above 86%, which is comparable to the state-of-the-art classifications
From Detection to Explanation: Effective Learning Strategies for LLMs in Online Abusive Language Research
No full textAbusive language detection relies on understanding different levels of intensity, expressiveness and targeted groups, which requires commonsense reasoning, world knowledge and linguistic nuances that evolve over time. Here, we frame the problem as a knowledge-guided learning task, and demonstrate that LLMs' implicit knowledge without an accurate strategy is not suitable for multi-class detection nor explanation generation. We publicly release GLlama Alarm, the knowledge-Guided version of Llama-2 instruction fine-tuned for multi-class abusive language detection and explanation generation. By being fine-tuned on structured explanations and external reliable knowledge sources, our model mitigates bias and generates explanations that are relevant to the text and coherent with human reasoning, with an average 48.76% better alignment with human judgment according to our expert survey
Refining Statistical Data on the Web
Get PDFHarmelen, F.A.H. van [Promotor]Schlobach, K.S. [Copromotor]Scharnhorst, A. [Copromotor
OntoScope: Using a Divergent-Convergent Interaction Framework to Support LLM-based Ontology Scoping
Get PDFAn ontology is a formal, explicit specification of a shared conceptualization that, with problem‑solving and reasoning methods, supports efficient semantic technology development. In ontology engineering, Competency Questions (CQs) capture functional requirements that define an ontology's application domain. Auditing this domain scope with CQs is challenging because in nature, there are no clear domain boundaries, and ontology engineers must then decide which subdomains to cover (horizontal coverage) and how much detail to model (vertical granularity) in an ontology. LLM‑based systems can generate many candidate CQs to guide these decisions, but current tools underuse this potential: they lack support for users' divergent (lateral) and convergent (vertical) thinking in a visualized CQs space organized by coverage and granularity. As a result, users struggle to systematically decide which CQs to adopt, discard, or refine. We propose an interaction framework that fills this gap, demonstrated through OntoScope, an LLM‑based interactive system, and a user study with 15 ontology engineers. To our knowledge, this is the first validated interaction framework with an LLM‑based system that helps ontology engineers audit domain boundaries and unifies fragmented, expert‑driven ontology scoping practices into a coherent, accessible approach. More broadly, it shows how LLM‑based systems can transparently and accountably support a wider range of knowledge‑intensive tasks
OntoScope: Using a Divergent-Convergent Interaction Framework to Support LLM-based Ontology Scoping
Get PDFAn ontology is a formal, explicit specification of a shared conceptualization that, with problem‑solving and reasoning methods, supports efficient semantic technology development. In ontology engineering, Competency Questions (CQs) capture functional requirements that define an ontology's application domain. Auditing this domain scope with CQs is challenging because in nature, there are no clear domain boundaries, and ontology engineers must then decide which subdomains to cover (horizontal coverage) and how much detail to model (vertical granularity) in an ontology. LLM‑based systems can generate many candidate CQs to guide these decisions, but current tools underuse this potential: they lack support for users' divergent (lateral) and convergent (vertical) thinking in a visualized CQs space organized by coverage and granularity. As a result, users struggle to systematically decide which CQs to adopt, discard, or refine. We propose an interaction framework that fills this gap, demonstrated through OntoScope, an LLM‑based interactive system, and a user study with 15 ontology engineers. To our knowledge, this is the first validated interaction framework with an LLM‑based system that helps ontology engineers audit domain boundaries and unifies fragmented, expert‑driven ontology scoping practices into a coherent, accessible approach. More broadly, it shows how LLM‑based systems can transparently and accountably support a wider range of knowledge‑intensive tasks
Towards Explainable Automated Knowledge Engineering with Human-in-the-loop
Get PDFKnowledge graphs are important in human-centered AI as they provide large labeled machine learning datasets, enhance retrieval-augmented generation, and generate explanations. However, knowledge graph construction has evolved into a complex, semi-automatic process that increasingly relies on black-box deep learning models and heterogeneous data sources to scale. The knowledge graph lifecycle is not transparent, accountability is limited, and there are no accounts of, or indeed methods to determine, how fair a knowledge graph is in downstream applications. Knowledge graphs are thus at odds with AI regulation, for instance, the EU's AI Act, and with ongoing efforts elsewhere in AI to audit and debias data and algorithms. This paper reports on work towards designing explainable (XAI) knowledge graph construction pipelines with humans in-the-loop and discusses research topics in this area. Our work is based on a systematic literature review, in which we study tasks in knowledge graph construction that are often automated, as well as common methods to explain how they work and their outcomes, and an interview study with 13 experts from the knowledge engineering community. To analyze the related literature, we introduce use cases, their related goals for XAI methods in knowledge graph construction, and the gaps in each use case. To gain an understanding of the role of XAI models in practical scenarios, and reveal the requirements for improving the current XAI methods, we designed interview questions covering broad transparency and explainability topics, along with example discussion sessions using examples from the literature review. From practical knowledge engineering experience, we collect requirements for designing XAI methods, propose design blueprints, and outline directions for future research: (i) tasks in knowledge graph construction where manual input remains essential and where AI assistance could be beneficial; (ii) integrating XAI methods into established knowledge engineering practices to improve stakeholder experience; (iii) the need to evaluate how effective explanations genuinely are making human-machine collaboration in knowledge graph construction more trustworthy; (iv) adapting explanations for multiple use cases; and (v) verifying and applying the XAI design blueprint in practical settings
