1,720,966 research outputs found
Domain-Aware Dialogue State Tracker for Multi-Domain Dialogue Systems
No full textIn task-oriented dialogue systems the dialogue state tracker component (DST) is responsible for predicting the current state of the dialogue based on the dialogue history and the user utterance. Current DST approaches rely on a predefined domain ontology, a fact that limits their effective usage for large scale conversational agents, where the DST constantly needs to be interfaced with ever-increasing services and APIs. Focused towards overcoming this drawback, we propose a domain-aware dialogue state tracker, that is completely data-driven and it is modeled to predict for dynamic service schemas, including zero-shot domains. Unlike approaches that propose separate models for prediction of intents, requested slots, slot status, categorical slots and non-categorical slots, we propose a single model in an end-to-end architecture. The proposed model utilizes domain and slot information to extract both domain and slot specific representations from a given dialogue, and then uses such representations to predict the values of the corresponding slot in a given domain. Integrating this mechanism with pretrained language models, our approach can effectively learn semantic relations and effectively perform transfer learning between domains or zero-shot tracking for domains not present in training
Recent Neural Methods on Dialogue State Tracking for Task-Oriented Dialogue Systems: A Survey
No full textThis paper aims at providing a comprehensive overview of recent developments in dialogue state tracking (DST) for task-oriented conversational systems. We introduce the task, the main datasets that have been exploited as well as their evaluation metrics, and we analyze several proposed approaches. We distinguish between static ontology DST models, which predict a fixed set of dialogue states, and dynamic ontology models, which can predict dialogue states even when the ontology changes. We also discuss the model’s ability to track either single or multiple domains and to scale to new domains, both in terms of knowledge transfer and zero-shot learning. We cover a period from 2013 to 2020, showing a significant increase of multiple domain methods, most of them utilizing pre-trained language models
Scalable Neural Dialogue State Tracking
No full textA Dialogue State Tracker (DST) is a key component in a dialogue system aiming at estimating the beliefs of possible user goals at each dialogue turn. Most of the current DST trackers make use of recurrent neural networks and are based on complex architectures that manage several aspects of a dialogue, including the user utterance, the system actions, and the slot-value pairs defined in a domain ontology. However, the complexity of such neural architectures incurs into a considerable latency in the dialogue state prediction, which limits the deployments of the models in real-world applications, particularly when task scalability (i.e. amount of slots) is a crucial factor. In this paper, we propose an innovative neural model for dialogue state tracking, named Global encoder and Slot-Attentive decoders (G-SAT), which can predict the dialogue state with a very low latency time, while maintaining high-level performance. We report experiments on three different languages (English, Italian, and German) of the WOZ2.0 dataset, and show that the proposed approach provides competitive advantages over state-of-art DST systems, both in terms of accuracy and in terms of time complexity for predictions, being over 15 times faster than the other systems
Investigating Proactivity in Task-Oriented Dialogues
Get PDFProactivity (i.e., the capacity to provide useful information even when not explicitly required) is a fundamental characteristic of human dialogues. Although current task-oriented dialogue systems are good at providing information explicitly requested by the user, they are poor in exhibiting proactivity, which is typical in human-human interactions. In this study, we investigate the presence of proactive behaviours in several available dialogue collections, both human-human and human-machine and show how the data acquisition decision affects the proactive behaviour present in the dataset. We adopt a two-step approach to semi-automatically detect proactive situations in the datasets, where proactivity is not annotated, and show that the dialogues collected with approaches that provide more freedom to the agent/user, exhibit high proactivity
What’s in a Food Name: Knowledge Induction from Gazetteers of Food Main Ingredient
Get PDFWe investigate head-noun identification in complex noun-compounds (e.g. table is the head-noun in three legs table with white marble top). The task is of high relevancy in several application scenarios, including utterance interpretation for dialogue systems, particularly in the context of e-commerce applications, where dozens of thousand of product descriptions for several domains and different languages have to be analyzed. We define guidelines for data annotation and propose a supervised neural model that is able to achieve 0.79 F1 on Italian food noun-compounds, which we consider an excellent result given both the minimal supervision required and the high linguistic complexity of the domain.Affrontiamo il problema di identificare head-noun in nomi composti complessi (ad esempio “tavolo” is the headnoun in “tavolo con tre gambe e piano in marmo bianco”). Il compito é di alta rilevanza in numerosi contesti applicativi, inclusa l’interpretazione di enunciati nei sistemi di dialogo, in particolare nelle applicazioni di e-commerce, dove decine di migliaia di descrizioni di prodotti per vari domini e lingue differenti devono essere analizzate. Proponiamo un modello neurale supervisionato che riesce a raggiungere lo 0.79 di F-measure, che consideriamo un risultato eccellente data la minima quantitá di supervisione richiesta e la alta complessitá linguistica del dominio
Benchmarking machine learning models on multi-centre eICU critical care dataset
Get PDFProgress of machine learning in critical care has been difficult to track, in
part due to absence of public benchmarks. Other fields of research (such as
computer vision and natural language processing) have established various
competitions and public benchmarks. Recent availability of large clinical
datasets has enabled the possibility of establishing public benchmarks. Taking
advantage of this opportunity, we propose a public benchmark suite to address
four areas of critical care, namely mortality prediction, estimation of length
of stay, patient phenotyping and risk of decompensation. We define each task
and compare the performance of both clinical models as well as baseline and
deep learning models using eICU critical care dataset of around 73,000
patients. This is the first public benchmark on a multi-centre critical care
dataset, comparing the performance of clinical gold standard with our
predictive model. We also investigate the impact of numerical variables as well
as handling of categorical variables on each of the defined tasks. The source
code, detailing our methods and experiments is publicly available such that
anyone can replicate our results and build upon our work.Comment: Source code to replicate the results
https://github.com/mostafaalishahi/eICU_Benchmar
A Robust Data-Driven Approach for Dialogue State Tracking of Unseen Slot Values
No full textA Dialogue State Tracker is a key component in dialogue systems which
estimates the beliefs of possible user goals at each dialogue turn. Deep
learning approaches using recurrent neural networks have shown state-of-the-art
performance for the task of dialogue state tracking. Generally, these
approaches assume a predefined candidate list and struggle to predict any new
dialogue state values that are not seen during training. This makes extending
the candidate list for a slot without model retaining infeasible and also has
limitations in modelling for low resource domains where training data for slot
values are expensive. In this paper, we propose a novel dialogue state tracker
based on copying mechanism that can effectively track such unseen slot values
without compromising performance on slot values seen during training. The
proposed model is also flexible in extending the candidate list without
requiring any retraining or change in the model. We evaluate the proposed model
on various benchmark datasets (DSTC2, DSTC3 and WoZ2.0) and show that our
approach, outperform other end-to-end data-driven approaches in tracking unseen
slot values and also provides significant advantages in modelling for DST
Proactive Systems and Influenceable Users: Simulating Proactivity in Task-oriented Dialogues
No full textWe investigate proactivity, the capacity of a dialogue system to provide relevant information even when not explicitly requested, in the context of task-oriented dialogues. We propose to extend the current task-oriented framework, and have investigated four aspects of proactivity: (i) the degree of proactivity provided by the system during a dialogue; (ii) the propensity of the user to be influenced by the system proactivity; (iii) the complexity of the domain ontology; (iv) the relation between user needs and application domain, in terms of expected failure situations. Under the hypothesis that proactivity helps to increase effectiveness and efficiency of dialogues, we set up a framework based on dialogue simulations, and experimented the four aspects mentioned above. Although the current implementation allows to simulate a limited amount of dialogue phenomena (e.g., system initiative only), we are able to show that proactivity might have strong effects on dialogues, reducing up to 60% of dialogue turns in an application domain of medium complexity
How to Use Gazetteers for Entity Recognition with Neural Models
No full textAlthough the use of end-to-end neural architectures has been proven to be effective on several sequence labeling tasks, the use of gazetteers in these architectures is still rather unexplored. We investigate several options, aiming at exploiting gazetteers to extract relevant features, and then at integrating these
features in a neural model for entity recognition. We provide experimental evidences on two datasets (named entities and nominal entities) and two languages (English and Italian), showing that extracting features from a rich model of the gazetteer and then concatenating such features with the input embeddings of a
neural model is the best strategy in all our experimental settings, significantly outperforming more conventional approaches
Toward zero-shot Entity Recognition in Task-oriented Conversational Agents
No full textWe present a domain portable zero-shot learning approach for entity recognition in task-oriented conversational agents, which does not assume any annotated sentences at training time. Rather, we derive a neural model of the entity names based only on available gazetteers, and then apply the model to recognize new entities in the context of user utterances. In order to evaluate our working hypothesis we focus on nominal entities that are largely used in ecommerce to name products. Through a set of experiments in two languages (English and Italian) and three different domains (furniture, food, clothing), we show that the neural gazetteer-based approach outperforms several competitive baselines, with minimal requirements of linguistic features
- …
