1,721,123 research outputs found
Multimodal cooperation with the DenK system
No full textIn this chapter we present the DenK project, a long-term effort where the aim is to build a generic cooperative human-computer interface combining multiple input and output modalities. We discuss the view on human-computer interaction that underlies the project and the emerging DenK system. The project integrates results from fundamental research in knowledge representation, communication, natural language semantics and pragmatics, and object-oriented animation. Central stage in the project is occupied by the design of a cooperative and knowledge-able electronic assistant that communicates in natural language and that has internal access to an application domain which is presented visually to the user. The assistant, that we call the Cooperative Assistant, has an information state that is represented in a rich form of type theory, a formalism that enables us to model the inherent cognitive dynamics of a dialogue participant. This formalism is used both for modeling domain knowledge, for representing the current dialogue context, and for implementing a context-change theory of communication
Multimodal Cooperative Resolution of Referential Expressions in the DENK System
Full text linkWe present an approach to the resolution of multimodal referential expressions in a cooperative human-machine communication setting, provided by the DenK system. We discuss how references involving multiple modalities are resolved, and we also indicate how the system can respond cooperatively in case the resolution process fails
- …
