1,721,318 research outputs found
Analysis of the Dynamics of Cognitive Processes
Full text linkJonker, C.M. [Promotor]Treur, J. [Promotor
Coactive Design: Designing Support for Interdependence in Human-Robot Teamwork
No full textCoactive Design breaks with traditional approaches by focusing on effective management of the interdependencies among human-machine team members. Providing support for interdependence enables members of a human-machine team to recognize problems and adapt. Support for a variety of interdependence relations makes a team flexible. Flexibility, in turn, makes the team resilient by providing alternative ways to recognize and handle unexpected situations.Interactive IntelligenceElectrical Engineering, Mathematics and Computer Scienc
Cognitive Coordination for Cooperative Multi-Robot Teamwork
No full textMulti-robot teams have potential advantages over a single robot. Robots in a team can serve different functionalities, so a team of robots can be more efficient, robust and reliable than a single robot. In this dissertation, we are in particular interested in human level intelligent multi-robot teams. Social deliberation should be taken into consideration in such a multi-robot system, which requires that the robots are capable of generating long term plans to achieve a global or team goal, rather than just dealing with the problems at hand. Robots in a team have to cope with dynamic environments due to the presence of the others. Thus, a robot cannot foresee what its environment will be because other robots may change the environment. Moreover, multiple robots may interfere with each other. We can say that the need for coordination in a robot team stems from interdependence relationships between the robots. More specifically, one robot performing an activity may influence another robot's activity. In order to achieve good team performance, the robots in a team all need to well coordinate their activities. This dissertation studies the multi-robot teamwork in the context of search and retrieval, which is known as foraging in robotics. In a foraging task, a team of robots is required to search targets of interest in the environment and also deliver them back to a home base. Many practical applications require search and retrieval such as urban search and rescue robots, deep-sea mining robots, and autonomous warehouse robots. Requiring both searching and delivering makes a foraging task more complicated than a pure searching, exploration or coverage task. Foraging robots have to consider not only where to explore but also when to explore. Coordination for a foraging task concerns how to direct the movements of the robots and how to distribute the workload more evenly in a team. In this dissertation, we first proposed an agent-based cognitive robot architecture that is used to bridge the gap between low-level robotic control with high-level cognitive reasoning. Cognitive agents realized by means of the agent programming language GOAL are used to control both real and simulated robots. We carried out an empirical study to investigate the role of communication and its impact on team performance. The results and findings were used to study the multi-robot pathfinding and multi-robot task allocation problems. A novel fully decentralized approach was proposed to deal with the multi-robot pathfinding problem, which also reduces the communication overhead, compared to usual decentralized approaches. An auction-based approach and a prediction approach were proposed to deal with the dynamic foraging task allocation problem. The difference is that the prediction approach performs better with respect to completion time, while the auction-based approach performs better with respect to travel costs. In order to facilitate the identification of interdependence relationships between the agents in the early design phase of a multi-agent system, we developed a formal domain-independent graphical language that reflects the need for coordination in multi-agent teamwork.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc
Qualitative multi-criteria preference representation and reasoning
No full textThe research reported on in this thesis is part of a larger research project that aims to develop a negotiation support system called the Pocket Negotiator. This thesis focuses on the question how such a system can represent and reason about a user’s preferences between the possible outcomes of a negotiation. In real-world negotiations, there are many negotiation issues which can have many different values, resulting in a large space of complex outcomes. A negotiation support system needs to have a model of the user’s preferences over this outcome space. Although most current negotiation support systems use numerical measures such as utility to represent preferences, such quantitative preferences are hard to specify for human users, and so it would be more natural to model the user’s preferences in a qualitative way. Moreover, due to the exponential size of the outcome space, it is not feasible to specify a preference ordering directly. Therefore, we aim to represent the preferences in a more compact way by aggregating multiple evaluation criteria that influence preference. The main research objective of this thesis is to develop a framework for the representation of, and reasoning about such qualitative multi-criteria preferences. The thesis makes the following contributions. \u95 We propose strategies to derive preferences from incomplete or uncertain information about the objects to be compared. The decisive and safe strategy for incomplete information is based on the notion of least and most preferred completions of objects. The strategies for uncertain information are based on an ordinal representation of the certainty levels of facts. \u95 We argue that instead of negotiation issues, the negotiators’ underlying interests should be chosen as criteria, especially if the issues are not preferentially independent. We show that the use of interests as criteria is more flexible than modelling conditional preferences, and provides a better explanation of the derived preferences. \u95 We present a general framework for the representation of qualitative, multicriteria preferences, called Qualitative Preference Systems (QPS). The framework defines outcomes as value assignments to a set of variables which can have arbitrary domains, includes a knowledge base that can impose (hard) constraints and define new (abstract) concepts, and defines three types of criteria that can be combined in a tree structure. We show that the QPS framework is expressive, as it can model conditional preferences and underlying interests, goal-based preferences, bipolar preferences, and preferences represented in two other well-known approaches that are representative for a large number of purely qualitative preference modelling approaches. Moreover, we show that the goal-based variant of QPS is just as expressive. \u95 For all proposed preference representation frameworks we define corresponding argumentation frameworks that include a logical language, a set of inference rules, and a defeat relation. Some of the argumentation frameworks also provide the possibility to reason with background knowledge to derive information about the values of variables by default. \u95 We propose a mechanism to generate explanations for preferences represented in a QPS. We use the intuition that preferences can be explained by the criteria that are deciding in the overall preference. Moreover, we show how a system can use user-provided explanations to update its current preference model. \u95 Finally, we introduce a modal logic, called Multi-Attribute Preference Logic (MPL), that provides a language for expressing several strategies to qualitatively derive a preference between objects from property rankings. Three such strategies from the literature on prioritized goals are modelled. The additional value of the logic is that it is possible to reason not only about which objects are preferred according to a certain ordering, but also about the relation between different orderings.Interactive IntelligenceElectrical Engineering, Mathematics and Computer Scienc
Multimodal Surveillance: Behavior analysis for recognizing stress and aggression
No full textNowadays, camera systems are installed in military areas as well as in public spaces like schools, shopping malls, airports, and football stadiums. Human operators are monitoring the screens, looking for any signs of unwanted behavior and negative incidents. The task requires working personnel 24/7. With the ever increasing number of cameras, surveillance operators become overloaded. The nature of the task to constantly watch screens and the sparsity of notable events are bound to decrease the operators' focus. Furthermore, some events are hard to distinguish by video only: severe events such as gunshots and screams are much easier to hear than see. For these reasons, negative events may go by unnoticed and typically the recorded footage is inspected after the fact. A solution to these problems is the development of automatic multimodal (audio-visual) surveillance systems, which was the aim of this research thesis. Such systems should not take over the decisions of the operators, but should assist them in identifying unwanted behaviour. Operators would be notified when and where to focus. This is likely to reduce the number of missed events caused by screen prioritising or external and internal distractions. It is important to note that such a system should not be limited to recognizing violence. It has been shown that negative emotions and stress might precede aggression. Recognizing them in an early stage is very relevant since adopting proper measures at an early time can prevent the situation from escalating. Therefore, in this research thesis, besides a variety of manifestations of aggression, we have focused on automatically recognizing stress. Our aim was to design and implement a surveillance system that is able to emulate human perception. For that reason, we asked people to annotate stress and aggression on audio-visual recordings. We investigated several approaches to compute their annotations automatically. Recordings from real surveillance cameras are in general not available due to privacy reasons. We had to construct our own datasets. In order to ensure a high degree of realism as well as sufficient samples of stress and aggression, we have designed scenarios and hired semi-professional actors to play them. The actors were free to improvise after they received roles and short scenario descriptions. We have recorded stressful scenes at service desk and aggression related scenarios in a train and train station. To automatically recognize the stress and aggression levels, we have extracted acoustic, linguistic and visual features, referred to as low-level features. Using classifiers, we trained models which can be used to make prediction of stress or aggression level on new data samples. One shortcoming of this approach is that there is a semantic gap between the low-level features and the high-level stress and aggression assessment. We have contributed by bridging the semantic gap with semantically-meaningful intermediate representations of the stress concept. The intermediate representation of stress consists of the degrees to which stress is conveyed by speech and gestures with respect to the semantic message and the way in which the semantic message is expressed (e.g. intonation for speech, speed, rhythm, tension for gestures). Adding such a representation as an intermediate level in the stress recognition architecture improves the stress assessment, especially when the level of stress is high. Having both audio and video offers the possibility to construct a more complete representation of the scene. The multimodal fusion approach is expected to be a solution to deal with the shortcomings of each modality (e.g. noise for audio, occlusion for video). Despite the expected benefits, fusing information coming from different modalities is challenging. Typical problems are that some pieces of information are only apparent in one modality (e.g. verbal fight), and that multiple people in the scene can have different behaviors which might lead to different assessments based on where the focus is. These problems can lead to incongruent, or even contradicting information from the different modalities, which makes coming to the correct interpretation hard. To deal with the problem of fusing incongruent information we have proposed and validated five meta-features: audio-focus, video-focus, context, semantics and history. The meta-features and the audio-only and video-only aggression assessments form the intermediate level of the aggression recognition model. This novel approach significantly improved automatic aggression recognition by multimodal fusion.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc
Designing Human-Centered Systems for Reflective Decision Making
No full textTaking major life decisions, e.g. what career to follow, is difficult and sometimes emotional. One has to find out what exactly one wants, consider the long-term consequences of the decisions and be empathetic for loved ones affected by the decisions. Decision making also deals with establishing and browsing a vast number of alternatives and weighing options according to one’s preferences. Decision support systems can offer help in this process. However, current systems are built on economic models and less suited for untrained decision makers. Therefore, this dissertation focuses on designing decision support systems from a human-centered perspective empowering people to take decisions. Investigations were two-fold, i.e. focusing on requirements and concrete design guidelines and on the methods for engaging of stakeholders in the design process. Requirements were derived from interdisciplinary literature research and exploratory studies with domain experts and users. These highlighted the crucial preparation phase of decision making and the social factors of the process. Design guidelines for the overall system, and in particular preference construction and value-reflection, were derived through design-based research involving experts and users. A dominant theme was the delicate balance between supporting human ways of thinking and reflecting and giving intelligent guidance created by system designers. This balance can only be achieved through close, iterative interactions with end-users, domain experts and designers throughout the design process supported by skilled facilitators. This thesis marks a shift in DSS research from engineering expert systems taking over decision making to designing human-centered support for people to make their own, informed decisions.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc
Sensing what matters
No full textRecently, the Royal Netherlands Navy (RNLN) is executing missions in coastal regions with a lot of civil traffic. Furthermore, the opponent of a typical modern mission is not as apparent as was the case during e.g., the Cold War. In the direct vicinity of naval vessels are many objects and it is increasingly complex to identify which of those objects pose a threat. This is the main reason for the need of decision support and automation aboard RNLN’s ships. This thesis introduces a new classification methodology that is suited for the military application domain. This methodology is based on fitting the incoming sensor information on predefined situation knowledge inserted by the operator. To verify the performance new evaluation criteria are introduced that are suited for the characteristics of the application domain. Multiple classifiers result from this new methodology and their results are combined using Dezert-Smarandache Theory. The performance gain of this new approach is shown in a simulation and using existing and new evaluation criteria compared to other known classifiers. The system introduced in this thesis additional has advantages in terms of user interaction. Furthermore, this new system enables the automation of describing the information requirements for classification. This in turn enables the automation of sensor management processes. Finally, this thesis argues that it is essential to integrate existing sensor performance programs in order to automate sensor management.Man-Machine Interaction GroupElectrical Engineering, Mathematics and Computer Scienc
What to bid and when to stop
No full textNegotiation is an important activity in human society, and is studied by various disciplines, ranging from economics and game theory, to electronic commerce, social psychology, and artificial intelligence. Traditionally, negotiation is a necessary, but also time-consuming and expensive activity. Therefore, in the last decades there has been a large interest in the automation of negotiation, for example in the setting of e-commerce. This interest is fueled by the promise of automated agents eventually being able to negotiate on behalf of human negotiators.Every year, automated negotiation agents are improving in various ways, and there is now a large body of negotiation strategies available, all with their unique strengths and weaknesses. For example, some agents are able to predict the opponent's preferences very well, while others focus more on having a sophisticated bidding strategy. The problem however, is that there is little incremental improvement in agent design, as the agents are tested in varying negotiation settings, using a diverse set of performance measures. This makes it very difficult to meaningfully compare the agents, let alone their underlying techniques. As a result, we lack a reliable way to pinpoint the most effective components in a negotiating agent.There are two major advantages of distinguishing between the different components of a negotiating agent's strategy: first, it allows the study of the behavior and performance of the components in isolation. For example, it becomes possible to compare the preference learning component of all agents, and to identify the best among them. Second, we can proceed to mix and match different components to create new negotiation strategies., e.g.: replacing the preference learning technique of an agent and then examining whether this makes a difference. Such a procedure enables us to combine the individual components to systematically explore the space of possible negotiation strategies.To develop a compositional approach to evaluate and combine the components, we identify structure in most agent designs by introducing the BOA architecture, in which we can develop and integrate the different components of a negotiating agent. We identify three main components of a general negotiation strategy; namely a bidding strategy (B), possibly an opponent model (O), and an acceptance strategy (A). The bidding strategy considers what concessions it deems appropriate given its own preferences, and takes the opponent into account by using an opponent model. The acceptance strategy decides whether offers proposed by the opponent should be accepted.The BOA architecture is integrated into a generic negotiation environment called Genius, which is a software environment for designing and evaluating negotiation strategies. To explore the negotiation strategy space of the negotiation research community, we amend the Genius repository with various existing agents and scenarios from literature. Additionally, we organize a yearly international negotiation competition (ANAC) to harvest even more strategies and scenarios. ANAC also acts as an evaluation tool for negotiation strategies, and encourages the design of negotiation strategies and scenarios.We re-implement agents from literature and ANAC and decouple them to fit into the BOA architecture without introducing any changes in their behavior. For each of the three components, we manage to find and analyze the best ones for specific cases, as described below. We show that the BOA framework leads to significant improvements in agent design by wining ANAC 2013, which had 19 participating teams from 8 international institutions, with an agent that is designed using the BOA framework and is informed by a preliminary analysis of the different components.In every negotiation, one of the negotiating parties must accept an offer to reach an agreement. Therefore, it is important that a negotiator employs a proficient mechanism to decide under which conditions to accept. When contemplating whether to accept an offer, the agent is faced with the acceptance dilemma: accepting the offer may be suboptimal, as better offers may still be presented before time runs out. On the other hand, accepting too late may prevent an agreement from being reached, resulting in a break off with no gain for either party. We classify and compare state-of-the-art generic acceptance conditions. We propose new acceptance strategies and we demonstrate that they outperform the other conditions. We also provide insight into why some conditions work better than others and investigate correlations between the properties of the negotiation scenario and the efficacy of acceptance conditions.Later, we adopt a more principled approach by applying optimal stopping theory to calculate the optimal decision on the acceptance of an offer. We approach the decision of whether to accept as a sequential decision problem, by modeling the bids received as a stochastic process. We determine the optimal acceptance policies for particular opponent classes and we present an approach to estimate the expected range of offers when the type of opponent is unknown. We show that the proposed approach is able to find the optimal time to accept, and improves upon all existing acceptance strategies.Another principal component of a negotiating agent's strategy is its ability to take the opponent's preferences into account. The quality of an opponent model can be measured in two different ways. One is to use the agent's performance as a benchmark for the model's quality. We evaluate and compare the performance of a selection of state-of-the-art opponent modeling techniques in negotiation. We provide an overview of the factors influencing the quality of a model and we analyze how the performance of opponent models depends on the negotiation setting. We identify a class of simple and surprisingly effective opponent modeling techniques that did not receive much previous attention in literature.The other way to measure the quality of an opponent model is to directly evaluate its accuracy by using similarity measures. We review all methods to measure the accuracy of an opponent model and we then analyze how changes in accuracy translate into performance differences. Moreover, we pinpoint the best predictors for good performance. This leads to new insights concerning how to construct an opponent model, and what we need to measure when optimizing performance.Finally, we take two different approaches to gain more insight into effective bidding strategies. We present a new classification method for negotiation strategies, based on their pattern of concession making against different kinds of opponents. We apply this technique to classify some well-known negotiating strategies, and we formulate guidelines on how agents should bid in order to be successful, which gives insight into the bidding strategy space of negotiating agents. Furthermore, we apply optimal stopping theory again, this time to find the concessions that maximize utility for the bidder against particular opponents. We show there is an interesting connection between optimal bidding and optimal acceptance strategies, in the sense that they are mirrored versions of each other.Lastly, after analyzing all components separately, we put the pieces back together again. We take all BOA components accumulated so far, including the best ones, and combine them all together to explore the space of negotiation strategies.We compute the contribution of each component to the overall negotiation result, and we study the interaction between components. We find that combining the best agent components indeed makes the strongest agents. This shows that the component-based view of the BOA architecture not only provides a useful basis for developing negotiating agents but also provides a useful analytical tool. By varying the BOA components we are able to demonstrate the contribution of each component to the negotiation result, and thus analyze the significance of each. The bidding strategy is by far the most important to consider, followed by the acceptance conditions and finally followed by the opponent model.Our results validate the analytical approach of the BOA framework to first optimize the individual components, and then to recombine them into a negotiating agent
Designing Generic and Efficient Negotiation Strategies
No full textThe central aim of this thesis is the design of generic and efficient automated strategies for two-party negotiations in which negotiating parties do not reveal their preferences explicitly. A strategy for negotiation is the decision mechanism for determining the actions of a negotiator. Generic refers to the idea that the strategy needs no forehand knowledge about the opponent or the domain of negotiation. A strategy thus should be generic in the sense that it can be successfully applied to any negotiation domain and fine-tuned to domainspecific features to produce even better results. Efficiency refers to the fact that the strategy should be able to negotiate effectively against another automated agent or human negotiator and obtain an outcome that cannot be improved for both parties. The design of the negotiating strategy that is proposed in this thesis is based on analyses of the state-of- the-art negotiation strategies using an analytical method that is also proposed in this work. The method significantly extends existing negotiation benchmarks by analysing dynamic properties of a negotiation strategy. One of the main findings of the analysis, in line with the management and social science literature on negotiation [20, 23], is that the strategy should learn the opponent’s preferences in order to increase the negotiation efficiency. We applied our results in learning the opponents’ profiles in a one-to-many negotiation setting. We additionally addressed the problem of issue-dependencies. Issue dependencies form an insurmountable barrier for the state of the art negotiation strategies [9]. Therefore, we developed an approximation method to eliminate dependencies. This part of the research seems a side track, however it was fundamental that we address this problem to prove the scalability and applicability of our research results.MediamaticsElectrical Engineering, Mathematics and Computer Scienc
Towards Robust Visual Speech Recognition: Automatic Systems for Lip Reading of Dutch
No full textIn the last two decades we witnessed a rapid increase of the computational power governed by Moore's Law. As a side effect, the affordability of cheaper and faster CPUs increased as well. Therefore, many new “smart” devices flooded the market and made informational systems widely spread. The number of users of information systems has also increased many folds, and the user's characteristics have changed to include not only a small number of initiates but also a majority of non technical people. To make this transition possible systems' developers had to change the computer user interfaces in order to make it simpler and more intuitive. However, the interaction was still based on rather artificial devices such as mouse and keyboard. Since the Moore's Law continues to work over and over again we came to a critical moment when the current systems can easily cope with other input streams such as video and audio, to name the most important, and others. We can, therefore, envision systems with which we can communicate through speech and body movements and that can automatically and transparently adapt to the environment and user. This can be done for instance by recognizing the user affective state, by understanding the task of the user and recognizing the context of the interaction. Automatic speech recognition by capturing and processing the audio signal is one development in this direction. Even though in controlled settings automatic speech recognition has achieved spectacular results, its performance is still dependent on the context (for instance on the level of the background noise). Automatic lip reading has appeared in this context as a way to enhance automatic speech recognition in noisy environments. Even though it is still a relatively novel research domain, other applications were found which employ lip reading as stand alone: interfaces for hearing impaired persons, security applications, speech recovery from mute of deteriorated films, silence interfaces. With the advances in visual signal processing the research in lip reading was also revitalized. However, at the moment of writing of this thesis lip reading was still waiting for its great leap. This thesis investigates several techniques for directing lip reading towards more robust performances. The thesis starts by introducing the relevant methodologies that govern automatic lip reading. Next it introduces all the concepts needed to understand the technologies, experiments, results and discussions presented later on. It is, therefore, one of the most important parts of the thesis. The presentation of the state of the art in lip reading is setting the starting point of the research presented. Before, continuing to follow the lip reading process the thesis introduces the mathematical foundations that give the theoretical support for the analysis. All our systems are based on the Hidden Markov Models approach. This paradigm has proved to be very useful in similar problems and we successfully employed it for lip reading. The main idea behind it is the Bayesian rule which says that starting from some a-priori knowledge we can always improve our understanding of a system through observation. Observation translates into processing the video stream in a set of features that describe what is being said by the speaker. However, in order to appropriately train lip reading systems, a large amount of data is needed. The first important contribution of our research is a large data corpus for the Dutch language. This corpus, named “New Delft University of Technology Audio Visual Speech Corpus”, is at the date of writing this thesis one of the largest corpora for lip reading in Dutch. The corpus contains dual view high speed recordings (i.e. 100Hz) of continuous speech in Dutch. During the building of the corpus, we also produced an incipient set of guidelines for building a data corpus for lip reading which we hope to be useful for other researchers. However, the core of this thesis consists in the data parametrization. Data parametrization is the process that transforms the input video data in a set of features that are used later on for training and testing the resulting recognizer. The parametrization should reduce the size of the input data while preserving the most important information related with what the speaker says. We investigated three data parametrization techniques each coming from a different category of algorithms. We used Active Appearance Models which generate a combined geometric and appearance based set of features, we used optical flow analysis which is an appearance based approach that directly accounts for the apparent movement on the speaker's face and we used a statistical approach which generates the geometry of lips without starting from an a-priori fixed model. During the research presented in this thesis we investigated the performances of these data parametrization techniques and we pointed out their strengths and weaknesses. We also analysed the performance of lip reading starting from other points of view. We analysed the influence of the sampling rate of the video data, the performance of the lip readers as a function of the recognition task but also the performance as a function of the size of the corpus used. Answering to all these questions improved our understanding of the limitations and the possible improvements of lip reading.MediamaticsElectrical Engineering, Mathematics and Computer Scienc
- …
