84 research outputs found
Computers to help with conversations : affective framework to enhance human nonverbal skills
No full textThesis: Ph. D., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (pages 179-199).Nonverbal behavior plays an integral part in a majority of social interaction scenarios. Being able to adjust nonverbal behavior and influence other's responses are considered valuable social skills. A deficiency in nonverbal behavior can have detrimental consequences in personal as well as in professional life. Many people desire help, but due to limited resources, logistics, and social stigma, they are unable to get the training that they require. Therefore, there is a need for developing automated interventions to enhance human nonverbal behaviors that are standardized, objective, repeatable, low-cost, and can be deployed outside of the clinic. In this thesis, I design and validate a computational framework designed to enhance human nonverbal behavior. As part of the framework, I developed My Automated Conversation coacH (MACH)-a novel system that provides ubiquitous access to social skills training. The system includes a virtual agent that reads facial expressions, speech, and prosody, and responds with verbal and nonverbal behaviors in real-time. As part of explorations on nonverbal behavior sensing, I present results on understanding the underlying meaning behind smiles elicited under frustration, delight or politeness. I demonstrate that it is useful to model the dynamic properties of smiles that evolve through time and that while a smile may occur in positive and in negative situations, its underlying temporal structures may help to disambiguate the underlying state, in some cases, better than humans. I demonstrate how the new insights and developed technology from this thesis became part of a real-time system that is able to provide visual feedback to the participants on their nonverbal behavior. In particular, the system is able to provide summary feedback on smile tracks, pauses, speaking rate, fillers and intonation. It is also able to provide focused feedback on volume modulation and enunciation, head gestures, and smiles for the entire interaction. Users are able to practice as many times as they wish and compare their data across sessions. I validate the MACH framework in the context of job interviews with 90 MIT undergraduate students. The findings indicate that MIT students using MACH are perceived as stronger candidates compared to the students in the control group. The results were reported based on the judgments of the independent MIT career counselors and Mechanical Turkers', who did not participate in the study, and were blind to the study conditions. Findings from this thesis could motivate further interaction possibilities of helping people with public speaking, social-communicative difficulties, language learning, dating and more..by Mohammed Ehsan Hoque.Ph. D
Rethinking commercial complex , Shapla Chottor, Motijheel: a vision to create a flagship project that would demonstrate the economic and social value of building high density structures at major transport nodes
No full textThis thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Architecture in Architecture, 2014.Cataloged from PDF version of thesis.Includes bibliographical references (page 72).The professional community that deals with urban issues, planning and development studies are thinking seriously about the urban issues in Dhaka city. They all think interventionist and revivalist projects on a massive scale in various aspects is required to rescue Dhaka City from its race to self-destruction. This paper will try to establish a form of invention, in typological, morphological and functional dimensions. The basic idea is to integrate the development projects in specific urban areas and merge it with city scale. These development projects can create a connected edifice of the services in the city. The idea came up with very basic analytical functions of what, why, how, where and when. Dhaka city need a sustainable design solution. Rather creating more new projects or function we need to rearrange all these functions. All urban services should be integrated and blend properly to remove the city chaos. These ideas are implemented in the design proposal for the Motijheel CBD area. Which will detailed to illustrate for an integrated development. Also it will develop guidelines, for future developments.Anika HoqueB. Architectur
Predicting video-conferencing conversation outcomes based on modeling facial expression synchronization
No full textEffective video-conferencing conversations are heavily influenced by each speaker's facial expression. In this study, we propose a novel probabilistic model to represent interactional synchrony of conversation partners' facial expressions in video-conferencing communication. In particular, we use a hidden Markov model (HMM) to capture temporal properties of each speaker's facial expression sequence. Based on the assumption of mutual influence between conversation partners, we couple their HMMs as two interacting processes. Furthermore, we summarize the multiple coupled HMMs with a stochastic process prior to discover a set of facial synchronization templates shared among the multiple conversation pairs. We validate the model, by utilizing the exhibition of these facial synchronization templates to predict the outcomes of video-conferencing conversations. The dataset includes 75 video-conferencing conversations from 150 Amazon Mechanical Turkers in the context of a new recruit negotiation. The results show that our proposed model achieves higher accuracy in predicting negotiation winners than support vector machine and canonical HMMs. Further analysis indicates that some synchronized nonverbal templates contribute more in predicting the negotiation outcomes
Mood meter: counting smiles in the wild
No full textIn this study, we created and evaluated a computer vision based system that automatically encouraged, recognized and counted smiles on a college campus. During a ten-week installation, passersby were able to interact with the system at four public locations. The aggregated data was displayed in real time in various intuitive and interactive formats on a public website. We found privacy to be one of the main design constraints, and transparency to be the best strategy to gain participants' acceptance. In a survey (with 300 responses), participants reported that the system made them smile more than they expected, and it made them and others around them feel momentarily better. Quantitative analysis of the interactions revealed periodic patterns (e.g., more smiles during the weekends) and strong correlation with campus events (e.g., fewer smiles during exams, most smiles the day after graduation), reflecting the emotional responses of a large community.Massachusetts Institute of Technology. Council for the ArtsCaja Madrid (Fellowship)Massachusetts Institute of Technology (Festival of Art, Science, and Technology (FAST)
MACH: My Automated Conversation coacH
No full textMACH--My Automated Conversation coacH--is a novel system that provides ubiquitous access to social skills training. The system includes a virtual agent that reads facial expressions, speech, and prosody and responds with verbal and nonverbal behaviors in real time. This paper presents an application of MACH in the context of training for job interviews. During the training, MACH asks interview questions, automatically mimics certain behavior issued by the user, and exhibit appropriate nonverbal behaviors. Following the interaction, MACH provides visual feedback on the user's performance. The development of this application draws on data from 28 interview sessions, involving employment-seeking students and career counselors. The effectiveness of MACH was assessed through a weeklong trial with 90 MIT undergraduates. Students who interacted with MACH were rated by human experts to have improved in overall interview performance, while the ratings of students in control groups did not improve. Post-experiment interviews indicate that participants found the interview experience informative about their behaviors and expressed interest in using MACH in the future.Samsung (Firm)MIT Media Lab Consortiu
Modeling and mediating conversational norm violations
No full textThesis (Ph. D.)--University of Rochester. Department of Computer Science, 2021.This thesis focuses on improving human-human interaction during group discussions through
feedback from effective human-machine interaction. Identifying verbal and non-verbal attributes
and allowing people to be aware of them is crucial for maintaining a safe and effective exchange of
ideas. This thesis work explores capturing the related behavioral and affective attributes from audio,
video, and language information from videoconferencing-based group meetings, and developing
automated feedback systems (chatbot, conversational agents, visualization) for mediation purposes.
First, I present a multi-modal dataset for interpersonal disrespect or toxicity from 59 YouTube
News Show dyadic remote discussion videos, and propose an algorithm to define a speaker-wise
toxicity score. Our models perform with an accuracy of over 60% using visual features and close
to 80% on audial features to recognize the attributes of disrespect. Next, I elaborate on developing automated systems to capture multi-modal features from meetings and designing privacy-preserving feedback. The work handles behavioral and contextual features such as talk-time, turn-taking, interruption, volume, sentiment, valence, attitude, shared smile, attention, anger, surprise, engagement, questions, consensus. For keeping heated discussions respectful, we develop a videoconferencing platform integrated with real-time feedback processed on the client-side. Validated by 40 participants, our findings reveal that real-time feedback can reduce expressiveness during the discussion, yet improves the follow-up discussion even without feedback. For post-meeting reflection, we develop a fully automated collaboration platform ‘CoCo’ that is capable of holding video conferencing meetings, processing data post-session in bulk on the server-side, and presenting feedback through an interactive chatbot. Evaluation from 39 participants shows the improvement in group dynamics in successive discussions. We also explore post-session feedback in an in situ workplace setting. We survey the challenges faced in remote meetings (N = 150), and as per the needs design and evaluate a wireframe prototype (N = 16) and an interactive feedback dashboard named ‘MeetingCoach’ (N = 23). The study supports our hypotheses that actionable suggestions, personalized modeling, and privacy-preserving feedback can potentially improve meeting effectiveness and inclusivity. For pre-meeting training, we present a suggestive chatbot incorporated with motivational interviewing (MI) technique for improving conversational skills. Evaluation from a consensus-based résumés evaluation study with 21 participants showcases the effectiveness of the suggestive MI chatbot in encouraging users to apply the information delivered by agents. We highlight strategies on how to reach a consensus that fulfills individual and team goals. We explore improving agent capabilities in terms of empathy and affect. We design the dialogue of an empathetic conversational agent and evaluate it in a Wizard-of-Oz-based study with
34 participants. Our results show improved human-machine interaction mitigating negative affect. We also explore sentiment detection on a human-machine interaction dataset. We build and compare multimodal LSTM fusion (accuracynothreshold = 67.5%, accuracythreshold = 71.8%) and hierarchical (accuracynothreshold = 60.9%, accuracythreshold = 71.8%) models. The results show the importance of increasing agent capabilities in becoming more affective and interactive to effectively interact with users. Overall, the findings of this thesis work provide useful information to the research community regarding modeling conversations to understand group dynamics and mediating discussions through effective feedback agents
A framework to provide neurological screening by integrating multiple task modalities
No full textThesis (Ph. D.)--University of Rochester. Department of Computer Science, 2023.In this thesis, we tackle four research questions related to two neurological disorders: Parkinson’s Disease (PD) and Ataxia. The first three questions focus on screening for PD using a speech task, measuring PD severity using the finger-tapping task, and predicting Ataxia severity using the gait task. And finally, we build and validate a neurological framework to communicate the output of the AI models in the neurological health space in an easy and intuitive manner. To screen for PD using speech, we collected data from 726 unique participants (262 PD, 464 non-PD) uttering “The quick brown fox jumps over the lazy dog...”. We extracted both standard acoustic features (MFCC, jitter, and shimmer variants), and domain-specific dysphonia features to build ML models that achieved 0.753 AUC in screening PD. We also showed that the model focuses on MFCC and dysphonia features (through SHAP analysis) and performs well for different gender and age groups. To measure PD severity using finger-tapping, we collected 250 videos from unique participants rated by three expert neurologists and an MDS-UPDRS clinician. We developed algorithms to obtain motor-function measurements and trained an ML model that outperformed the MDS-UPDRS certified rater – with a mean absolute error (MAE) of 0.59 compared to the rater’s MAE of 0.79. However, the model performed slightly worse than the expert neurologists (0.53 MAE). Then, we assessed Ataxia by 1) collecting 155 neurologist-rated gait videos from 89 unique participants (24 controls and 65 diagnosed with Ataxia), 2) detecting, tracking, and separating the participants from their surroundings, and constructing several features from their body pose coordinates to capture gait characteristics such as step width, step length, swing, stability, speed, etc., 3) building ML models for both Ataxia risk-prediction and severity-assessment. Our models achieve 83.06% accuracy in Ataxia risk-prediction and 0.6225 MAE and 0.7268 PCC in severity-assessment. Besides, they perform competitively when evaluated on data from medical sites not used during training. In the end, we present a user-centric validation of a teleneurology platform, assessing its effectiveness in conveying screening information, facilitating user queries, and offering resources to enhance user empowerment – with PD as a case study. Our validation method demonstrates to users a mock PD risk assessment and provides access to relevant resources, including a chatbot driven by GPT, locations of local neurologists, and actionable and scientifically-backed PD prevention and management recommendations. We share findings from 101 participants (50 with PD, 51 Non-PD) and show that our framework was rated positively by 80.49% (standard deviation ±8.93%) of the participants, and achieved an above-average 71.25 median System-Usability-Scale (SUS) score
- …
