1,720,971 research outputs found
Exploring Temporal Patterns in Classifying Frustrated and Delighted Smiles
Full text linkWe create two experimental situations to elicit two affective states: frustration, and delight. In the first experiment, participants were asked to recall situations while expressing either delight or frustration, while the second experiment tried to elicit these states naturally through a frustrating experience and through a delightful video. There were two significant differences in the nature of the acted versus natural occurrences of expressions. First, the acted instances were much easier for the computer to classify. Second, in 90 percent of the acted cases, participants did not smile when frustrated, whereas in 90 percent of the natural cases, participants smiled during the frustrating interaction, despite self-reporting significant frustration with the experience. As a follow up study, we develop an automated system to distinguish between naturally occurring spontaneous smiles under frustrating and delightful stimuli by exploring their temporal patterns given video of both. We extracted local and global features related to human smile dynamics. Next, we evaluated and compared two variants of Support Vector Machine (SVM), Hidden Markov Models (HMM), and Hidden-state Conditional Random Fields (HCRF) for binary classification. While human classification of the smile videos under frustrating stimuli was below chance, an accuracy of 92 percent distinguishing smiles under frustrating and delighted stimuli was obtained using a dynamic SVM classifier.MIT Media Lab ConsortiumProcter & Gamble Compan
Improvements in Remote Cardiopulmonary Measurement Using a Five Band Digital Camera
No full textRemote measurement of the blood volume pulse via photoplethysmography (PPG) using digital cameras and ambient light has great potential for healthcare and affective computing. However, traditional RGB cameras have limited frequency resolution. We present results of PPG measurements from a novel five band camera and show that alternate frequency bands, in particular an orange band, allowed physiological measurements much more highly correlated with an FDA approved contact PPG sensor. In a study with participants (n = 10) at rest and under stress, correlations of over 0.92 (p <; 0.01) were obtained for heart rate, breathing rate, and heart rate variability measurements. In addition, the remotely measured heart rate variability spectrograms closely matched those from the contact approach. The best results were obtained using a combination of cyan, green, and orange (CGO) bands; incorporating red and blue channel observations did not improve performance. In short, RGB is not optimal for this problem: CGO is better. Incorporating alternative color channel sensors should not increase the cost of such cameras dramatically
Biophone: Physiology monitoring from peripheral smartphone motions
No full textThe large-scale adoption of smartphones during recent years has created many opportunities to improve health monitoring and care delivery. In this work, we demonstrate that motion sensors available in off-the-shelf smartphones can capture physiological parameters of a person during stationary postures, even while being carried in a bag or a pocket. In particular, we develop methods to extract heart and breathing rates from accelerometer data and compare them with measurements obtained with FDA-cleared sensors. We evaluated their accuracy on 12 people across different still body postures (pre- and post- exercise) and were able to reach mean absolute errors of 1.16 beats per minute (STD: 3) and 0.26 breaths per minute (STD: 0.5) when considering different conditions. Furthermore, we evaluated the same methods during regular phone activities, such as when watching a video or listening to a conversation, yielding increased but still comparable error rates for some conditions.National Science Foundation (U.S.) (NSF CCF-1029585)Samsung (Firm)MIT Media Lab Consortiu
Acume: A New Visualization Tool for Understanding Facial Expression and Gesture Data
No full textFacial and head actions contain significant affective information. To date, these actions have mostly been studied in isolation because the space of naturalistic combinations is vast. Interactive visualization tools could enable new explorations of dynamically changing combinations of actions as people interact with natural stimuli. This paper describes a new open-source tool that enables navigation of and interaction with dynamic face and gesture data across large groups of people, making it easy to see when multiple facial actions co-occur, and how these patterns compare and cluster across groups of participants. We share two case studies that demonstrate how the tool allows researchers to quickly view an entire corpus of data for single or multiple participants, stimuli and actions. Acume yielded patterns of actions across participants and across stimuli, and helped give insight into how our automated facial analysis methods could be better designed. The results of these case studies are used to demonstrate the efficacy of the tool. The open-source code is designed to directly address the needs of the face and gesture research community, while also being extensible and flexible for accommodating other kinds of behavioral data. Source code, application and documentation are available at http://affect.media.mit.edu/acume.Procter & Gamble Compan
Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera
No full textWe present a new method for measuring photoplethysmogram signals remotely using ambient light and a digital camera that allows for accurate recovery of the waveform morphology (from a distance of 3 m). In particular, we show that the peak-to-peak time between the systolic peak and diastolic peak/inflection can be automatically recovered using the second-order derivative of the remotely measured waveform. We compare measurements from the face with those captured using a contact fingertip sensor and show high agreement in peak and interval timings. Furthermore, we show that results can be significantly improved using orange, green, and cyan color channels compared to the tradition red, green, and blue channel combination. The absolute error in interbeat intervals was 26 ms and the absolute error in mean systolic-diastolic peak-to-peak times was 12 ms. The mean systolic-diastolic peak-to-peak times measured using the contact sensor and the camera were highly correlated, ρ = 0.94 (p <; 0.001). The results were obtained with a camera frame-rate of only 30 Hz. This technology has significant potential for advancing healthcare.MIT Media Member consortiumNihon Denki Kabushiki Kaisha (NEC fellowship
Predicting Ad Liking and Purchase Intent: Large-Scale Analysis of Facial Responses to Ads
No full textBillions of online video ads are viewed every month. We present a large-scale analysis of facial responses to video content measured over the Internet and their relationship to marketing effectiveness. We collected over 12,000 facial responses from 1,223 people to 170 ads from a range of markets and product categories. The facial responses were automatically coded frame-by-frame. Collection and coding of these 3.7 million frames would not have been feasible with traditional research methods. We show that detected expressions are sparse but that aggregate responses reveal rich emotion trajectories. By modeling the relationship between the facial responses and ad effectiveness, we show that ad liking can be predicted accurately (ROC AUC = 0.85) from webcam facial responses. Furthermore, the prediction of a change in purchase intent is possible (ROC AUC = 0.78). Ad liking is shown by eliciting expressions, particularly positive expressions. Driving purchase intent is more complex than just making viewers smile: peak positive responses that are immediately preceded by a brand appearance are more likely to be effective. The results presented here demonstrate a reliable and generalizable system for predicting ad effectiveness automatically from facial responses without a need to elicit self-report responses from the viewers. In addition we can gain insight into the structure of effective ads.MIT Media Lab ConsortiumNEC CorporationMAR
Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected In-the-Wild
Full text linkComputer classification of facial expressions requires large amounts of data and this data needs to reflect the diversity of conditions seen in real applications. Public datasets help accelerate the progress of research by providing researchers with a benchmark resource. We present a comprehensively labeled dataset of ecologically valid spontaneous facial responses recorded in natural settings over the Internet. To collect the data, online viewers watched one of three intentionally amusing Super Bowl commercials and were simultaneously filmed using their webcam. They answered three self-report questions about their experience. A subset of viewers additionally gave consent for their data to be shared publicly with other researchers. This subset consists of 242 facial videos (168,359 frames) recorded in real world conditions. The dataset is comprehensively labeled for the following: 1) frame-by-frame labels for the presence of 10 symmetrical FACS action units, 4 asymmetric (unilateral) FACS action units, 2 head movements, smile, general expressiveness, feature tracker fails and gender; 2) the location of 22 automatically detected landmark points; 3) self-report responses of familiarity with, liking of, and desire to watch again for the stimuli videos and 4) baseline performance of detection algorithms on this dataset. This data is available for distribution to researchers online, the EULA can be found at: http://www.affectiva.com/facial-expression-dataset-am-fed/
Non-contact, automated cardiac pulse measurements using video imaging and blind source separation
No full textRemote measurements of the cardiac pulse can provide
comfortable physiological assessment without electrodes. However,
attempts so far are non-automated, susceptible to motion artifacts and
typically expensive. In this paper, we introduce a new methodology that
overcomes these problems. This novel approach can be applied to color
video recordings of the human face and is based on automatic face tracking
along with blind source separation of the color channels into independent
components. Using Bland-Altman and correlation analysis, we compared
the cardiac pulse rate extracted from videos recorded by a basic webcam to
an FDA-approved finger blood volume pulse (BVP) sensor and achieved
high accuracy and correlation even in the presence of movement artifacts.
Furthermore, we applied this technique to perform heart rate measurements
from three participants simultaneously. This is the first demonstration of a
low-cost accurate video-based method for contact-free heart rate
measurements that is automated, motion-tolerant and capable of performing
concomitant measurements on more than one person at a time.Massachusetts Institute of Technology. Media LaboratoryThings That Think ConsortiumNancy Lurie Marks Family Foundatio
BioWatch: Estimation of Heart and Breathing Rates from Wrist Motions
Full text linkContinued developments of sensor technology including hardware miniaturization and increased sensitivity have enabled the development of less intrusive methods to monitor physiological parameters during daily life. In this work, we present methods to recover cardiac and respiratory parameters using accelerometer and gyroscope sensors on the wrist. We demonstrate accurate measurements in a controlled laboratory study where participants (n = 12) held three different positions (standing up, sitting down and lying down) under relaxed and aroused conditions. In particular, we show it is possible to achieve a mean absolute error of 1.27 beats per minute (STD: 3.37) for heart rate and 0.38 breaths per minute (STD: 1.19) for breathing rate when comparing performance with FDA-cleared sensors. Furthermore, we show comparable performance with a state-of-the-art wrist-worn heart rate monitor, and when monitoring heart rate of three individuals during two consecutive nights of in-situ sleep measurements.National Science Foundation (U.S.) (CCF-1029585)Samsung (Firm). Think Tank TeamMIT Media Lab Consortiu
Remote measurement of cognitive stress via heart rate variability
No full textRemote detection of cognitive load has many powerful applications, such as measuring stress in the workplace. Cognitive tasks have an impact on breathing and heart rate variability (HRV). We show that changes in physiological parameters during cognitive stress can be captured remotely (at a distance of 3m) using a digital camera. A study (n=10) was conducted with participants at rest and under cognitive stress. A novel five band digital camera was used to capture videos of the face of the participant. Significantly higher normalized low frequency HRV components and breathing rates were measured in the stress condition when compared to the rest condition. Heart rates were not significantly different between the two conditions. We built a person-independent classifier to predict cognitive stress based on the remotely detected physiological parameters (heart rate, breathing rate and heart rate variability). The accuracy of the model was 85% (35% greater than chance).MIT Media Lab Consortiu
- …
