35 research outputs found
Synergy of Lip-Motion and Acoustic Features in Biometric Speech and Speaker Recognition
No full textAbstract—This paper presents the scheme and evaluation of a robust audio-visual digit-and-speaker-recognition system using lip motion and speech biometrics. Moreover, a liveness verification barrier based on a person’s lip movement is added to the system to guard against advanced spoofing attempts such as replayed videos. The acoustic and visual features are integrated at the feature level and evaluated first by a Support Vector Machine for digit and speaker identification and, then, by a Gaussian Mixture Model for speaker verification. Based on 300 different personal identities, this paper represents, to our knowledge, the first extensive study investigating the added value of lip motion features for speaker and speech-recognition applications. Digit recognition and person-identification and verification experiments are conducted on the publicly available XM2VTS database showing favorable results (speaker verification is 98 percent, speaker identification is 100 percent, and digit identification is 83 percent to 100 percent)
Lip-motion and speech biometrics in person recognition
No full textBiometric identification techniques are frequently used to improve security, e.g. in financial transactions, computer networks and secure critical locations. The purpose of biometric authentication systems is to verify an individual by her biological characteristics including those generating characterisitic behaviour. It is not only fingerprints that are used for authentication; our lips, eyes, speech, signatures and even facial temperature are now being used to identify us. This presumably increases security since these traits are harder to copy, steal or lose.This thesis attempts to present an effective scheme to extract descriminative features based on a novel motion estimation algorithm for lip movement. Motion is defined as the distribution of apparent velocities in the changes of brightness patterns in an image. The velocity components of a lip sequence are computed by the well-known 3D structure tensor using 1D processing, in 2D manifolds. Since the velocities are computed without extracting the speaker's lip contours, more robust visual features can be obtained. The velocity estimation is performed in rectangular lip regions, which affords increased computational efficiency.To investigate the usefulness of the proposed motion features we implement a person authentication system based on lip movements information with (and without) speech information. It yields a speaker verification rate of 98% with lip and speech information. Comparisons are made with an alternative motion estimation technique and a description of our proposed feature fusion technique is given. Beside its value in authentication, the technique can be used naturally to evaluate the liveness i.e. to determine if the biometric data is be captured from a legitimate user, live user who is physically present at the point of acquisition, of a speaking person as it can be used in a text-prompted dialog.</p
Person verification by lip-motion
No full textThis paper describes a new motion based feature extraction technique for speaker recognition using orientation estimation in 2D manifolds. The motion is estimated by computing the components of the structure tensor from which normal flows are extracted. By projecting the 3D spatiotemporal data to 2-D planes we obtain projection coefficients which we use to evaluate the 3-D orientations of brightness patterns in TV like 2D image sequences. This corresponds to the solutions of simple matrix eigenvalue problems in 2D, affording increased computational efficiency. An implementation based on joint lip movements and speech is presented along with experiments which confirm the theory, exhibiting a recognition rate of 98% on the publicly available XM2VTS database.©2006 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.</p
Audio–visual person authentication using lip-motion from orientation maps
No full textThis paper describes a new identity authentication technique by a synergetic use of lip-motion and speech. The lip-motion is defined as the distribution of apparent velocities in the movement of brightness patterns in an image and is estimated by computing the velocity components of the structure tensor by 1D processing, in 2D manifolds. Since the velocities are computed without extracting the speaker’s lip-contours, more robust visual features can be obtained in comparison to motion features extracted from lip-contours. The motion estimations are performed in a rectangular lip-region, which affords increased computational efficiency. A person authentication implementation based on lip-movements and speech is presented along with experiments exhibiting a recognition rate of 98%. Besides its value in authentication, the technique can be used naturally to evaluate the “liveness” of someone speaking as it can be used in text-prompted dialogue. The XM2VTS database was used for performance quantification as it is currently the largest publicly available database (≈300 persons) containing both lip-motion and speech. Comparisons with other techniques are presented.</p
Lip motion features for biometric person recognition
No full textThe present chapter reports on the use of lip motion as a stand alone biometric modality as well as a modality integrated with audio speech for identity recognition using digit recognition as a support. First, the auhtors estimate motion vectors from images of lip movements. The motion is modeled as the distribution of apparent line velocities in the movement of brightness patterns in an image. Then, they construct compact lip-motion features from the regional statistics of the local velocities. These can be used as alone or merged with audio features to recognize identity or the uttered digit. The author's present person recognition results using the XM2VTS database representing the video and audio data of 295 people. Furthermore, we present results on digit recognition when it is used in a text prompted mode to verify the liveness of the user. Such user challenges have the intention to reduce replay attack risks of the audio system
Lip Biometrics for Digit Recognition
No full textThis paper presents a speaker-independent audio-visual digit recognition system that utilizes speech and visual lip signals. The extracted visual features are based on line-motion estimation obtained from video sequences with low resolution (128 ×128 pixels) to increase the robustness of audio recognition. The core experiments investigate lip motion biometrics as stand-alone as well as merged modality in speech recognition system. It uses Support Vector Machines, showing favourable experimental results with digit recognition featuring 83% to 100% on the XM2VTS database depending on the amount of available visual information.</p
Speaker and Digit Recognition by Audio-Visual Lip Biometrics
No full textThis paper proposes a new robust bi-modal audio visual digit and speaker recognition system by lip-motion and speech biometrics. To increase the robustness of digit and speaker recognition, we have proposed a method using speaker lip motion information extracted from video sequences with low resolution (128 ×128 pixels). In this paper we investigate a biometric system for digit recognition and speaker identification based using line-motion estimation with speech information and Support Vector Machines. The acoustic and visual features are fused at the feature level showing favourable results with digit recognition being 83% to 100% and speaker recognition 100% on the XM2VTS database.</p
