1,720,983 research outputs found
mitenand DSGS/Ensemble LSF/Insieme LIS
No full textThis dataset is part of our research paper at LREC-COLING 2024:
Jiang, Zifan, Göhring, Anne, Moryossef, Amit, Sennrich, Rico, Ebling, Sarah. 2024. SwissSLi: The Multi-parallel Sign Language Corpus for Switzerland. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Online: https://aclanthology.org/2024.lrec-main.1342
Our study contains the sign language version of two sets of television programs in three spoken/sign languages released between 2020 and 2023 fall. They were created by the Swiss Broadcasting Corporation (SRG) in spoken language and then translated into sign language.
This dataset contains mitenand (German/DSGS), Ensemble (French/LSF-CH), and Insieme (Italian/LIS-CH) which report the work of charity organizations, are parallel and produced weekly. Most episodes originate from German and are then translated into other languages.
mitenand in Gebärdensprache:
https://www.srf.ch/play/tv/sendung/mitenand-in-gebaerdensprache?id=1d478b33-be72-4258-a330-c8c15bc1f6aa
Ensemble - signé:
https://www.rts.ch/play/tv/emission/ensemble-signe?id=12546856
Insieme LIS:
https://www.rsi.ch/play/tv/programma/insieme-lis?id=70397
Helveticus DSGS/LSF/LIS
No full textThis dataset is part of our research paper at LREC-COLING 2024:
Jiang, Zifan, Göhring, Anne, Moryossef, Amit, Sennrich, Rico, Ebling, Sarah. 2024. SwissSLi: The Multi-parallel Sign Language Corpus for Switzerland. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Online: https://aclanthology.org/2024.lrec-main.1342
Our study contains the sign language version of two sets of television programs in three spoken/sign languages released between 2020 and 2023 fall. They were created by the Swiss Broadcasting Corporation (SRG) in spoken language and then translated into sign language.
This dataset contains Helveticus in three languages which covers historical content with animations and is produced irregularly. Most episodes originate from German and are then translated into other languages.
Helveticus in Gebärdensprache:
https://www.srf.ch/play/tv/sendung/helveticus-in-gebaerdensprache?id=46863500-56d8-4414-a3d9-e84a6a843287
Helveticus signé:
https://www.rts.ch/play/tv/emission/helveticus-signe?id=10953398
Helveticus LIS:
https://www.rsi.ch/play/tv/programma/helveticus-lis?id=70391
sign.mt: Real-Time Multilingual Sign Language Translation Application
Get PDFThis demo paper presents sign.mt, an open-source application pioneering real-time multilingual bi-directional translation between spoken and signed languages. Harnessing state-of-the-art open-source models, this tool aims to address the communication divide between the hearing and the deaf, facilitating seamless translation in both spoken-to-signed and signed-to-spoken translation directions.
Promising reliable and unrestricted communication, sign.mt offers offline functionality, crucial in areas with limited internet connectivity. It further enhances user engagement by offering customizable photo-realistic sign language avatars, thereby encouraging a more personalized and authentic user experience.
Licensed under CC BY-NC-SA 4.0, sign.mt signifies an important stride towards open, inclusive communication. The app can be used, and modified for personal and academic uses, and even supports a translation API, fostering integration into a wider range of applications. However, it is by no means a finished product.
We invite the NLP community to contribute towards the evolution of sign.mt. Whether it be the integration of more refined models, the development of innovative pipelines, or user experience improvements, your contributions can propel this project to new heights. Available at https://sign.mt, it stands as a testament to what we can achieve together, as we strive to make communication accessible to all
Addressing the Blind Spots in Spoken Language Processing
Get PDFThis paper explores the critical but often overlooked role of non-verbal cues, including co-speech gestures and facial expressions, in human communication and their implications for Natural Language Processing (NLP). We argue that understanding human communication requires a more holistic approach that goes beyond textual or spoken words to include non-verbal elements. Borrowing from advances in sign language processing, we propose the development of universal automatic gesture segmentation and transcription models to transcribe these non-verbal cues into textual form. Such a methodology aims to bridge the blind spots in spoken language understanding, enhancing the scope and applicability of NLP models. Through motivating examples, we demonstrate the limitations of relying solely on text-based models. We propose a computationally efficient and flexible approach for incorporating non-verbal cues, which can seamlessly integrate with existing NLP pipelines. We conclude by calling upon the research community to contribute to the development of universal transcription methods and to validate their effectiveness in capturing the complexities of real-world, multi-modal interactions
Optimizing Hand Region Detection in MediaPipe Holistic Full-Body Pose Estimation to Improve Accuracy and Avoid Downstream Errors
Get PDFThis paper addresses a critical flaw in MediaPipe Holistic's hand Region of
Interest (ROI) prediction, which struggles with non-ideal hand orientations,
affecting sign language recognition accuracy. We propose a data-driven approach
to enhance ROI estimation, leveraging an enriched feature set including
additional hand keypoints and the z-dimension. Our results demonstrate better
estimates, with higher Intersection-over-Union compared to the current method.
Our code and optimizations are available at
https://github.com/sign-language-processing/mediapipe-hand-crop-fix
Addressing the Blind Spots in Spoken Language Processing
Get PDFThis paper explores the critical but often overlooked role of non-verbal
cues, including co-speech gestures and facial expressions, in human
communication and their implications for Natural Language Processing (NLP). We
argue that understanding human communication requires a more holistic approach
that goes beyond textual or spoken words to include non-verbal elements.
Borrowing from advances in sign language processing, we propose the development
of universal automatic gesture segmentation and transcription models to
transcribe these non-verbal cues into textual form. Such a methodology aims to
bridge the blind spots in spoken language understanding, enhancing the scope
and applicability of NLP models. Through motivating examples, we demonstrate
the limitations of relying solely on text-based models. We propose a
computationally efficient and flexible approach for incorporating non-verbal
cues, which can seamlessly integrate with existing NLP pipelines. We conclude
by calling upon the research community to contribute to the development of
universal transcription methods and to validate their effectiveness in
capturing the complexities of real-world, multi-modal interactions.Comment: 5 page
Real-Time Multilingual Sign Language Processing
No full textSign Language Processing (SLP) is an interdisciplinary field comprised of Natural Language Processing (NLP) and Computer Vision. It is focused on the computational understanding, translation, and production of signed languages. Traditional approaches have often been constrained by the use of gloss-based systems that are both language-specific and inadequate for capturing the multidimensional nature of sign language. These limitations have hindered the development of technology capable of processing signed languages effectively.
This thesis aims to revolutionize the field of SLP by proposing a simple paradigm that can bridge this existing technological gap. We propose the use of SignWiring, a universal sign language transcription notation system, to serve as an intermediary link between the visual-gestural modality of signed languages and text-based linguistic representations.
We contribute foundational libraries and resources to the SLP community, thereby setting the stage for a more in-depth exploration of the tasks of sign language translation and production. These tasks encompass the translation of sign language from video to spoken language text and vice versa. Through empirical evaluations, we establish the efficacy of our transcription method as a pivot for enabling faster, more targeted research, that can lead to more natural and accurate translations across a range of languages.
The universal nature of our transcription-based paradigm also paves the way for real-time, multilingual applications in SLP, thereby offering a more inclusive and accessible approach to language technology. This is a significant step toward universal accessibility, enabling a wider reach of AI-driven language technologies to include the deaf and hard-of-hearing community.PhD Thesi
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
- …
