38 research outputs found
LearnerVoice: A Dataset of Non-Native English Learners’ Spontaneous Speech
No full textPrevalent ungrammatical expressions and disfluencies in spontaneous speech from second language (L2) learners pose unique challenges to Automatic Speech Recognition (ASR) systems. However, few datasets are tailored to L2 learner speech. We publicly release LearnerVoice, a dataset consisting of 50.04 hours of audio and transcriptions of L2 learners\u27 spontaneous speech. Our linguistic analysis reveals that transcriptions in our dataset contain L2S (L2 learner\u27s Spontaneous speech) features, consisting of ungrammatical expressions and disfluencies (e.g., filler words, word repetitions, self-repairs, false starts), significantly more than native speech datasets. Fine-tuning whisper-small.en with LearnerVoice achieves a WER of 10.26%, 44.2% lower than vanilla whisper-small.en. Furthermore, our qualitative analysis indicates that 54.2% of errors from the vanilla model on LearnerVoice are attributable to L2S features, with 48.1% of them being reduced in the fine-tuned model.Proceedings of Interspeec
Anisotropic He-ion irradiation damages in nanocolumnarWthin films
No full textThe effects of He-ion irradiation on the microstructures and the mechanical, thermal properties of sputter-deposited nanocolumnar tungsten thin films have been studied. 200 keV He+ ion irradiation with a fluence of 2x1017 ions/cm2 was performed in the growth direction of the W thin films. Small scale mechanical testing methods, such as nanoindentation and square membrane deflection experiments, were carried out, and the thermal conductivity measurement was performed based on the electrical resistivity measurement and the Wiedemann–Franz law for the unirradiated and irradiated W thin films. It was revealed that the properties in the out-of-plane direction are not changed much, but a significant degradation occurs in the in-plane direction after the He-ion irradiation. The microstructure of the film and the distribution of He-ion induced damages are responsible for the anisotropic property changes by He-ion irradiation.
Reconstructing the Genetic Relationship between Ancient and Present-Day Siberian Populations
No full textHuman populations across a vast area in northern Eurasia, from Fennoscandia to Chukotka, share a distinct genetic component often referred to as the Siberian ancestry. Most enriched in present-day Samoyedic-speaking populations such as Nganasans, its origins and history still remain elusive despite the growing list of ancient and present-day genomes from Siberia. Here, we reanalyze published ancient and present-day Siberian genomes focusing on the Baikal and Yakutia, resolving key questions regarding their genetic history. First, we show a long-term presence of a unique genetic profile in southern Siberia, up to 6,000 yr ago, which distinctly shares a deep ancestral connection with Native Americans. Second, we provide plausible historical models tracing genetic changes in West Baikal and Yakutia in fine resolution. Third, the Middle Neolithic individual from Yakutia, belonging to the Belkachi culture, serves as the best source so far available for the spread of the Siberian ancestry into Fennoscandia and Greenland. These findings shed light on the genetic legacy of the Siberian ancestry and provide insights into the complex interplay between different populations in northern Eurasia throughout history.Y
A Survey of Robot Intelligence with Large Language Models
No full textSince the emergence of ChatGPT, research on large language models (LLMs) has actively progressed across various fields. LLMs, pre-trained on vast text datasets, have exhibited exceptional abilities in understanding natural language and planning tasks. These abilities of LLMs are promising in robotics. In general, traditional supervised learning-based robot intelligence systems have a significant lack of adaptability to dynamically changing environments. However, LLMs help a robot intelligence system to improve its generalization ability in dynamic and complex real-world environments. Indeed, findings from ongoing robotics studies indicate that LLMs can significantly improve robots’ behavior planning and execution capabilities. Additionally, vision-language models (VLMs), trained on extensive visual and linguistic data for the vision question answering (VQA) problem, excel at integrating computer vision with natural language processing. VLMs can comprehend visual contexts and execute actions through natural language. They also provide descriptions of scenes in natural language. Several studies have explored the enhancement of robot intelligence using multimodal data, including object recognition and description by VLMs, along with the execution of language-driven commands integrated with visual information. This review paper thoroughly investigates how foundation models such as LLMs and VLMs have been employed to boost robot intelligence. For clarity, the research areas are categorized into five topics: reward design in reinforcement learning, low-level control, high-level planning, manipulation, and scene understanding. This review also summarizes studies that show how foundation models, such as the Eureka model for automating reward function design in reinforcement learning, RT-2 for integrating visual data, language, and robot actions in vision-language-action models, and AutoRT for generating feasible tasks and executing robot behavior policies via LLMs, have improved robot intelligence
FPGA Implementation of Keyword Spotting System Using Depthwise Separable Binarized and Ternarized Neural Networks
No full textKeyword spotting (KWS) systems are used for human–machine communications in various applications. In many cases, KWS involves a combination of wake-up-word (WUW) recognition for device activation and voice command classification tasks. These tasks present a challenge for embedded systems due to the complexity of deep learning algorithms and the need for optimized networks for each application. In this paper, we propose a depthwise separable binarized/ternarized neural network (DS-BTNN) hardware accelerator capable of performing both WUW recognition and command classification on a single device. The design achieves significant area efficiency by redundantly utilizing bitwise operators in the computation of the binarized neural network (BNN) and ternary neural network (TNN). In a complementary metal-oxide semiconductor (CMOS) 40 nm process environment, the DS-BTNN accelerator demonstrated significant efficiency. Compared with a design approach where BNN and TNN were independently developed and subsequently integrated as two separate modules into the system, our method achieved a 49.3% area reduction while yielding an area of 0.558 mm2. The designed KWS system, which was implemented on a Xilinx UltraScale+ ZCU104 field-programmable gate array (FPGA) board, receives real-time data from the microphone, preprocesses them into a mel spectrogram, and uses this as input to the classifier. Depending on the order, the network operates as a BNN or a TNN for WUW recognition and command classification, respectively. Operating at 170 MHz, our system achieved 97.1% accuracy in BNN-based WUW recognition and 90.5% in TNN-based command classification
Generalizable Novel-View Synthesis using a Stereo Camera
No full textIn this paper, we propose the first generalizable view synthesis approach
that specifically targets multi-view stereo-camera images. Since recent stereo
matching has demonstrated accurate geometry prediction, we introduce stereo
matching into novel-view synthesis for high-quality geometry reconstruction. To
this end, this paper proposes a novel framework, dubbed StereoNeRF, which
integrates stereo matching into a NeRF-based generalizable view synthesis
approach. StereoNeRF is equipped with three key components to effectively
exploit stereo matching in novel-view synthesis: a stereo feature extractor, a
depth-guided plane-sweeping, and a stereo depth loss. Moreover, we propose the
StereoNVS dataset, the first multi-view dataset of stereo-camera images,
encompassing a wide variety of both real and synthetic scenes. Our experimental
results demonstrate that StereoNeRF surpasses previous approaches in
generalizable view synthesis.Comment: Accepted to CVPR 2024. Project page URL:
https://jinwonjoon.github.io/stereonerf
