1,720,963 research outputs found
Synthesis of New Words for Improved Dysarthric Speech Recognition on an Expanded Vocabulary
No full textDysarthria is a condition where people experience a reduction in speech intelligibility due to a neuromotor disorder. Previous works in dysarthric speech recognition have focused on accurate recognition of words encountered in training data. Due to the rarity of dysarthria in the general population, a relatively small amount of publicly-available training data exists for dysarthric speech. The number of unique words in these datasets is small, so ASR systems trained with existing dysarthric speech data are limited to recognition of those words. In this paper, we propose a data augmentation method using voice conversion that allows dysarthric ASR systems to accurately recognize words outside of the training set vocabulary. We demonstrate that a small amount of dysarthric speech data can be used to capture the relevant vocal characteristics of a speaker with dysarthria through a parallel voice conversion system. We show that it's possible to synthesize utterances of new words that were never recorded by speakers with dysarthria, and that these synthesized utterances can be used to train a dysarthric ASR system
3차원 점구름 기반 객체 탐지 및 분할 기법에서의 딥러닝을 활용한 접근법
No full text학위논문(석사) - 한국과학기술원 : 전기및전자공학부, 2022.8,[vi, 60 p. :]This dissertation considers a 3D object detection and instance segmentation method based on point clouds. 3D object detection is a task that predicts bounding boxes including objects in point cloud datasets. 3D instance segmentation is a task that segment each pixel or point into a object that the point belongs to. Despite several previous works, these tasks have some limitations : (1) While PointNet-based permutation-invariant network extracts features of point cloud for object detection, independent network are used to extract the feature of grouped points without interaction between grouped points. (2) Bounding box detection is sensitive to rotation error and translation error for flat objects. (3) Incorrect semantic prediction propogates to final instance prediction in one-hot semantic based grouping instance segmentation. To mitigate these problems, we propose (1) Group-Point Attention(GPAN) to enhance feature representation of 3D point clouds, adapting cross-attention between base point and grouped points to encourage interaction. (2) SphereSeg by predicting spheres replacing bounding boxes that are invariant to object orientation and more robust to localization error. (3) SoftGroup to improve the quality of instance proposals, further refine proposals for accurate 3D instance segmentation. Experiments on various 3D point cloud datasets show the efficacy of the proposed methods.한국과학기술원 :전기및전자공학부
장면 분할을 위한 연상 도메인 적응 기법
No full text학위논문(석사) - 한국과학기술원 : 전기및전자공학부, 2018.2,[iii, 21 p. :]This paper considers an associative unsupervised domain adaptation learning algorithm for performing semantic segmentation on real urban drive-cam data using photo-realistic synthetic training data. To circumvent the difficulty of collecting and laboriously annotating a large amounts of real urban scene data, large amounts of computer-annotated synthetic training data is provided as a substitutehowever, without any consideration to domain mismatch, a significant decreases in prediction performance is observed. Inspired by the recent success of an associative domain adaptation algorithm for simple classification, this algorithm is adapted to semantic segmentation to reduce domain mismatch between training and testing. Considering associative learning for multiple instances within a single high-resolution image and ambiguous and undecided labels in a semantic segmentation training dataset, this adaptation is not straightforward. In this paper, an algorithm is proposed to address such difficulties in adapting associative learning to semantic segmentation by partitioning an image into patches and associating labeled patches with unlabeled patches. The results from the model using SYNTHIA and GTA5 dataset as a source dataset shows state-of-the-art performance on the CityScapes dataset.한국과학기술원 :전기및전자공학부
generic-attribute-ppse network for fine-grained visual categorization using multi-attribute attention module
No full text학위논문(석사) - 한국과학기술원 : 전기및전자공학부, 2020.2,[iv, 33 p. :]This paper proposes a Generic-Attribute-Pose Network (GAPNet) that attends spatial regions to discriminate for fine-grained visual categorization. Compared to a prototypical image classification task with reasonably large variation between classes, fine-grained visual categorization is a task that involves small inter-class variation and large intra-class variation. The GAPNet attends salient regions that can discriminate between classes, but is common within a class. The GAPNet is composed of four streams: the Generic-, Pose-, Part-, and Attribute-stream. The Generic-stream is the main-stream that attends the backbone features with respect to the pose and part-attributes by an attention module referred to as Multi-Attribute Attention Module (MAAM). The Pose-stream extracts pose-specific feature from the backbone feature, while the Part-Attribute streams output features specific pertaining to parts classified in the Pose-stream. The MAAM takes pose, part, and attribute features as query and backbone feature as key-value, and performs cross-inner dot-product between the channels of them followed by max-pooling, to attend the channels of key-value according to best matched query channel. To evaluate the performance of GAPNet, quantitative evaluation and ablation studies on Caltech-UCSD Birds (CUB-Birds) and NABirds are conducted. The functionality of the MAAM module is also verified. Moreover, the effects of each stream in GAPNet are evaluated quantitatively and qualitatively. The influence of stream-order applied to the backbone feature is analyzed. The experiments for GAPNet based on weakly-supervised methods without part annotations are implemented.한국과학기술원 :전기및전자공학부
비디오 그룹 내 순간 검색 기법 및 비디오 기반의 대화 기법에서의 딥러닝을 활용한 접근법
No full text학위논문(석사) - 한국과학기술원 : 전기및전자공학부, 2022.2,[iv, 33 p. :]This dissertation considers Video Corpus Moment Retrieval(VCMR) and Video Grounded Dialogue System(VGDS). VCMR is to localize temporal moment in a corpus of videos that best corresponds to the given text query. VGDS is to answer the question with video and a series of questions and answers related to the content of the video. Despite several previous works, these tasks have some limitations : (1) VCMR is performed mainly in a supervised manner(2) In VCMR, lack of associating among features obtained from various multimodal inputs sharing common semantics(3) VGDSs answer the question based on linguistic correlation instead of video content. For solving these limitations, we propose (1) Weakly-supervised Moment Retrieval Network performs VCMR in a weakly-supervised manner, (2) Semantic Association Network for VCMR associates common semantics within the same modality and across the different modalities, and (3) Counterfactual Object Replacement for VGDS generates counterfactual augmented training samples. Our proposed networks and method show better performance than previous networks and methods, and several analyzes show the results of solving the problems.한국과학기술원 :전기및전자공학부
딥러닝을 활용한 멀티모달 비디오 질의응답 기법
No full text학위논문(박사) - 한국과학기술원 : 전기및전자공학부, 2021.2,[v, 65 p. :]This dissertation considers the problem of Multimodal Video Question Answering (MVQA) which aims at joint understanding of video and accompanied subtitles to answer the given question. Compared to visual question answering (VQA) which is question answering on a single image, MVQA is challenging in two aspects: (1) it requires pinpointing the temporal parts relevant to answer the question as input is long untrimmed video, and (2) it involves reasoning on heterogeneous modality where different question requires different modality to answer the question. We propose two MVQA networks to address aforementioned challenges: (1) Progressive Attention Memory Network (PAMN), and (2) Modality Shifting Attention Network (MSAN). Experimental results on MovieQA and TVQA shows proposed PAMN and MSAN achieves significant performance improvement compared to previous state-of-the-art methods. Furthermore, we propose Structured Co-reference Graph Attention for Video-grounded Dialog (VideoDial) task and showed performance boost on AVSD benchmark.한국과학기술원 :전기및전자공학부
초해상도 단일 영상 복원 및 영상 화질 개선을 위한 심층 나선형 컨볼루셔널 신경망 연구
No full text학위논문(박사) - 한국과학기술원 : 전기및전자공학부, 2018.2,[vii, 102 p. :]Convolutional neural network (CNN) based super-resolution (SR) and restoration algorithms have recently achieved a significant improvement on single image super-resolution (SISR) and various image enhancement (IE) tasks. The main objective of SR and IE are to generate a high-quality, high-resolution (HR) image from a given single low-resolution (LR) image or corrupted noisy image. Despite the powerful learning strength of deep networks, the previous CNN-based SR and IE algorithms still have limitations in recovering fine-textured HR results, although they have shown a high numerical similarity score such as a peak signal-to-noise ratio. This dissertation considers a fully end-to-end trainable texture-enhanced multi-scale SR network (TE-MSRN) and IE networks (e.g., multi-scale denoising network (MsDNN), multi-scale deblurring network (MsDBN) and video quality enhancement network (VQENet)) based on a deep spiral CNN while mitigating the limitations of previous deep SR and IE networks in terms of the SR and IE performance, training efficiency, and in recovering fine-textured details. When the SR and IE networks getting deeper, learning the long-range dependencies of the complex relationships between corrupted LR image and HR image becomes more difficult. Generally, the deeper networks suffer from not only the additional increase of the computational complexity and memory cost but also the difficulty in training networks due to the over-fitting and gradient explosion/vanishing/shattered problems. To overcome these difficulties, this dissertation investigates six extensions: an upscaling network with multi-scale feature embedding, multi-scale restoration network, both global and local residual learning, texture evaluating network, a deep spiral CNN, and combination of multiple loss. The TE-MSRN takes a LR image and reconstructs a HR image using an upscaling network and restoration network while not only minimizing corresponding residuals but also enhancing the texture by enforcing the HR prediction to generate ground truth texture through a texture evaluating network. The global residual between intermediate HR prediction and ground truth is minimized in a recurrent manner while reducing each local residual using a deep spiral CNN, and the intermediate output of each recurrent state in the restoration network is supervised by the intermediate auxiliary loss. While reconstructing the HR output, the texture evaluating network is cascaded on to the restoration network such that an accurate texture prediction can be made from the output of the restoration network during training, with this then removed during testing. A deep spiral CNN is considered via a recurrent structure while recurrently minimizing the restoration residual in multiple stages: multi-scale recurrent CNN takes its previous output as input and produces an output that is closer to ground truth residual. With each iteration, the residual gradually reduced, and the HR prediction becomes closer to ground truth. The entire process is reminiscent of a spiraling staircase reaching its destination. Nevertheless, it remains jointly optimized in a unified single architecture using all subnetworks specialized for their own purposes from scratch while increasing the training efficiency and yielding superior SR performance. The considered TE-MSRN is trained to produce a fine-textured HR image with suitable combinations of losses: -loss, -loss, perceptual structural similarity loss, and intermediate auxiliary loss. Based on a combination of loss functions, the TE-MSRN is explicitly trained to reduce visually implausible artifacts further, leading to a more accurate HR result. This is demonstrated to be effective when used to reduce visually implausible artifacts further, leading to a more accurate HR result. The TE-MSRN is completely end-to-end trainable with integration into a unified single architecture. The main architecture of TE-MSRN is modified to MsDNN, MsDBN, and VQENet for each task. The performance of the TE-MSRN is evaluated on six standard benchmark datasets for SR including two datasets consisting of only textures, four benchmark datasets for IE, and three benchmark datasets for complex video scene analysis (VSA) with video quality enhancement (VQE). Extensive experimental results show that the TE-MSRN, MsDNN, and MsDBN achieve the best performance while making better texture predictions compared to the current state-of-the-art SR and IE algorithms, and show that VQENet helps to increase VSA performance.한국과학기술원 :전기및전자공학부
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
비원어민의 음성인식 향상을 위한 딥러닝을 활용한 접근법
No full text학위논문(석사) - 한국과학기술원 : 전기및전자공학부, 2023.8,[v, 35 p. :]Automatic Speech Recognition (ASR) is a task that converts a spoken language into written text, and these systems have attained unprecedented performance with large speech models pre-trained based on self-supervised speech representation learning. However, these pre-trained speech models suffer from representational bias as they tend to better represent those prominent accents (i.e., native (L1) English accent) in the pre-training speech corpus than less represented accents, resulting in a deteriorated performance for non-native (L2) English accents. Although there have been some approaches to mitigate this issue, all of these methods require updating the pre-trained model weights. In this paper, we propose Information Theoretic Adversarial Prompt Tuning (INTapt), which introduces prompts concatenated to the original input that can re-modulate the attention of the pre-trained model such that the corresponding input resembles a native (L1) English speech without updating the backbone weights. INTapt is trained simultaneously in the following two manners: (1) adversarial training to reduce accent feature dependence between the original input and the prompt-concatenated input and (2) training to minimize CTC loss for improving ASR performance to a prompt-concatenated input. Experimental results show that INTapt improves the performance of L2 English and increases feature similarity between L2 and L1 accents.한국과학기술원 :전기및전자공학부
- …
