1,721,014 research outputs found
Front-End Processing for Speech Applications with Deep Learning Techniques
Get PDFL’elaborazione vocale front-end gioca un ruolo fondamentale per varie applicazioni quotidiane come le teleconferenze e le conversazioni telefoniche, apparecchi acustici, assistenti vocali e molto altro.
Questo termine incorpora una larga varietà di attività e assolve tante di queste quante sono le possibili applicazioni: attività di riconoscimento vocale e di parole chiave, cancellazione del rumore e del riverbero, diarizzazione e così via, ognuno dei quali ricopre una fase di pre-elaborazione essenziale per un particolare caso d'uso a valle.
L’obiettivo di questa tesi è fornire una visione generale dell’elaborazione vocale front end e presentare diversi contributi a questa importante linea di ricerca che fa riferimento a molti problemi pratici.
Più in dettaglio, si focalizza in particolare sull’uso delle tecniche di deep learning, spesso supportate dalle tecniche classiche di processing per immagini, per affrontare i compiti front-end di miglioramento del parlato multicanale, selezione dei canali, individuazione delle parole chiave, conteggio degli altoparlanti e diarizzazione.
L'accento è posto su approcci a bassa complessità computazionale e/o a bassa latenza, nonché sull'integrazione tra diversi componenti di front-end per raggiungere un obiettivo particolare, ad esempio il rilevamento dell'attività vocale insieme alla separazione del parlato per ottenere la diarizzazione o l'uso di caratteristiche spaziali per migliorare il conteggio dei parlanti.
Per quanto riguarda il potenziamento del parlato multicanale, presentiamo uno studio sull'uso di gruppi di filtri apprendibili per la formazione di fasci acustici, che può aprire interessanti direzioni di ricerca future verso applicazioni a bassa latenza.
Viene affrontato inoltre il problema della selezione dei canali e si propone di formularlo come un problema di apprendimento per rango. L'algoritmo MicRank proposto è leggero e in alcuni casi può raggiungere prestazioni vicine alle tecniche di selezione con oracolo.
I bassi requisiti computazionali sono anche l'obiettivo principale del nostro framework di cancellazione implicita dell'eco acustico, che consente di individuare in modo robusto e in streaming le parole chiave e di rilevare il parlato su dispositivi edge. È anche uno degli obiettivi principali del nostro studio sul rilevamento del parlato sovrapposto e sul conteggio dei parlanti su corpora di riunioni del mondo reale.
Per quanto riguarda quest'ultimo aspetto, dimostriamo che le caratteristiche basate sullo spazio possono aumentare notevolmente le prestazioni e allo stesso tempo mantenere il costo computazionale contenuto.
Infine, si presenta un lavoro di diarizzazione guidata della separazione del parlato per le conversazioni telefoniche, in cui si pone particolare attenzione ai casi d'uso a bassa latenza. I risultati sono promettenti in termini di prestazioni di riconoscimento e diarizzazione e aprono prospettive interessanti per applicazioni come il live captioning.Front-end speech processing plays a vital role in many everyday applications such as teleconferencing and telephone conversations, hearing aid devices, voice-enabled assistants and more.
Such term encompasses a wide variety of tasks
and absolves to at least as many tasks as are the potential applications: voice activity detection and keyword spotting, denoising, dereverberation, diarization and so on, each performing an essential pre-processing step for a particular downstream use-case.
The goal of this dissertation is to give an overview of front-end speech processing and present different contributions to this important line of research that address many practical problems.
More in detail, here we focus especially on the use of deep learning techniques, often supported by classical signal processing techniques, to tackle the front-end tasks of multi-channel speech enhancement, channel selection, keyword spotting, speaker counting and diarization.
Emphasis is placed on low computational complexity and/or low-latency approaches as well as integration between different front-end components to achieve one particular goal e.g. voice activity detection together with speech separation to obtain diarization or the use of spatial features to improve speaker counting.
Regarding multi-channel speech enhancement we present a study on the use of learnable filterbanks for acoustic beamforming which can open up interesting future research directions towards low-latency applications.
We also address the channel selection problem and propose to formulate it as a learning to rank problem. Our proposed MicRank algorithm is lightweight and can achieve performance in some instances close to oracle selection techniques.
Low computational requirements are also the primary goal of our implicit acoustic echo cancellation framework, which allows for streamable robust keyword spotting and device-directed speech detection on edge devices. It is also one of the main focuses of our study on overlapped speech detection and speaker counting on real world meeting corpora.
Regarding this latter, we show that spatial based features could boost considerably the performance and at the same time keep the computational cost contained.
Finally we present a work on speech separation guided diarization for telephone conversations, in which we place special attention on extreme low-latency use-cases. The results are promising in terms of recognition and diarization performance and open up exciting prospects for applications such as live captioning
Tackling the Linear Sum Assignment Problem with Graph Neural Networks
No full textLinear Assignment Problems are fundamental combinatorial optimization problems that appear throughout domains such as logistics, robotics and telecommunications. In general, solving assignment problems to optimality is computationally infeasible even for contexts of small dimensionality, and so heuristic algorithms are often employed to find near-optimal solutions. The handcrafting of a heuristic usually requires expert-knowledge to exploit the problem structure to be addressed, however if the problem description changes slightly, a previously derived heuristic may no longer be appropriate.
This work explores a more general-purpose learning approach, based on the description of the problem through a bipartite graph, and the use of a Message Passing Graph Neural Network model, to attain the correct assignment permutation.
The simulation results indicate that the proposed structure allows for a significant increase in classification accuracy if compared with two different DNN approaches based on Dense Networks and Convolutional Neural Networks, furthermore, the GNN has proved to be very efficient with regard to the processing time and memory requirements, thanks to intrinsic parameter-sharing capability
A graph-based neural approach to linear sum assignment problems
Get PDFLinear assignment problems are well-known combinatorial optimization problems involving domains such as logistics, robotics, and telecommunications. In general, obtaining an optimal solution to such problems is computationally infeasible even in small settings, so heuristic algorithms are often used to find near-optimal solutions.
In order to attain the right assignment permutation, this study investigates a general-purpose learning strategy that uses a bipartite graph to describe the problem structure and a Message Passing Graph Neural Network (GNN) model to learn the correct mapping.
Comparing the proposed structure with two existing DNN solutions, simulation results show that the proposed approach significantly improves classification accuracy, proving to be very efficient in terms of processing time and memory requirements, due to its inherent parameter sharing capability.
Among the many practical uses that require solving allocation problems in everyday scenarios, we decided to apply the proposed approach to address the scheduling of electric smart meters access within an electricity distribution smart grid infrastructure, since near-real-time energy monitoring is a key element of the green transition that has become increasingly important in recent times. The results obtained show that the proposed graph-based solver, although sub-optimal, exhibits the highest scalability, compared with other state-of-the-art heuristic approaches. To foster the reproducibility of the results, we made the code available at https://github.com/aircarlo/GNN\_LSAP
Learning Filterbanks for End-to-End Acoustic Beamforming
No full textRecent work on monaural source separation has shown that performance can be increased by using fully learned filterbanks with short windows. On the other hand it is widely known that, for conventional beamforming techniques, performance increases with long analysis windows. This applies also to most hybrid neural beamforming methods which rely on a deep neural network (DNN) to estimate the spatial covariance matrices. In this work we try to bridge the gap between these two worlds and explore fully end-to-end hybrid neural beamforming in which, instead of using the Short-TimeFourier Transform, also the analysis and synthesis filterbanks are learnt jointly with the DNN. In detail, we explore two different types of learned filterbanks: fully learned and analytic. We perform a detailed analysis using the recent Clarity Challenge data and show that by using learnt filterbanks it is possible to surpass oracle-mask based beamforming for short windows
One Model to Rule Them All ? Towards End-to-End Joint Speaker Diarization and Speech Recognition
No full textThis paper presents a novel framework for joint speaker diarization (SD) and automatic speech recognition (ASR), named SLIDAR (sliding-window diarization-augmented recognition). SLIDAR can process arbitrary length inputs and can handle any number of speakers, effectively solving “who spoke what, when” concurrently. SLIDAR leverages a sliding window approach and consists of an end-to-end diarization-augmented speech transcription (E2E DAST) model which provides, locally, for each window: transcripts, diarization and speaker embeddings. The E2E DAST model is based on an encoder-decoder architecture and leverages recent techniques such as serialized output training and “Whisper-style” prompting. The local outputs are then combined to get the final SD+ASR result by clustering the speaker embeddings to get global speaker identities. Experiments performed on monaural recordings from the AMI corpus confirm the effectiveness of the method in both close-talk and far-field speech scenario
A Score-aware Generative Approach for Music Signals Inpainting
No full textSeveral issues can seriously degrade the quality of digital audio, such as packet loss on IP-based networks or damaged storage media, impacting intelligibility and user experience. This paper presents a generative approach, aiming to repair lost fragments in audio streams. Inspired by the well-established image-to-image translation ability of generative adversarial networks (GANs) and based on the bin2bin framework, previously introduced for speech inpainting, we propose an enhanced framework which performs the translation task from CQT magnitude spectrograms of music signal frames with lost regions, to reliable spectrograms. The goal is to effectively reconstruct missing audio segments, enabling a seamless listening experience for the audience. The proposed pipeline combines the traditional GAN discriminative loss function with two additional objectives: a loss function related to perceptual audio quality, and a second one based on the L2 norm between the true and predicted piano-roll, estimated by the CQT reconstruction. Through comprehensive evaluations on gaps of 375ms and 750ms, which are considered in the literature to be respectively of "small" and "medium" duration, we demonstrate the robustness and effectiveness of our framework in producing coherent reconstructions with reduced artifacts. The proposed approach outperforms a baseline cGAN-based method, GACELA. In terms of ODG score, a metric inspired by a human-based scoring system, we achieve a gain in performance up to 13.3%, while the improvement in Structural Similarity (SSIM) between the clean and restored spectrograms reaches 13.6%
Conversational Speech Separation: an Evaluation Study for Streaming Applications
Get PDFContinuous speech separation (CSS) is a recently proposed framework which aims at separating each speaker from an input mixture signal in a streaming fashion.
Hereafter we perform an evaluation study on practical design considerations for a CSS system, addressing important aspects which have been neglected in recent works. In particular, we focus on the trade-off between separation performance, computational requirements and output latency showing how an offline separation algorithm can be used to perform CSS with a desired latency.
We carry out an extensive analysis on the choice of CSS processing window size and hop size on sparsely overlapped data. We find out that the best trade-off between computational burden and performance is obtained for a window of 5 s
End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations
Get PDFRecent works show that speech separation guided diarization (SSGD) is an
increasingly promising direction, mainly thanks to the recent progress in
speech separation. It performs diarization by first separating the speakers and
then applying voice activity detection (VAD) on each separated stream. In this
work we conduct an in-depth study of SSGD in the conversational telephone
speech (CTS) domain, focusing mainly on low-latency streaming diarization
applications. We consider three state-of-the-art speech separation (SSep)
algorithms and study their performance both in online and offline scenarios,
considering non-causal and causal implementations as well as continuous SSep
(CSS) windowed inference. We compare different SSGD algorithms on two widely
used CTS datasets: CALLHOME and Fisher Corpus (Part 1 and 2) and evaluate both
separation and diarization performance. To improve performance, a novel, causal
and computationally efficient leakage removal algorithm is proposed, which
significantly decreases false alarms. We also explore, for the first time,
fully end-to-end SSGD integration between SSep and VAD modules. Crucially, this
enables fine-tuning on real-world data for which oracle speakers sources are
not available. In particular, our best model achieves 8.8% DER on CALLHOME,
which outperforms the current state-of-the-art end-to-end neural diarization
model, despite being trained on an order of magnitude less data and having
significantly lower latency, i.e., 0.1 vs. 1 s. Finally, we also show that the
separated signals can be readily used also for automatic speech recognition,
reaching performance close to using oracle sources in some configurations.Comment: 16 pages, 7 figure
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
