1,939 research outputs found
AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation
No full textAttention is the core mechanism of today's most used architectures for
natural language processing and has been analyzed from many perspectives,
including its effectiveness for machine translation-related tasks. Among these
studies, attention resulted to be a useful source of information to get
insights about word alignment also when the input text is substituted with
audio segments, as in the case of the speech translation (ST) task. In this
paper, we propose AlignAtt, a novel policy for simultaneous ST (SimulST) that
exploits the attention information to generate source-target alignments that
guide the model during inference. Through experiments on the 8 language pairs
of MuST-C v1.0, we show that AlignAtt outperforms previous state-of-the-art
SimulST policies applied to offline-trained models with gains in terms of BLEU
of 2 points and latency reductions ranging from 0.5s to 0.8s across the 8
languages.Comment: Accepted at Interspeech 202
Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023
No full textThis paper describes the FBK's participation in the Simultaneous Translation
and Automatic Subtitling tracks of the IWSLT 2023 Evaluation Campaign. Our
submission focused on the use of direct architectures to perform both tasks:
for the simultaneous one, we leveraged the knowledge already acquired by
offline-trained models and directly applied a policy to obtain the real-time
inference; for the subtitling one, we adapted the direct ST model to produce
well-formed subtitles and exploited the same architecture to produce timestamps
needed for the subtitle synchronization with audiovisual content. Our
English-German SimulST system shows a reduced computational-aware latency
compared to the one achieved by the top-ranked systems in the 2021 and 2022
rounds of the task, with gains of up to 3.5 BLEU. Our automatic subtitling
system outperforms the only existing solution based on a direct system by 3.7
and 1.7 SubER in English-German and English-Spanish respectively.Comment: Published at IWSTL 202
Attention as a Guide for Simultaneous Speech Translation
No full textThe study of the attention mechanism has sparked interest in many fields,
such as language modeling and machine translation. Although its patterns have
been exploited to perform different tasks, from neural network understanding to
textual alignment, no previous work has analysed the encoder-decoder attention
behavior in speech translation (ST) nor used it to improve ST on a specific
task. In this paper, we fill this gap by proposing an attention-based policy
(EDAtt) for simultaneous ST (SimulST) that is motivated by an analysis of the
existing attention relations between audio input and textual output. Its goal
is to leverage the encoder-decoder attention scores to guide inference in real
time. Results on en->{de, es} show that the EDAtt policy achieves overall
better results compared to the SimulST state of the art, especially in terms of
computational-aware latency.Comment: Accepted to ACL 202
Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation
No full textSimultaneous speech translation (SimulST) systems aim at generating their
output with the lowest possible latency, which is normally computed in terms of
Average Lagging (AL). In this paper we highlight that, despite its widespread
adoption, AL provides underestimated scores for systems that generate longer
predictions compared to the corresponding references. We also show that this
problem has practical relevance, as recent SimulST systems have indeed a
tendency to over-generate. As a solution, we propose LAAL (Length-Adaptive
Average Lagging), a modified version of the metric that takes into account the
over-generation phenomenon and allows for unbiased evaluation of both
under-/over-generating systems.Comment: AutoSimTrans Workshop @ NAACL202
Does Simultaneous Speech Translation need Simultaneous Models?
No full textIn simultaneous speech translation (SimulST), finding the best trade-off
between high translation quality and low latency is a challenging task. To meet
the latency constraints posed by the different application scenarios, multiple
dedicated SimulST models are usually trained and maintained, generating high
computational costs. In this paper, motivated by the increased social and
environmental impact caused by these costs, we investigate whether a single
model trained offline can serve not only the offline but also the simultaneous
task without the need for any additional training or adaptation. Experiments on
en->{de, es} indicate that, aside from facilitating the adoption of
well-established offline techniques and architectures without affecting
latency, the offline solution achieves similar or better translation quality
compared to the same model trained in simultaneous settings, as well as being
competitive with the SimulST state of the art.Comment: Findings of EMNLP 202
Speechformer: Reducing Information Loss in Direct Speech Translation
No full textTransformer-based models have gained increasing popularity achieving state-of-the-art performance in many research fields including speech translation. However, Transformer’s quadratic complexity with respect to the input sequence length prevents its adoption as is with audio signals, which are typically represented by long sequences. Current solutions resort to an initial sub-optimal compression based on a fixed sampling of raw audio features. Therefore, potentially useful linguistic information is not accessible to higher-level layers in the architecture. To solve this issue, we propose Speechformer, an architecture that, thanks to reduced memory usage in the attention layers, avoids the initial lossy compression and aggregates information only at a higher level according to more informed linguistic criteria. Experiments on three language pairs (en→de/es/nl) show the efficacy of our solution, with gains of up to 0.8 BLEU on the standard MuST-C corpus and of up to 4.0 BLEU in a low resource scenario
On the Generation of Synthetic Fingerprint Alterations
No full textIn this paper we propose some techniques to generate synthetic altered fingerprints and prove the utility of the generated datasets for developing, tuning and evaluating algorithms for altered fingerprint detection/matching. Due to the lack of public databases of altered fingerprints the generation tool proposed (and made freely available) can be a valid instrument to boost research on these challenging problems
Dealing with training and test segmentation mismatch: FBK@IWSLT2021
No full textThis paper describes FBK's system submission to the IWSLT 2021 Offline Speech
Translation task. We participated with a direct model, which is a
Transformer-based architecture trained to translate English speech audio data
into German texts. The training pipeline is characterized by knowledge
distillation and a two-step fine-tuning procedure. Both knowledge distillation
and the first fine-tuning step are carried out on manually segmented real and
synthetic data, the latter being generated with an MT system trained on the
available corpora. Differently, the second fine-tuning step is carried out on a
random segmentation of the MuST-C v2 En-De dataset. Its main goal is to reduce
the performance drops occurring when a speech translation model trained on
manually segmented data (i.e. an ideal, sentence-like segmentation) is
evaluated on automatically segmented audio (i.e. actual, more realistic testing
conditions). For the same purpose, a custom hybrid segmentation procedure that
accounts for both audio content (pauses) and for the length of the produced
segments is applied to the test data before passing them to the system. At
inference time, we compared this procedure with a baseline segmentation method
based on Voice Activity Detection (VAD). Our results indicate the effectiveness
of the proposed hybrid approach, shown by a reduction of the gap with manual
segmentation from 8.3 to 1.4 BLEU points.Comment: Accepted at IWSLT202
TUTELA DEL LAVORO E LIBERTA' D'IMPRESA NEI PROCESSI DI ESTERNALIZZAZIONE
No full textL’elaborato analizza le conseguenze lavoristiche della successione fra imprenditori, muovendo da una ricognizione delle varie tipologie di esternalizzazione con le relative esigenze e principali criticità.
L’indagine si concentra in primo luogo sul trasferimento d’azienda, esaminando la normativa e la giurisprudenza europee per passare poi alla disciplina di diritto interno, alle procedure sindacali e a uno specifico focus sul trasferimento delle aziende in crisi.
Successivamente l’autore si sofferma sull’appalto, prendendone in particolare considerazione gli indici di genuinità, i criteri di distinzione dalla somministrazione illecita di manodopera e la tutela delle maestranze in caso di avvicendamento fra imprese.
Da ultimo, la ricerca approfondisce le c.d. “clausole sociali”, sia di prima che di seconda generazione, valutandone la compatibilità con il diritto eurounitario e con la costituzione nonché riflettendo sui possibili rimedi in caso di loro violazione.The author analyzes the labour consequences of the succession between entrepreneurs, starting from a recognition of the various types of outsourcing with the related needs and main critical issues.
The survey focuses primarily on the transfer of businesses, examining European legislation and case-law and then moving on to internal legislation, trade union procedures and a specific focus on the transfer of companies in crisis.
The author then dwells on the contract, taking into account in particular the indications of authenticity, the criteria of distinction from the illicit administration of labour and the protection of workers in the event of turnover between companies.
Finally, the research deepens the "social clauses", both first and second generation, assessing their compatibility with European law and with the constitution and reflecting on possible remedies in case of their violation
How Do Hyenas Deal with Human Speech? Speech Recognition and Translation with ConfHyena
No full textThe attention mechanism, a cornerstone of state-of-the-art neural models, faces computational hurdles in processing long sequences due to its quadratic complexity. Consequently, research efforts in the last few years focused on finding more efficient alternatives. Among them, Hyena (Poli et al., 2023) stands out for achieving competitive results in both language modeling and image classification, while offering sub-quadratic memory and computational complexity. Building on these promising results, we propose ConfHyena, a Conformer whose encoder self-attentions are replaced with an adaptation of Hyena for speech processing, where the long input sequences cause high computational costs. Through experiments in automatic speech recognition (for English) and translation (from English into 8 target languages), we show that our best ConfHyena model significantly reduces the training time by 27%, at the cost of minimal quality degradation (∼1%), which, in most cases, is not statistically significant
- …
