41 research outputs found
Implications of Sepedi/English code switching for ASR systems
No full textCode switching (the process of switching from one language to another during a conversation) is a common phenomenon in multilingual environments. Where a minority and dominant language coincide, code switching from the minority language to the dominant language can become particularly frequent. We analyse one such scenario: Sepedi spoken in South Africa, where English is the dominant language; and determine the frequency and mechanisms of code switching through the analysis of radio broadcasts. We also perform an initial acoustic analysis to determine the impact of such code switching on speech recognition performance. We find that the frequency of code switching is unexpectedly high, and that the continuum of code switching (from unmodified embedded words to loan words absorbed in the matrix language) makes this a particularly challenging task for speech recognition systems.http://www.prasa.org/index.php/2012-03-07-10-55-1
Context-dependent modelling of English vowels in Sepedi code-switched speech
No full textWhen modelling code-switched speech (utterances that contain a mixture of languages), the embedded language often contains phones not found in the matrix language. These are typically dealt with by either extending the phone set or mapping each phone to a matrix language counterpart. We use acoustic log likelihoods to assist us in identifying the optimal mapping strategy at a context-dependent level (that is, at triphone, rather than monophone level) and obtain new insights in the way English/Sepedi code-switched vowels are produce
Developing Speech Resources from Parliamentary Data for South African English
No full textAbstractThe official languages of South Africa can still be classified as under-resourced with respect to the speech resources that are required for technology development. Harvesting speech data from existing sources is one means to create additional resources. The aim of the study reported on in this paper was to improve the harvesting and transcription accuracy of a corpus derived from parliamentary data. This aim was achieved by improving on the text normalisation process and pronunciation modelling as well as by iteratively training more accurate in-domain acoustic models. In this manner, more data could be harvested with higher confidence than using baseline pronunciation dictionaries and out-of-domain speech data
Automatic Recognition of Code-Switched Speech in Sepedi
No full textCode switching (CS) is a natural phenomenon that is often observed in multilingual
speakers. These speakers use words, phrases or sentences from foreign languages and
embed them in sentences in the primary language. Automatic speech recognition (ASR)
systems find code-switched speech difficult to process, and ASR performance is known
to degrade in CS environments. We study the Sepedi/English CS phenomenon in the context of Sepedi ASR. Using experimentation, data collection and quantitative data analysis, we analyse techniques that can be used to effectively model code-switched speech in resource-scarce environments. The focus is on techniques that modify the pronunciation dictionary, in order to improve recognition accuracy. For this purpose, three new speech resources are designed, collected and curated: (1) the Radio Broadcast Corpus contains real examples of code-switching as observed during radio broadcasts; (2) the Sepedi Prompted Code-Switched (SPCS) Corpus is based on true code switching prompts, with each individual prompt recorded by multiple speakers in order to capture pronunciation variability occurring in code-switched speech; and (3) the National Center for Human Language Technology (NCHLT) Sepedi-English code switched subset (NSECSS) corpus does not contain naturally occurring code-switched speech, but rather English as spoken by Sepedi speakers. The latter corpus is particularly useful as its recording conditions and format match two related corpora: English produced by English speakers and Sepedi produced by Sepedi speakers. As part of corpus development, resource collection and analysis tools were developed and evaluated. Utilising these corpora, the implications of code-switched speech for ASR systems were evaluated. Various approaches to pronunciation modelling of code-switched speech were investigated and a novel method for pronunciation prediction developed. This new variant selection approach to modelling code-switched speech requires a two-step process: after grapheme-to-phoneme prediction of foreign words, phoneme-to-phoneme prediction (mapping the foreign phonemes to in-language phonemes) does not only take phoneme identity into account, but also graphemic context. A practical implementation of such an algorithm performed well during recognition experiments, both as a single approach and in combination with other existing approaches. The best overall results were obtained when multiple variants were generated per CS word, and variant-selection included in this process. Even though specifically applied to the Sepedi/English task, the methods themselves are language-independent. In addition, the methods, frequency of and reasons for code switching observed among Sepedi speakers were studied using corpus analysis. Among other results, it was found that the prevalence of code switching within naturally occurring Sepedi speech was much higher than initially anticipated, making this a task well worth studying.Doctora
The evaluation of a code-switched Sepedi-English automatic speech recognition system
No full textSpeech technology is a field that encompasses various techniques and tools
used to enable machines to interact with speech, such as automatic speech
recognition (ASR), spoken dialog systems, and others, allowing a device to
capture spoken words through a microphone from a human speaker. End-to-end
approaches such as Connectionist Temporal Classification (CTC) and
attention-based methods are the most used for the development of ASR systems.
However, these techniques were commonly used for research and development for
many high-resourced languages with large amounts of speech data for training
and evaluation, leaving low-resource languages relatively underdeveloped. While
the CTC method has been successfully used for other languages, its
effectiveness for the Sepedi language remains uncertain. In this study, we
present the evaluation of the Sepedi-English code-switched automatic speech
recognition system. This end-to-end system was developed using the Sepedi
Prompted Code Switching corpus and the CTC approach. The performance of the
system was evaluated using both the NCHLT Sepedi test corpus and the Sepedi
Prompted Code Switching corpus. The model produced the lowest WER of 41.9%,
however, the model faced challenges in recognizing the Sepedi only text.Comment: 13 pages,2 figures,2nd International Conference on NLP & AI (NLPAI
2024
Lwazi II Sotho Pronunciation Dictionaries
No full textPronunciation dictionaries for Sepedi, Sesotho and Setswana with and without affricates, as well as the maps that were used to split the affricates into their constituting sounds. Note that these dictionaries were developed based on the Lwazi I dictionaries, available from the South African RMA (http://rma.nwu.ac.za/)
Predicting vowel substitution in code-switched speech
No full textAbstract—The accuracy of automatic speech recognition
(ASR) systems typically degrades when encountering codeswitched
speech. Some of this degradation is due to the
unexpected pronunciation effects introduced when languages
are mixed. Embedded (foreign) phonemes typically show more
variation than phonemes from the matrix language: either
approximating the embedded language pronunciation fairly
closely, or realised as any of a set of phonemic counterparts
from the matrix language. In this paper we describe a technique
for predicting the phoneme substitutions that are expected
to occur during code-switching, using non-acoustic features
only. As case study we consider Sepedi/English code switching
and analyse the different realisations of the English schwa.
A code-switched speech corpus is used as input and vowel
substitutions identified by auto-tagging this corpus based on
acoustic characteristics. We first evaluate the accuracy of our
auto-tagging process, before determining the predictability of
our auto-tagged corpus, using non-acoustic features.This work was partially supported by the National Research
Foundation. Any opinion, findings and conclusions or
recommendations expressed in this material are those of the
author(s) and therefore the NRF do not accept any liability
in regard thereto
NCHLT isiNdebele Speech Corpus
No full textOrthographically transcribed broadband speech corpus of approximately 56 hours, including a test suite of 8 speakers
Transformer-based Text Generation for Code-Switched Sepedi-English News
No full text<p>Code-switched data is rarely available in written form and this makes the development of large datasets required to train codeswitched language models difficult. Currently, available Sepedi-English code-switched corpora are not large enough to train a Transformer-based model for this language pair. In prior work, larger synthetic datasets have been constructed using a combination of a monolingual and a parallel corpus to approximate authentic code-switched text. In this study, we develop and analyse a new Sepedi-English news dataset (SepEnews). We collect and curate data from local radio news bulletins and use this to augment two existing sources collected from Sepedi newspapers and news headlines, respectively. We then develop and train a Transformer-based model for generating historic code-switched news, and demonstrate and analyse the system's performance. </p>
