Common Language Resources and Technology Infrastructure - Slovenia
Not a member yet
840 research outputs found
Sort by
Multilingual dataset of COVID tweets for relation-level metaphor analysis TCMeta 1.0
No full textTCMeta is a dataset of noun phrase constructions from COVID-related tweets, annotated for relation-level metaphor.
It contains 2,138 Slovene and 2,221 English instances in tab-separated tabular format .tsv, where each line presents a unique phrase under consideration, extracted from a COVID-related tweet. The primary annotations include the COVID metaphor label (whether the phrase expresses a metaphor relating to COVID), but also additional ones for idioms, metaphors not relating to COVID, or metaphors not evident on the relation-level.
The complete user tweet could not be published due to the ToS of the then Twitter platform. We recommend retrieving the text of the tweets via their IDs using the Hydrator tool [https://github.com/docnow/hydrator] or similar.
The dataset is further described in:
Brglez, M., Zayed, O. & Buitelaar, P. TCMeta: a multilingual dataset of COVID tweets for relation-level metaphor analysis. Lang Resources & Evaluation 59, 437–475 (2025). https://doi.org/10.1007/s10579-024-09725-z.
@article{brglez2025tcmeta,
title={{TCMeta}: a multilingual dataset of {COVID} tweets for relation-level metaphor analysis},
author={Brglez, Mojca and Zayed, Omnia and Buitelaar, Paul},
journal={Language Resources and Evaluation},
pages={437--475},
volume={59},
year={2025},
publisher={Springer},
doi = {10.1007/s10579-024-09725-z}
Spoken corpus Gos 2.0 (transcriptions)
No full textThe spoken corpus Gos 2.0 is the reference speech corpus of the Slovenian language. This second edition contains about 300 hours of speech, or 2.4 million words, 127 thousand utterances and 1,500 texts.
Gos 2.0 is composed from three different sources:
(1) Spoken corpus Gos 1.1 (http://hdl.handle.net/11356/1438), 112 hours, 1 million words
(2) Spoken corpus Gos VideoLectures 4.2 (http://hdl.handle.net/11356/1444), 22 hours, 179,000 words
(3) A selection from the ASR database ARTUR 1.0 (http://hdl.handle.net/11356/1772), 185 hours, 1.2 mllion words, including:
(3a) Artur-J-Splosni, 62 hours, 422,000 words: transcriptions of media recordings, online recordings of conferences, workshops, education videos, etc.
(3b) Artur-N-Prosti, 61 hours, 324,000 words: transcriptions of monologues and dialogues between two persons, recorded for the purposes of the Artur database. Speakers were asked to freely conversate or freely explain on casual topics.
(3c) Artur-P-SejeDZ, 62 hours, 450,000 words: a selection of transcriptions of speech from the Slovene National Assembly. The maximum length of single speaker speech is 4,000 words.
Note that various encoding changes have been made to the original Gos and Gos VideloLectures corpora so that the encoding of Gos 2.0 is uniform across the three sources.
All transcriptions are manual and made in two modes:
- pronunciation-based or citation-phonemic transcriptions (containing the output phoneme string derived from the orthographic form by letter-to-sound rules)
- standardised or expanded orthographic transcriptions (the standard Slovene spelling is used to indicate the spoken words, but there are additional rules and word-lists for non-standard lexis).
Part-of-speech tagging with MULTEXT-East morphosyntactic descriptions and lemmatisation was performed automatically with CLASSLA (https://github.com/clarinsi/classla).
The corpus is distributed in TEI (XML) format and in vertical file format, the latter used by the CQP familiy of concordancers, such as (no)Sketch Engine
ASR database ARTUR 1.0 (transcriptions)
No full textArtur 1.0 is a speech database designed for the needs of developing automatic speech recognition for the Slovenian language. The complete database includes 1,067 hours of speech, of which 884 hours are transcribed, while the remaining 183 hours are recordings only. This repository entry includes transcriptions only, while the audio files are available on http://hdl.handle.net/11356/1776.
Transcriptions are available in the original TRS format of the Transcriber 1.5.1 tool which was used for making the transcriptions. All transcriptions were made manually or manually corrected.
The data are structured as follows:
(1) Artur-B, read speech, 573 hours in total. It includes:
(1a) Artur-B-Brani, 485 hours: Readings of sentences which were pre-selected from a 10% increment in the Gigafida 2.0 corpus. The sentences were chosen in such a way that they reflect the natural or the actual distribution of triphones in the words. They were distributed between 1,000 speakers, so that we recorded approx. 30 min in read form from each speaker. The speakers were balanced according to gender, age, region, and a small proportion of speakers were non-native speakers of Slovene. Each sentence is its own transcription file and has a corresponding audio file.
(1b) Artur-B-Crkovani, 10 hours: Spellings. Speakers were asked to spell abbreviations and personal names and surnames, all chosen so that all Slovene letters were covered, plus the most common foreign letters. The transcriptions were corrected manually.
(1c) Artur-B-Studio, 51 hours: Designed for the development of speech synthesis. The sentences were read in a studio by a single speaker. Each sentence is its own transcription file and has a corresponding recording.
(1d) Artur-B-Izloceno, 27 hours: in trs format only. The recordings that correspond to these transcriptions include different types of errors, typically, incorrect reading of sentences or a noisy environment.
(2) Artur-J, public speech, 62 hours in total. It includes:
(2a) Artur-J-Splosni, 62 hours: manual transcriptions of media recordings, online recordings of conferences, workshops, education videos, etc.
Transcriptions were made in two modes:
- 'pog' files include the pronunciation-based or citation-phonemic transcriptions (containing the output phoneme string derived from the orthographic form by letter-to-sound rules)
- 'std' files include standardised or expanded orthographic transcriptions (the standard Slovenian spelling is used to indicate the spoken words, but there are additional rules and word-lists for non-standard lexis)
(3) Artur-N, private speech, 74 hours in total. It includes:
(3a) Artur-N-Obrazi, 6 hours: Speakers were asked to describe faces on pictures. Designed for a face-description domain-specific speech recognition.
(3b) Artur-N-PDom, 7 hours: Speakers were asked to read pre-written sentences, as well as to express instructions for a potential smart-home system freely. Designed for a smart-home domain-specific speech recognition.
(3c) Artur-N-Prosti, 61 hours: Monologues and dialogues between two persons, recorded for the purposes of the Artur database creation. Speakers were asked to conversate or explain freely on casual topics.
The manual transcriptions were done in two modes, the same as for Artur-J.
(4) Artur-P, parliamentary speech, 201 hours in total. It includes:
(4a) Artur-P-SejeDZ, 201 hours: Transcriptions of speech from the Slovene National Assembly.
Manual transcriptions were done in two modes, the same as for Artur-J.
Further information on the database, including various statistics, are available in the Artur-DOC directory, which is part of Artur_1.0_TRS
Icelandic web corpus MaCoCu-is 1.0
No full textThe Icelandic web corpus MaCoCu-is 1.0 was built by crawling the ".is" internet top-level domain in 2021, extending the crawl dynamically to other domains as well. The crawler is available at https://github.com/macocu/MaCoCu-crawler.
Considerable efforts were devoted into cleaning the extracted text to provide a high-quality web corpus. This was achieved by removing boilerplate (https://corpus.tools/wiki/Justext) and near-duplicated paragraphs (https://corpus.tools/wiki/Onion), discarding very short texts as well as texts that are not in the target language. The dataset is characterized by extensive metadata which allows filtering the dataset based on text quality and other criteria (https://github.com/bitextor/monotextor), making the corpus highly useful for corpus linguistics studies, as well as for training language models and other language technologies.
In the XML format, each document is accompanied by the following metadata: title, crawl date, url, domain, file type of the original document, distribution of languages inside the document, and a fluency score based on a language model. The text of each document is divided into paragraphs that are accompanied by metadata on the information whether a paragraph is a heading or not, metadata on the paragraph quality and fluency, the automatically identified language of the text in the paragraph, and information whether the paragraph contains sensitive information (identified via the Biroamer tool - https://github.com/bitextor/biroamer).
The TSV format delivers sentence-level data, and contains the following metadata: sentence URL, paragraph and sentence ID within the document, a simhash and a quality score, which allow filtering out near-duplicate sentences (all sentences with the same simhash can be deleted, except for the one with the highest quality score), the language of the sentence, information on sentence fluency, and information whether the sentence contains personal or sensitive information (identified via the Biroamer sensitive data and named entity recognizer).
Notice and take down: Should you consider that our data contains material that is owned by you and should therefore not be reproduced here, please: (1) Clearly identify yourself, with detailed contact data such as an address, telephone number or email address at which you can be contacted. (2) Clearly identify the copyrighted work claimed to be infringed. (3) Clearly identify the material that is claimed to be infringing and information reasonably sufficient in order to allow us to locate the material. (4) Please write to the contact person for this resource whose email is available in the full item record. We will comply with legitimate requests by removing the affected sources from the next release of the corpus.
This action has received funding from the European Union's Connecting Europe Facility 2014-2020 - CEF Telecom, under Grant Agreement No. INEA/CEF/ICT/A2020/2278341. This communication reflects only the author’s view. The Agency is not responsible for any use that may be made of the information it contains
Corpus of Legislation texts of Republic of Serbia 1.0
No full textThe dataset was created using a large number of Serbian Legislation texts gathered from the https://www.pravno-informacioni-sistem.rs/ website. The gathered texts were used for fine-tuning a neural network called SRBerta on the masked language modeling task. The dataset contains texts which are part of the following legislation categories:
• Constitution of the Republic of Serbia and state regulation
• Justice
• Defense, military and internal affairs
• Public incomes
• Monetary system, financial organizations and busines
Annotated sample of the Slovenian Biographical Lexicon SBL-51abbr 1.0
No full textThis dataset consists of 51 randomly selected entries from the Slovenian Biographical Lexicon (1925–1991). The text of each entry has been manually tokenised and sentence segmented, marked with named entities and the words lemmatised. It has also been automatically annotated with PoS tags (MULTEXT-East morphosyntactic descriptions) and Universal Dependencies PoS tags, morphological features and dependency parses.
Crucially for the envisaged use of the corpus, the abbreviations in the corpus (of which there are 2,041) have been manually expanded so that the expanded abbreviations are also in the correct inflected form, given their context.
The corpus is available in the canonical TEI encoding, and derived plain text and CoNLL-U files. The plain-text file has abbreviations and their expansions marked up with [[...]]((...)). There are two CoNLL-U files, one with the text stream with abbreviations, and one with the text stream with expansions. Note that only the one with expansions has syntactic parses. Both CoNLL-U files have the expansions / abbreviations and named entities marked up in IOB format in the last column
Slovenian keyword extraction dataset from SentiNews 1.0
No full textThe dataset consists of 7514 Slovenian news articles from the SentiNews 1.0 corpus by Bučar et al. 2017 (http://hdl.handle.net/11356/1110) which had available article keywords.
We provide the train and test data splits (5995 articles for training and 1519 for testing) that can be used for keyword extraction experiments.
The format is a json file, containing the following fields: title, keywords, lang (always Slovene) and body (with the content of the article).
In our paper we addressed keyword extraction in a cross-lingual setting:
Koloski, Boshko, et al. "Out of Thin Air: Is Zero-Shot Cross-Lingual Keyword Detection Better Than Unsupervised?." arXiv preprint arXiv:2202.06650 (2022). [https://arxiv.org/pdf/2202.06650.pdf]
For reproducing the results, you can use keyword datasets from the dataset http://hdl.handle.net/11356/1403 described in:
Koloski, B., Pollak, S., Škrlj, B., & Martinc, M. (2021). Extending Neural Keyword Extraction with TF-IDF tagset matching. In: Proceedings of the EACL Hackashop on News Media Content Analysis and Automated Report Generation, Kiev, Ukraine, pages 22–29
Developmental corpus Šolar 3.0
No full textThe Developmental corpus Šolar consists of 5,485 texts written by students in Slovenian secondary schools (age 15-19) and pupils in the 7th-9th grade of primary school (13-15), with a small percentage also from the 6th grade. The information on school (elementary or secondary), subject, level (grade or year), type of text, region, and date of production is provided for each text. School essays form the majority of the corpus while other material includes texts created during lessons, such as text recapitulations or descriptions, examples of formal applications, etc.
Part of the corpus (2,094 texts) is annotated with teachers' corrections using a system of labels described in the attached document (in Slovenian). Teacher corrections were part of the original files and reflect real classroom situations of essay marking. Corrections were then inserted into texts by annotators and subsequently categorized.
The corpus was linguistically annotated with the CLASSLA v1.1.1 pipeline (https://github.com/clarinsi/classla/) at the levels of tokenization, sentence segmentation, lemmatization, MULTEXT-East v6 MSD-tags (https://nl.ijs.si/ME/V6/msd/html/msd-sl.html), JOS dependency syntax (https://nl.ijs.si/jos/bib/jos-skladnja-navodila.pdf), and named entities (https://nl.ijs.si/janes/wp-content/uploads/2017/09/SlovenianNER-eng-v1.1.pdf).
The corpus is available in TEI format, where the original and corrected versions of the texts are encoded separately, while intertextual links with error labels give the relations between the two. Additionally, the corpus is available also in the CoNLL-U and JSON formats, as well as vertical files for use with Sketch Engine type concordancers.
As opposed to the previous version 2.0, which was also available in two separate versions, i.e. Šolar Clear 2.0 (http://hdl.handle.net/11356/1219), with the students' text without teacher corrections, and Šolar Error (http://hdl.handle.net/11356/1231), with only those sentences that have teacher corrections, the current version has a different encoding, error annotations were manually edited in cca. 350 texts, and the linguistic annotation was performed with a better tool
Corpus of Montenegrin language-related news articles MetaLangNEWS-Me
No full textA comprehensive corpus of news articles on the topic of language, published in major Montenegrin daily newspapers and news portals in the five-year period of January 1, 2015 - January 1, 2020. The corpus is designed to facilitate research on metalanguage (‘language about language’), linguistic ideologies, language policy and planning, as well as the specific contemporary debates on language defining, naming, and standardisation, ongoing in post-Yugoslav societies. The corpus is available in plain text version and XML with full metadata.
MetaLangNEWS-Me is complemented with a separate corpus of citizen metalanguage comments, i.e. online comments to the news articles, available as MetaLangNEWS-COMMENTS-Me (http://hdl.handle.net/11356/1689). Parallel versions from Slovenia (http://hdl.handle.net/11356/1360), Croatia (http://hdl.handle.net/11356/1369), Serbia (http://hdl.handle.net/11356/1371), North Macedonia (http://hdl.handle.net/11356/1652) and Bosnia and Herzegovina (http://hdl.handle.net/11356/1690) are also available
Corpus of questions and answers of the Terminologišče terminological counselling service
No full textTerminological counselling at the Terminologošče web site is a service intended for the expert public facing specific terminological naming problems. When elaborating terminological answers, terminologists at the Fran Ramovš Institute of the Slovenian language ZRC SAZU follow basic terminological principles, i.e. they recommend which solution would be the most appropriate from the point of view of terminological theory.
This corpus collects 509 questions from the public and answers by terminologists. Each section of the corpus gives an indicative title of the questions, the date of the question, the authors of the reply, followed by the questions and the answer. The corpus is encoded in XML according to the Text Encoding Initiative Guidelines