Common Language Resources and Technology Infrastructure - Slovenia
Not a member yet
840 research outputs found
Sort by
The Twitter user dataset for discriminating between Bosnian, Croatian, Montenegrin and Serbian Twitter-HBS 1.0
No full textThe Twitter-HBS dataset consists of Twitter users, their tweets, and the label of their predominantly used language - Bosnian, Croatian, Montenegrin, or Serbian. Among the tweets, there are also tweets in other languages (mainly English) as the label encodes the predominantly used language of a user only. The main intended usage of this dataset is discrimination between closely-related languages on the level of a Twitter user (not a single tweet). The only pre-processing performed on the texts of the tweets is the transliteration from the Cyrillic into the Latin script so that the dataset measures the quality of the user classifications regardless of the script used
Corpus of textbooks for learning Slovenian as L2 KUUS 1.0
No full textThe KUUS corpus comprises 17 textbooks for Slovenian as a second and foreign language published between 2002 and 2022 at the Centre for Slovene as a Second and Foreign Language (Faculty of Arts, University of Ljubljana). These textbooks were widely used in the teaching of Slovenian as a second and foreign language to children, adolescents and adults in Slovenia and abroad at the time of the creation of the corpus. The KUUS consists of 520,796 words. It was linguistically annotated with the CLASSLA v1.1.1 pipeline (https://github.com/clarinsi/classla/) at the levels of tokenization, sentence segmentation, lemmatization, MULTEXT-East v6 MSD-tags (https://nl.ijs.si/ME/V6/msd/html/msd-sl.html), JOS dependency syntax (https://nl.ijs.si/jos/bib/jos-skladnja-navodila.pdf), and named entities (https://nl.ijs.si/janes/wp-content/uploads/2017/09/SlovenianNER-eng-v1.1.pdf). The metadata for each of the textbooks includes the information about the title, subtitle, authors, year of publication, publisher, CEFR level, target audience, and the estimated number of lessons for the textbook.
The corpus is presented in more detail in: KLEMEN, Matej, ARHAR HOLDT, Špela, POLLAK, Senja, KOSEM, Iztok, HUBER, Damjan, LUTAR, Mateja, 2022: Korpus učbenikov za učenje slovenščine kot drugega in tujega jezika. Nataša Pirih Svetina, Ina Ferbežar (eds.): Na stičišču svetov: slovenščina kot drugi in tuji jezik. Obdobja 41. Ljubljana: Založba Univerze v Ljubljani. 165–174. DOI: https://doi.org/10.4312/Obdobja.41.2784-715
Core vocabulary for Slovenian as L2 1.0
No full textThe Core vocabulary for Slovenian as L2 is based on an analysis of the vocabulary appearing in the KUUS corpus (http://hdl.handle.net/11356/1696), which includes textbooks for Slovenian as a second and foreign language. By exporting lemmas, comparing them with the Reference list of Slovene frequent common words (Pollak et al. 2020, http://hdl.handle.net/11356/1346) and manual review, a list of 5273 words was compiled. The lemmas were classified into the first three CEFR levels. The list includes 350 words with the assigned label A1-core, 864 words with the label A1-larger, 1451 words with the label A2 and 2608 words at level B1. The file is in a tab separated format, containing lemma, part-of-speech (following the MULTEXT-East tagset for Slovenian), the information if the lemma appears in the Reference List of Slovene Frequent Common Words or not, and the relative average frequency.
The word lists are presented in more detail in: KLEMEN, Matej, ARHAR HOLDT, Špela, POLLAK, Senja, KOSEM, Iztok, HUBER, Damjan, LUTAR, Mateja, 2022: Korpus učbenikov za učenje slovenščine kot drugega in tujega jezika. Nataša Pirih Svetina, Ina Ferbežar (eds.): Na stičišču svetov: slovenščina kot drugi in tuji jezik. Obdobja 41. Ljubljana: Založba Univerze v Ljubljani. 165–174. DOI: https://doi.org/10.4312/Obdobja.41.2784-7152
Frequency list of words from the Trendi corpus 2019
No full textThis frequency list of words was prepared by extracting words (i.e. lemmas with their lexical features) from the Trendi Monitor Corpus of Slovene (see e.g. http://hdl.handle.net/11356/1590) covering the period between 1 January 2019 and 31 December 2019 using the LIST corpus extraction tool (http://hdl.handle.net/11356/1227). The Trendi frequency list was then compared to the frequency list of words from the Gigafida 2.0 corpus of Slovene, which covers the period between 1991 and 2018. The words were compared using the simple maths formula implemented by SketchEngine (see https://www.sketchengine.eu/documentation/simple-maths/).
The final list contains lemmas, their lexical features, their absolute and relative frequencies from the first (1991–2018) and second periods (2019), and the simple maths value indicating if the word is more frequent in 2019 (simple maths > 1.00) or in 1991–2018 (simple maths < 1.00)
Semantic change detection datasets for Slovenian 1.0
No full textThis dataset is meant for evaluation of systems for semantic change detection in Slovenian.
The entry contains the following files:
1) "gigafida_to_1997_vs_2018.tsv" - contains sources from the Gigafida 2.0 reference corpus (http://www.gigafida.net/), dating either from year 1997 (or earlier) or year 2018. The corpus, which can be used for training, domain adaptation or word representation extraction is in a .tsv format with 4 columns:
- 'title': Title of the text
- 'publisher': Text's publisher name
- 'date': Year of the text's publishing
- 'type': Text's type (e.g., whether text was scraped from the internet or it appeared in print)
- 'text': Text in non-processed form
2) "word_usage_annotations_1997_2018.tsv" - contains example word usages for 105 predefined words. For each word, we extract from the Gigafida 2.0 corpus 30 usage examples (sentences) from year 1997 and 30 usage examples from year 2018. The sentences from both time periods are randomly matched (e.g. each pair contains a random sentence from 1997 and a random sentence from 2018, both containing the same target word), resulting in 3150 sentence pairs. These pairs were annotated by three human annotators on a scale from 1 to 4:
1: usages in the sentences are unrelated
2: usages in the sentences are distantly related
3: usages in the sentences are closely related
4: usages are identical, i.e. they have the same sense
Label 0 was also allowed, meaning "I can't decide", e.g. due to insufficient context.
The file in the .tsv format contains the following columns:
- 'id': id of the sentence pair
- 'word': target word
- 'sentence 1997': sentence from year 1997
- 'sentence 2018': sentence from year 2018
- 'score_anno1': score given by annotator 1
- 'score_anno2': score given by annotator 2
- 'score_anno3': score given by annotator 3
3) "semantic_shift_scores.tsv": contains final "gold standard" scores for each word, obtained by averaging scores across sentence pairs and across all three annotators in order to obtain a single numerical value for each word in the list. The examples containing zeros were excluded and the word 'zenit' was excluded from the list due to too many sentence pairs containing zeros.
The file in the .tsv format contains the following columns:
- 'word': target word
- 'score': semantic change score
4) "RSDO_semanticni-premiki_navodila_v0.pdf": annotation guidelines (in Slovenian
Dictionary of the Serbian Older Literature (ELEXIS)
No full textРјечник из књижевних старина српских (Dictionary of Serbian Literary Antiquity) is a historical dictionary by Serbian philologist, historian of language and lexicographer Đuro Daničić
Extensions to the Slovene translation of SuperGLUE
No full textSuperGLUE is a benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a public leaderboard. It is comprised of 8 corpora (BoolQ, CB, COPA, MultiRC, ReCoRD, RTE, WiC, WSC), which cover 4 different types of tasks (QA, NLI, WSD, coref.). Slovene translation of SuperGLUE consists of machine and human translations of the benchmark. ReCoRD is completely translated by the Google Machine Translation service. Questions and answers from the project "Slovene in the Palm of your Hand (Slovenščina na dlani)" are also included for the BoolQ, MultiRC and ReCoRD tasks and are in form of extensions to the existing datasets. The data is provided in jsonl format
Text classification model fastText-Trendi-Topics 1.0
No full textThe fastText-Trendi-Topics model is a text classification model for categorizing news texts with one of 13 topic labels. It was trained on a set of approx. 36,000 Slovene texts from various Slovene news sources included in the Trendi Monitor Corpus of Slovene (http://hdl.handle.net/11356/1590) such as "rtvslo.si", "sta.si", "delo.si", "dnevnik.si", "vecer.com", "24ur.com", "siol.net", "gorenjskiglas.si", etc.
The texts were semi-automatically categorized into 13 categories based on the sections under which they were published (i.e. URLs). The set of labels was developed in accordance with related categorization schemas used in other corpora and comprises the following topics: "črna kronika" (crime and accidents), "gospodarstvo, posel, finance" (economy, business, finance), "izobraževanje" (education), "okolje" (environment), "prosti čas" (free time), "šport" (sport), "umetnost, kultura" (art, culture), "vreme" (weather), "zabava" (entertainment), "zdravje" (health), "znanost in tehnologija" (science and technology), "politika" (politics), and "družba" (society). The categorization process is explained in more detail in Kosem et al. (2022): https://nl.ijs.si/jtdh22/pdf/JTDH2022_Kosem-et-al_Spremljevalni-korpus-Trendi.pdf
The model was trained on the labeled texts using the word embeddings CLARIN.SI-embed.sl 1.0 (http://hdl.handle.net/11356/1204) and validated on a development set of 1,293 texts using the fastText library, 1000 epochs, and default values for the rest of the hyperparameters (see https://github.com/TajaKuzman/FastText-Classification-SLED for the full code).
The model achieves a macro-F1-score of 0.85 on a test set of 1,295 texts (best for "vreme" at 0.97, worst for "prosti čas" at 0.67).
Please note that the SloBERTa-Trendi-Topics 1.0 text classification model is also available (http://hdl.handle.net/11356/1709) that achieves higher classification accuracy, but is slower and computationally more demanding
Frequency lists of word-level n-grams from the Trendi corpus 2021
No full textFrequency lists of word-level n-grams (or word sets) were extracted from the Trendi Monitor Corpus of Slovene (version 2022-05: http://hdl.handle.net/11356/1590) using the LIST corpus extraction tool (http://hdl.handle.net/11356/1227). The lists contain all word-level 2-, 3-, 4- and 5-grams with minimum relative frequency of 2 per million occurring in the corpus in texts published in 2021, along with their absolute and relative frequencies and percentages.
The n-grams were extracted from lower-case word forms along with lemmas and morphosyntactic tags.
For frequency lists of n-grams extracted from texts from previous years (e.g. 2019 and 2020), please refer to earlier versions of this entry
Lexicon Palaeoslovenico-Graeco-Latinum: Digital edition (ELEXIS)
No full textLexicon Palaeoslovenico-Graeco-Latinum is the digitised version of the resource first published in 1865. Although written a century and a half ago, it is still one of the most relevant dictionaries in Early Slavic studies. It consists of 1,172 pages with approximately 42,000 words and their equivalents in Greek and Latin