Common Language Resources and Technology Infrastructure - Slovenia
Not a member yet
840 research outputs found
Sort by
Annotated Corpus of Pre-Standardized Balkan Slavic Literature 1.1
Get PDFThe corpus contains 23 linguistically annotated samples of "damaskini" and other Balkan Slavic manuscripts and print editions from the 15th-19th century, together with over 50 thousand tokens. The texts have similar prose narratives and topics, i.e. hagiographies and apocalyptic themes. The majority of the texts are various editions of the "Life of St. Petka" by Patriarch Euthymius of Tarnovo. The primary goal of the corpus is to provide data for studies of developments in both spoken and written language in the mentioned area and period, especially the features typical for the Balkan sprachbund, e.g. postponed articles, and the analytic infinitive.
The corpus has been manually lemmatized and annotated with MULTEXT-East morphosyntactic descriptions developed specifically for this corpus (http://nl.ijs.si/ME/V6/msd/html/msd-bg-dam.html) as well as syntactically analyzed with dependency relations following the Universal Dependencies guidelines (up to Level 2 validation).The annotation reflects a wide spectrum of morphosyntactic features of both archaic (Church Slavonic) and innovative (Bulgarian, Macedonian) varieties. The corpus is available both in source .tsv and derived CoNLL-U formats. The .tsv format includes tokens including accentuation and interpunction, while CoNLL-U uses a diplomatized transcript, more useful for a full-text search.
The author would like to thank the following institutions for providing the source texts - National Library "Sv. sv. Kiril i Metodii" in Sofia, Church Historical and Archive Institute in Sofia, Archive of the Bulgarian Academy of Sciences in Sofia, Sofia University "Kliment Oxridski", National Library "Ivan Vazov" in Plovdiv, National and University Library of Slovenia in Ljubljana, Russian State Library in Moscow, Library of Matica Srpska in Novi Sad - as well as to Olivier Winistörfer for his invaluable help in preparing the corpus.
The update 1.1 includes:
- 8 new texts, with over 20k tokens
- more data: Cyrillic transcripts, lemma and cross-text references
- detailed philological and description of selected texts
- sources in plain text forma
Franček portal headword list
No full textThe headword list represents the core of the "Franček" educational portal data. The list is comprised of the most frequent headwords in the dictionaries of Slovenian language.
The dataset contains: headwords, homograph numbers, homograph-disambiguating indicators, part-of-speech information, and internal cross-references.
The dataset is cross-referenced from datasets: http://hdl.handle.net/11356/1451, http://hdl.handle.net/11356/1452 (the list is being expanded)
English YouTube Hate Speech Corpus
No full textWe present an English YouTube dataset manually annotated for hate speech types and targets. The comments to be annotated were sampled from the English YouTube comments on videos about the Covid-19 pandemic in the period from January 2020 to May 2020. Two sets were annotated: a training set with 51,655 comments (IMSyPP_EN_YouTube_comments_train.csv) and two evaluation sets, one annotated in-context (IMSyPP_EN_YouTube_comments_evaluation_context.csv), another out-of-context (IMSyPP_EN_YouTube_comments_evaluation_no_context.csv), each based on the same 10,759 comments. The dataset was annotated by 10 annotators with most (99.9%) of the comments being annotated by two annotators. It was used to train a classification model for hate speech types detection that is publicly available at the following URL: https://huggingface.co/IMSyPP/hate_speech_en.
The dataset consists of the following fields:
Video_ID - YouTube ID of the video under which the comment was posted
Comment_ID - YouTube ID of the comment
Text - text of the comment
Type - type of hate speech
Target - the target of hate speech
Annotator - code of the human annotato
The CLASSLA-StanfordNLP model for morphosyntactic annotation of standard Slovenian 1.1
No full textThis model for morphosyntactic annotation of standard Slovenian was built with the CLASSLA-StanfordNLP tool (https://github.com/clarinsi/classla-stanfordnlp) by training on the ssj500k training corpus (http://hdl.handle.net/11356/1210) and using the CLARIN.SI-embed.sl word embeddings (http://hdl.handle.net/11356/1204). The model produces simultaneously UPOS, FEATS and XPOS (MULTEXT-East) labels. The estimated F1 of the XPOS annotations is ~97.06.
The difference to the previous version of the model is that now the whole XPOS tag is predicted and not specific characters, as was the case in stanfordnlp, which resulted in illegal XPOS tags (and slightly decreased performance)
The CLASSLA-StanfordNLP model for lemmatisation of standard Slovenian 1.2
No full textThe model for lemmatisation of standard Slovenian was built with the CLASSLA-StanfordNLP tool (https://github.com/clarinsi/classla-stanfordnlp) by training on the ssj500k training corpus (http://hdl.handle.net/11356/1210) and using the Sloleks inflectional lexicon (http://hdl.handle.net/11356/1230). The estimated F1 of the lemma annotations is ~99.0.
The difference to the previous version is that now it relies solely on XPOS annotations, and not on a combination of UPOS, FEATS (lexicon lookup) and XPOS (lemma prediction) annotations
Frequency lists of word-level n-grams from the GOS 1.0 corpus 1.1
No full textFrequency lists of word-level n-grams (or word sets) were extracted from the GOS 1.0 Corpus of Spoken Slovene (http://hdl.handle.net/11356/1040) using the LIST corpus extraction tool (http://hdl.handle.net/11356/1227). The lists contain all word-level 2-, 3-, 4- and 5-grams occurring in the corpus along with their absolute and relative frequencies, percentages, distribution across the text-types included in the corpus taxonomy, and five collocation measures: Dice, t-score, MI, MI3, logDice, and simple LL.
The n-grams were extracted from lower-case word forms, standardized word forms, and morphosyntactic tags.
For large lists, shortened versions with the first 150,000 lines were also prepared to facilitate further processing in spreadsheet analysis software.
Compared to the previous version (http://hdl.handle.net/11356/1271), this one includes fixes of several typos and substitutes all instances of "normalized forms" with the more adequate term "standardized forms" (as used in the SSJ project)
Annotated Corpus of Pre-Standardized Balkan Slavic Literature
Get PDFThe corpus contains 15 linguistically annotated samples of "damaskini" and other Balkan Slavic manuscripts and print editions from the 16th-19th century, together with over 30 thousand tokens. The texts have similar prose narratives and topics, i.e. hagiographies and apocalyptic themes. The majority of the texts are various editions of the "Life of St. Petka" by Patriarch Euthymius of Tarnovo. The primary goal of the corpus is to provide data for studies of developments in both spoken and written language in the mentioned area and period, especially the features typical for the Balkan sprachbund, e.g. postponed articles, and the analytic infinitive.
The corpus has been manually lemmatized and annotated with MULTEXT-East morphosyntactic descriptions developed specifically for this corpus (http://nl.ijs.si/ME/V6/msd/html/msd-bg-dam.html) as well as syntactically analyzed with dependency relations following the Universal Dependencies guidelines (up to Level 2 validation).The annotation reflects a wide spectrum of morphosyntactic features of both archaic (Church Slavonic) and innovative (Bulgarian, Macedonian) varieties. The corpus is a vailable both in source TSV and derived CoNLL-U formats. The TSV format includes tokens including accentuation and interpunction, while CoNLL-U uses a diplomatized transcript, more useful for a full-text search.
The author would like to thank Olivier Winistörfer for his invaluable help in preparing the corpus
The CLASSLA-StanfordNLP model for lemmatisation of standard Macedonian 1.0
No full textThe model for lemmatisation of standard Macedonian was built with the CLASSLA-StanfordNLP tool (https://github.com/clarinsi/classla-stanfordnlp) by training on the 1984 training corpus (to be published). The estimated F1 of the lemma annotations is ~99.1
Semantic hypergraph corpus SemCRO 1.0
No full textThis corpus can be used to build and evaluate methods for extracting and presenting knowledge based on a semantic hypergraph. The corpus consists of 184 simple, complex and dependently complex sentences. All sentences are marked on the levels of tokenisation, sentence segmentation, morphosyntactic tagging, lemmatisation, syntactic dependencies, named entities, and semantic roles. This resource also includes, a representation of a subset of 176 sentences in the form of a semantic hypergraph that can be used to evaluate knowledge extraction methods for Croatian.
The sentences used in this corpora are taken from the textbook:
Hudeček, L., Mihaljević, M., Sršen, J. and Čamagajevac, S. (2017). Hrvatska Školska Gramatika. Zagreb: Institut za hrvatski jezik i jezikoslovlje. https://gramatika.hr/impresum
Lexikon of Viennese Dialect - Schranka (1905) (ELEXIS)
No full textWiener Dialekt-Lexikon.
Digitised version of a historic dialect dictionary of Viennese