Common Language Resources and Technology Infrastructure - Slovenia
Not a member yet
840 research outputs found
Sort by
Word embeddings CLARIN.SI-embed.sl 2.0
No full textCLARIN.SI-embed.sl contains word embeddings induced from a large collection of Slovene texts composed of existing corpora of Slovene, e.g GigaFida, Janes, KAS, slWaC, MaCoCu-sl, etc. The embeddings are based on the skip-gram model of fastText trained on 5,791,405,942 tokens of running text for 3,471,054 lowercased surface forms.
The difference to the previous version of the embeddings is that this version was trained on the original dataset expanded with the MaCoCu-sl web crawl corpus (http://hdl.handle.net/11356/1517)
The CLASSLA-Stanza model for morphosyntactic annotation of standard Slovenian 2.0
No full textThis model for morphosyntactic annotation of standard Slovenian was built with the CLASSLA-Stanza tool (https://github.com/clarinsi/classla) by training on the SUK training corpus (http://hdl.handle.net/11356/1747) and using the CLARIN.SI-embed.sl word embeddings (http://hdl.handle.net/11356/1204) that were expanded with the MaCoCu-sl Slovene web corpus (http://hdl.handle.net/11356/1517). The model produces simultaneously UPOS, FEATS and XPOS (MULTEXT-East) labels. The estimated F1 of the XPOS annotations is ~98.27.
The difference to the previous version of the model is that the model was trained using the SUK training corpus and uses new embeddings and the new version of the Slovene morphological lexicon Sloleks 3.0 (http://hdl.handle.net/11356/1745)
The CLASSLA-Stanza model for semantic role labeling of standard Slovenian 2.0
No full textThe model for semantic role labeling of standard Slovenian was built with the CLASSLA-Stanza tool (https://github.com/clarinsi/classla) by training on the SUK training corpus (http://hdl.handle.net/11356/1747) and using the CLARIN.SI-embed.sl word embeddings (http://hdl.handle.net/11356/1204) extended with the MaCoCu-sl Slovenian web corpus (http://hdl.handle.net/11356/1517). The estimated F1 of the semantic role annotations is ~76.24.
The difference to the previous version of the model is that the model was trained using the SUK training corpus and the updated word embeddings
The CLASSLA-Stanza model for lemmatisation of non-standard Croatian 2.1
No full textThe model for lemmatisation of non-standard Croatian was built with the CLASSLA-Stanza tool (https://github.com/clarinsi/classla) by training on the hr500k training corpus (http://hdl.handle.net/11356/1792) and the ReLDI-NormTagNER-hr corpus (http://hdl.handle.net/11356/1793), using the hrLex inflectional lexicon (http://hdl.handle.net/11356/1232). These corpora were additionally augmented for handling missing diacritics by repeating parts of the corpora with diacritics removed. The estimated F1 of the lemma annotations is ~94.23.
The difference to the previous version of the model is that this version is trained on a combination of two corpora (hr500k, ReLDI-NormTagNER-hr)
The CLASSLA-Stanza model for UD dependency parsing of standard Croatian 2.1
No full textThe model for UD dependency parsing of standard Croatian was built with the CLASSLA-Stanza tool (https://github.com/clarinsi/classla) by training on the UD-parsed portion of the hr500k training corpus (http://hdl.handle.net/11356/1792) and using the CLARIN.SI-embed.hr word embeddings (http://hdl.handle.net/11356/1790). The estimated LAS of the parser is ~87.46.
The difference to the previous version of the model is that this version was trained using the new version of the hr500k corpus and the new version of the Croatian word embeddings
Linguistically annotated multilingual comparable corpora of parliamentary debates ParlaMint.ana 4.0
No full textParlaMint 4.0 is a set of comparable corpora containing transcriptions of parliamentary debates of 29 European countries and autonomous regions, mostly starting in 2015 and extending to mid-2022. The individual corpora comprise between 9 and 126 million words and the complete set contains over 1.1 billion words.
The transcriptions are divided by days with information on the term, session and meeting, and contain speeches marked by the speaker and their role (e.g. chair, regular speaker). The speeches also contain marked-up transcriber comments, such as gaps in the transcription, interruptions, applause, etc. The corpora have extensive metadata, most importantly on speakers (name, gender, MP and minister status, party affiliation), the political parties and parliamentary groups (name, coalition/opposition status, Wikipedia-sourced left-to-right political orientation, and CHES variables, https://www.chesdata.eu/). Note that some corpora have further metadata, e.g. the year of birth of the speakers, links to their Wikipedia articles, their membership in various committees, etc. The transcriptions are also marked with the subcorpus they belong to ("reference", until 2020-01-30, "covid", from 2020-01-31, and "war", from 2022-02-24).
The corpora are encoded according to the Parla-CLARIN TEI recommendation (https://clarin-eric.github.io/parla-clarin/), but have been encoded against the compatible, but much stricter ParlaMint encoding guidelines (https://clarin-eric.github.io/ParlaMint/) and schemas (included in the distribution).
The ParlaMint.ana linguistic annotation includes tokenization; sentence segmentation; lemmatisation; Universal Dependencies part-of-speech, morphological features, and syntactic dependencies; and the 4-class CoNLL-2003 named entities. Some corpora also have further linguistic annotations, in particular PoS tagging according a language-specific scheme, with their corpus TEI headers giving further details on the annotation vocabularies and tools used.
This entry contains the ParlaMint.ana TEI-encoded linguistically annotated corpora; the derived CoNLL-U files along with TSV metadata of the speeches; and the derived vertical files (with their registry file), suitable for use with CQP-based concordancers, such as CWB, noSketch Engine or KonText. Also included is the 4.0 release of the sample data and scripts available at the GitHub repository of the ParlaMint project at https://github.com/clarin-eric/ParlaMint and the log files produced in the process of building the corpora for this release. The log files show e.g. known errors in the corpora, while more information about known problems is available in the (open) issues at the GitHub repository of the project.
This entry contains the linguistically marked-up version of the corpus, while the text version, i.e. without the linguistic annotation is available at http://hdl.handle.net/11356/1859.
Another related resource is the Linguistically annotated multilingual comparable corpora of parliamentary debates in English ParlaMint-en.ana 4.0 (http://hdl.handle.net/11356/1864).
As opposed to the previous version 3.0, this version adds corpora for Spain (ES), Finland (FI) and the Basque Country (ES-PV); extends the corpora for Austria (AT), Czechia (CZ), Hungary (HU), and Ukraine (UA) with more recent data; adds metadata to political parties and parliamentary groups on left-to-right political orientation from Wikipedia, as well as CHES variables; adds the information on whether a speaker was a minister and when for the corpora that previously lacked this information. The TEI encoding of some details has also changed, and many errors found in 3.0 corpora have been corrected. Furthermore, the vertical files (and hence the individual corpora available on the concordancers) have their meta-data in the local language of the corpus, and not English
Spoken corpus Gos 2.1 (transcriptions)
No full textThe spoken corpus Gos 2.1 is the reference speech corpus of the Slovenian language. This second edition contains about 300 hours of speech, or 2.4 million words, 127 thousand utterances and 1,500 texts, with added word-level temporal information, where available.
Gos2.1 is composed from three different sources:
(1) Spoken corpus Gos 1.1 (http://hdl.handle.net/11356/1438), 112 hours, 1 million words
(2) Spoken corpus Gos VideoLectures 4.2 (http://hdl.handle.net/11356/1444), 22 hours, 179,000 words
(3) A selection from the ASR database ARTUR 1.0 (http://hdl.handle.net/11356/1772), 185 hours, 1.2 mllion words, including:
(3a) Artur-J-Splosni, 62 hours, 422,000 words: transcriptions of media recordings, online recordings of conferences, workshops, education videos, etc.
(3b) Artur-N-Prosti, 61 hours, 324,000 words: transcriptions of monologues and dialogues between two persons, recorded for the purposes of the Artur database. Speakers were asked to freely conversate or freely explain on casual topics.
(3c) Artur-P-SejeDZ, 62 hours, 450,000 words: a selection of transcriptions of speech from the Slovene National Assembly. The maximum length of single speaker speech is 4,000 words.
All transcriptions are manual and made in two modes:
- pronunciation-based or citation-phonemic transcriptions (containing the output phoneme string derived from the orthographic form by letter-to-sound rules)
- standardised or expanded orthographic transcriptions (the standard Slovene spelling is used to indicate the spoken words, but there are additional rules and word-lists for non-standard lexis).
Part-of-speech tagging with MULTEXT-East morphosyntactic descriptions and lemmatisation was performed automatically with CLASSLA (https://github.com/clarinsi/classla).
In comparison with the preceding version, Gos 2.0, this version implements the following changes:
(1) Unification of casing and normalization decisions between Gos1.0, GosVL, and Artur subcorpora;
(2) Re-introduction of punctuation into the Artur subcorpus;
(3) Addition of word-level temporal information in form of a separate timeline element ();
(4) Re-encoding of the Gos and Gos VideoLectures subcorpora so that the encoding is uniform across all subcorpora.
The corpus is distributed in the source TRS XML (Transcriber) format, in the canonical TEI XML encoding, as well as TEI-derived plain text (TXT) and vertical (vert) format.
The corresponding audio (and, partially, video) files are available under a restricted licence from http://hdl.handle.net/11356/1973
Word embeddings CLARIN.SI-embed.hr 2.0
No full textCLARIN.SI-embed.hr contains word embeddings induced from a large collection of Croatian texts composed of the Croatian web corpus hrWaC, a 400-million-token-heavy collection of newspaper texts and MaCoCu-hr. The embeddings are based on the skip-gram model of fastText trained on 4,586,769,197 tokens of running text for 3,406,574 lowercased surface forms.
The difference to the previous version of the embeddings is that this version was trained on the original dataset expanded with the MaCoCu-hr web crawl corpus (http://hdl.handle.net/11356/1516)
The CLASSLA-Stanza model for JOS dependency parsing of standard Slovenian 2.0
No full textThis model for JOS dependency parsing of standard Slovenian was built with the CLASSLA-Stanza tool (https://github.com/clarinsi/classla) by training on the SUK training corpus (http://hdl.handle.net/11356/1747) and using the CLARIN.SI-embed.sl word embeddings (http://hdl.handle.net/11356/1204) expanded with the MaCoCu-sl Slovene web corpus (http://hdl.handle.net/11356/1517). The estimated LAS of the parser is ~93.89.
The difference to the previous version of the model is that the model was trained using the SUK training corpus and uses the updated embeddings
Multilingual comparable corpora of parliamentary debates ParlaMint 3.0
No full textParlaMint 3.0 is a multilingual set of 26 comparable corpora containing parliamentary debates mostly starting in 2015 and extending to mid-2022, with the individual corpora being between 9 and 125 million words in size.
The corpora have extensive metadata, including aspects of the parliament; the speakers (name, gender, MP status, party affiliation, party coalition/opposition); are structured into time-stamped terms, sessions and meetings; and with speeches being marked by the speaker and their role (e.g. chair, regular speaker). The speeches also contain marked-up transcriber comments, such as gaps in the transcription, interruptions, applause, etc. Note that some corpora have further information, e.g. the year of birth of the speakers, links to their Wikipedia articles, their membership in various committees, etc. The corpora are also marked to the subcorpus they belong to ("reference", until 2020-01-30, "covid", from 2020-01-31, and "war", from 2022-02-24).
The corpora are encoded according to the Parla-CLARIN TEI recommendation (https://clarin-eric.github.io/parla-clarin/), but have been encoded against the compatible, but much stricter ParlaMint encoding guidelines (https://clarin-eric.github.io/ParlaMint/) and schemas (included in this distribution.
This entry contains the ParlaMint TEI-encoded corpora with the derived plain text versions of the corpora along with TSV metadata of the speeches. Also included is the 3.0 release of the data and scripts available at the GitHub repository of the ParlaMint project.
Note that there also exists the linguistically marked-up version of the corpus, which is available at http://hdl.handle.net/11356/1488.
As opposed to the previous version 2.1, this version extends the corpus dates to (at least) mid 2022, does not contain the corpora for ES (Spanish) and Lithuanian (LT), and adds corpora for AT (Austria), BA (Bosnian), ES-CT (Catalonia), ES-GA (Galicia), GR (Greece), NO (Norway), PT (Portugal), RS (Serbia), SE (Sweden), and UA (Ukraine). The TEI encoding of some details has also changed