1,721,134 research outputs found
30 sentences annotated by 15 crowd workers (3/3)
No full text<p>These are 30 sentences annotated by 15 crowd workers each, within the context of the project Crowd Watson (http://crowd-watson.nl) for medical relation extraction.</p>
<p>Project members: Chris Welty (IBM Research), Lora Aroyo (VU University Amsterdam),</p
30 sentences annotated by 15 crowd workers (2/3)
No full text<p>These are 30 sentences annotated by 15 crowd workers each, within the context of the project Crowd Watson (http://crowd-watson.nl) for medical relation extraction.</p>
<p>Project members: Chris Welty (IBM Research), Lora Aroyo (VU University Amsterdam),</p
30 sentences annotated by 15 crowd workers (1/3)
No full text<p>These are 30 sentences annotated by 15 crowd workers each, within the context of the project Crowd Watson (http://crowd-watson.nl) for medical relation extraction. </p>
<p> </p>
<p>Project members: Chris Welty (IBM Research), Lora Aroyo (VU University Amsterdam), </p
FrameNet Semantic Frame Disambiguation with CrowdTruth
No full text<p>This repository contains a ground truth corpus for semantic frame disambiguation, acquired with crowdsourcing and processed with <strong><a href="http://crowdtruth.org/">CrowdTruth</a></strong> metrics that capture ambiguity in annotations by measuring inter-annotator disagreement.</p>
<p>The dataset contains annotations for 433 sentence-word pairs from the <a href="https://framenet.icsi.berkeley.edu/">FrameNet corpus v.1.7</a>, with each sentence-word pair annotated for frame disambiguation by 15 workers. The crowdsourced data was collected from <a href="https://www.mturk.com/">Amazon Mechanical Turk</a>.</p>
<p>The corpus has been referenced in the following paper:</p>
<ul>
<li>Anca Dumitrache, Lora Aroyo and Chris Welty: <strong><a href="https://arxiv.org/abs/1805.00270">Capturing and Interpreting Ambiguity in Crowdsourcing Frame Disambiguation</a></strong>. <a href="https://www.humancomputation.com/2018/">HCOMP 2018</a>.</li>
</ul>
<p>To replicate the data processing from the paper, use the Jupyter Notebook file <code>CrowdTruth metrics.ipynb</code>. It requires the installation of the <a href="https://github.com/CrowdTruth/CrowdTruth-core">CrowdTruth metrics</a> Python package (v >= 2.0).</p>
<p>The data aggregated with CrowdTruth metrics is available in folder <code>data/output/</code></p>
<p>The raw crowdsourcing data is available in folder <code>data/input/</code></p>
<p>If you find this data useful in your research, please consider citing:</p>
<pre><code>@inproceedings{dumitrache2018frames,
Author = {Anca Dumitrache and Lora Aroyo and Chris Welty},
Title = {Capturing Ambiguity in Crowdsourcing Frame Disambiguation},
Booktitle = {The sixth AAAI Conference on Human Computation and Crowdsourcing},
Year = {2018}
}
</code></pre>
CrowdTruth Corpus for Open Domain Relation Extraction from Sentences
No full text<p>This repository contains a ground truth corpus for open domain relation extraction from sentences, acquired with crowdsourcing and processed with <strong><a href="http://crowdtruth.org/">CrowdTruth</a></strong> metrics that capture ambiguity in annotations by measuring inter-annotator disagreement.</p>
<p>The dataset contains annotations for 4,100 sentences sampled from Angeli et al. (1) and Riedel et al. (2), over 16 relations, with each sentence annotated by 15 workers. The sentences have been pre-processed with Distant Supervision (3) using the Freebase knowledge base, in order to identify the term pairs in each sentence that are likely to express a relation. The crowdsourced data was collected from <a href="http://figure-eight.com/">Figure Eight</a> and <a href="https://www.mturk.com/">Amazon Mechanical Turk</a>.</p>
<p>This corpus has been discussed in the following papers:</p>
<ul>
<li>Anca Dumitrache, Lora Aroyo and Chris Welty: <strong><a href="https://arxiv.org/abs/1809.00537">Crowdsourcing Semantic Label Propagation in Relation Classification</a></strong>. <a href="http://fever.ai/">FEVER</a> Workshop at <a href="http://emnlp2018.org/">EMNLP 2018</a>.</li>
<li>Anca Dumitrache, Lora Aroyo and Chris Welty: <strong><a href="https://arxiv.org/abs/1711.05186">False Positive and Cross-relation Signals in Distant Supervision Data</a></strong>. <a href="http://www.akbc.ws/">AKBC</a> Workshop at <a href="http://nips.cc/">NIPS 2017</a>.</li>
<li>Anca Dumitrache, Lora Aroyo and Chris Welty: <strong><a href="http://crowdtruth.org/wp-content/uploads/2017/03/collint17-open-domain.pdf">Disagreement in Crowdsourcing and Active Learning for Better Distant Supervision Quality</a></strong>. <a href="http://collectiveintelligenceconference.org/">Collective Intelligence 2017</a>.</li>
</ul>
<p>Sentence-level data is available in file: <code>|--data/output/aggregated_sentences.csv</code></p>
<p>Worker-level data is available in file: <code>|--data/output/aggregated_workers.csv</code></p>
<p>Raw crowdsourcig data is available in folder: <code>|--data/input/</code></p>
<p>Results of the relation classification model are available in folder: <code>|--data/model_results/</code></p>
<p> </p>
<p>References</p>
<p>(1) Angeli, Gabor, et al. "Combining distant and partial supervision for relation extraction." Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014.</p>
<p>(2) Riedel, Sebastian, et al. "Relation extraction with matrix factorization and universal schemas." Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). 2013.</p>
<p>(3) Mintz, Mike, et al. "Distant supervision for relation extraction without labeled data." Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2. Association for Computational Linguistics, 2009.</p>
SASWeb 2012: Semantic and Adaptive Social Web
No full textSASWeb 2012: Semantic and Adaptive Social Web
organized by Lora Aroyo, Federica Cena, Antonina Dattolo, Pasquale Lops, Julita Vassileva
(1) Building multi-layer social knowledge maps with Google Maps API
MinEr Liang, Julio Guerra, Peter Brusilovsky
(2) Learning from a network of peers via peer-driven adjustment of a corpus
John Champaign, Robin Cohen
****Invited Talks
(4) Culture in User Modeling 3.0
Jacqueline Bourdeau
(5) Leveraging social and semantic components in adaptive environments
Cristina Gena
(6) Meaning is its use: towards the use of distributional semantics for content-based recommender systems
Cataldo Musto
(7) Exploring folksonomies for adaptive query expansion
Fabio Gasparett
Combining composition technologies and EUD to enhance visitors' experience at cultural heritage sites
No full textThis paper illustrates our approach to enhance the visit experience of archeological parks. It exploits composition technologies, End-User Development and participa-tory design approaches, in order to allow different stakeholders to create, use and share Personal Information Spaces. Heterogeneous content can be combined and manipulated to satisfy different information needs, thus enabling personalized vis-its to Cultural Heritage sites
OntoAIMS: Ontological Approach to Courseware Authoring
No full textIn this paper we discuss how ontology concepts can be beneficial for a flexible and semantic rich description of the authoring process and for the provision of authoring support of Intelligent Educational Systems (IES) with respect to three main authoring tasks: domain editing, course sequencing and resource management. We take a semantic perspective on the knowledge representation and explore the interoperability between the various ontological structures for domain, instructional and resource modeling and the modeling of the entire authoring process
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
