1,720,952 research outputs found
CrowdCO-OP : sharing risks and rewards in crowdsourcing
Full text linkPaid micro-task crowdsourcing has gained in popularity partly due to the increasing need for large-scale manually labelled datasets which are often used to train and evaluate Artificial Intelligence systems. Modern paid crowdsourcing platforms use a piecework approach to rewards, meaning that workers are paid for each task they complete, given that their work quality is considered sufficient by the requester or the platform. Such an approach creates risks for workers; their work may be rejected without being rewarded, and they may be working on poorly rewarded tasks, in light of the disproportionate time required to complete them. As a result, recent research has shown that crowd workers may tend to choose specific, simple, and familiar tasks and avoid new requesters to manage these risks. In this paper, we propose a novel crowdsourcing reward mechanism that allows workers to share these risks and achieve a standardized hourly wage equal for all participating workers. Reward-focused workers can thereby take up challenging and complex HITs without bearing the financial risk of not being rewarded for completed work. We experimentally compare different crowd reward schemes and observe their impact on worker performance and satisfaction. Our results show that 1) workers clearly perceive the benefits of the proposed reward scheme, 2) work effectiveness and efficiency are not impacted as compared to those of the piecework scheme, and 3) the presence of slow workers is limited and does not disrupt the proposed cooperation-based approaches
CaptureBias: Supporting Media Scholars with Ambiguity-Aware Bias Representation for News Videos
No full textIn this project we explore the presence of ambiguity in textual and visual media and its influence on accurately understanding andcapturing bias in news. We study this topic in the context of supportingmedia scholars and social scientists in their media analysis. Our focuslies on racial and gender bias as well as framing and the comparisonof their manifestation across modalities, cultures and languages. In thispaper we lay out a human in the loop approach to investigate the role ofambiguity in detection and interpretation of bias.Accepted Author ManuscriptWeb Information System
The Effects of Crowd Worker Biases in Fact-Checking Tasks
Full text linkDue to the increasing amount of information shared online every day, the need for sound and reliable ways of distinguishing between trustworthy and non-trustworthy information is as present as ever. One technique for performing fact-checking at scale is to employ human intelligence in the form of crowd workers. Although earlier work has suggested that crowd workers can reliably identify misinformation, cognitive biases of crowd workers may reduce the quality of truthfulness judgments in this context. We performed a systematic exploratory analysis of publicly available crowdsourced data to identify a set of potential systematic biases that may occur when crowd workers perform fact-checking tasks. Following this exploratory study, we collected a novel data set of crowdsourced truthfulness judgments to validate our hypotheses. Our findings suggest that workers generally overestimate the truthfulness of statements and that different individual characteristics (i.e., their belief in science) and cognitive biases (i.e., the affect heuristic and overconfidence) can affect their annotations. Interestingly, we find that, depending on the general judgment tendencies of workers, their biases may sometimes lead to more accurate judgments
Characterising and Mitigating Aggregation-Bias in Crowdsourced Toxicity Annotations
No full textTraining machine learning (ML) models for natural language processing usually requires large amount of data, often acquired through crowdsourcing. The way this data is collected and aggregated can have an effect on the outputs of the trained model such as ignoring the labels which differ from the majority. In this paper we investigate how label aggregation can bias the ML results towards certain data samples and propose a methodology to highlight and mitigate this bias. Although our work is applicable to any kind of label aggregation for data subject to multiple interpretations, we focus on the effects of the bias introduced by majority voting on toxicity prediction over sentences. Our preliminary results point out that we can mitigate the majority-bias and get increased prediction accuracy for the minority opinions if we take into account the different labels from annotators when training adapted models, rather than rely on the aggregated labels.Accepted Author ManuscriptWeb Information System
Investigating User Perception of Gender Bias in Image Search
Full text linkThere is growing evidence that search engines produce results that are socially biased, reinforcing a view of the world that aligns with prevalent social stereotypes. One means to promote greater transparency of search algorithms - which are typically complex and proprietary - is to raise user awareness of biased result sets. However, to date, little is known concerning how users perceive bias in search results, and the degree to which their perceptions differ and/or might be predicted based on user attributes. One particular area of search that has recently gained attention, and forms the focus of this study, is image retrieval and gender bias. We conduct a controlled experiment via crowdsourcing using participants recruited from three countries to measure the extent to which workers perceive a given image results set to be subjective or objective. Demographic information about the workers, along with measures of sexism, are gathered and analysed to investigate whether (gender) biases in the image search results can be detected. Amongst other findings, the results confirm that sexist people are less likely to detect and report gender biases in image search results.This work has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and under Grant Agreement No 732328 and the Government of the Republic of Cyprus through the Directorate General for European Programmes, Coordination and Development.
© Author. © ACM 2018. This is the accepted version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of SIGIR 2018, DOI: https://doi.org/10.1145/3209978.3210094, Jahna Otterbacher, Alessandro Checco, Gianluca Demartini, and Paul Clough, "Investigating user perception of gender bias in image search: the role of sexism".In SIGIR '18- The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. https://www.acm.org/publications/policies/copyright-polic
On the Interactions Between Multiple Overlapping WLANs Using Channel Bonding
No full textNext-generation wireless local area networks (WLANs) will support the use of wider channels, which is known as channel bonding, to achieve higher throughput. However, because both the channel center frequency and the channel width are autonomously selected by each WLAN, the use of wider channels may also increase the competition with other WLANs operating in the same area for the available channel resources. In this paper, we analyze the interactions between a group of neighboring WLANs that use channel bonding and evaluate the impact of those interactions on the achievable throughput. A continuous-time Markov network model that is able to capture the coupled dynamics of a group of overlapping WLANs is introduced and validated. The results show that the use of channel bonding can provide significant performance gains, even in scenarios with a high density of WLANs, although it may also cause unfair situations in which some WLANs receive most of the transmission opportunities while others starve.The research of the 1st and 4th authors was partially supported by the Spanish government (project TEC2012-32354), and by the Catalan Government (SGR2009#00617). The research of the 2nd author was financially supported by the Science Foundation Ireland (grant 11/PI/11771). The research of the 3rd author was financially supported by The Netherlands Organization for Scientific Research (NWO) through the TOP-GO grant 613.001.012
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
- …
