1,720,980 research outputs found
OpenSignal-WiFi dataset
No full textThis dataset is provided by OpenSignal for the research presented in the paper "Bayesian Modelling of Community-Based Multidimensional Trust in Participatory Sensing under Data Sparsity" by Venanzi, Matteo, Teacy, W.T.L., Rogers, Alex and Jennings, Nicholas R. (2015) In, International Joint Conference on Artificial Intelligence (IJCAI-15), Buenos Aires, AR, 25 - 31 Jul 2015. 8pp.</span
Trust, Kinship and Locality in the Iterated Prisoner's Dilemma
No full textThe Prisoner's Dilemma is maybe the best-known paradox in Game Theory. In this game, a player meets another player, and must choose to cooperate or defect: the best outcome for player i would be achieved if i defected while the opponent j cooperated, which would be the worst outcome for player j; both players would prefer mutual cooperation over mutual defection; and finally, the scenario is "symmetric". From the perspective of game theory, the rational choice in the prisoners dilemma is for both players to defect. It is called a "dilemma" because in fact both players would prefer mutual cooperation { but this outcome is impossible, because if one player cooperates, the other would rather defect. [Axelrod 1984] ran a tournament in which players played the prisoners dilemma against a number of opponents in a series of rounds (the "iterated prisoners dilemma"). He found that "cooperative" game playing strategies could flourish in this tournament if they were given the opportunity to encounter other cooperative strategies. This study is aimed at investigating the following issues around the iterated prisoners dilemma: (i) Trust: Suppose every agent is equipped with a value tw indicating how trustworthy it is; what happens if we take account of such a value in making decisions; how does this affect the dynamics of cooperation/defection? (ii) Kinship: Suppose we have a model of "family distance", so that agents are classified into families, being less likely to cooperate with those that are more distant in family terms. How do such concerns affect the dynamics of cooperation? (iii) Locality: Suppose agents are arranged on a graph, and retrieve trust information by querying and using trust of their neighbours. How does graph topology affect the dynamics of cooperation? E.g. is it the case that "gregarious" agents (with lots of neighbours) perform much better than "lonely" agents (with only 1 neighbour)? We will investigate through experiments the notions of Trust and Reliability applied to the Prisoner's Dilemma context, under multiple aspects. We will review works related to these topics in order to introduce some efficient solutions for dealing with the above issues
Trust-based algorithms for fusing crowdsourced estimates of continuous quantities
No full textCrowdsourcing has provided a viable way of gathering information at unprecedented volumes and speed by engaging individuals to perform simple micro–tasks. In particular, the crowdsourcing paradigm has been successfully applied to participatory sensing, in which the users perform sensing tasks and provide data using their mobile devices. In this way, people can help solve complex environmental sensing tasks, such as weather monitoring, nuclear radiation monitoring and cell tower mapping, in a highly decentralised and parallelised fashion. Traditionally, crowdsourcing technologies were primarily used for gathering data for classifications and image labelling tasks. In contrast, such crowd–based participatory sensing poses new challenges that relate to (i) dealing with human–reported sensor data that are available in the form of continuous estimates of an observed quantity such as a location, a temperature or a sound reading, (ii) dealing with possible spatial and temporal correlations within the data and (ii) issues of data trustworthiness due to the unknown capabilities and incentives of the participants and their devices. Solutions to these challenges need to be able to combine the data provided by multiple users to ensure the accuracy and the validity of the aggregated results. With this in mind, our goal is to provide methods to better aid the aggregation process of crowd–reported sensor estimates of continuous quantities when data are provided by individuals of varying trustworthiness. To achieve this, we develop a trust–based in- formation fusion framework that incorporates latent trustworthiness traits of the users within the data fusion process. Through this framework, we develop a set of four novel algorithms (MaxTrust, BACE, TrustGP and TrustLGCP) to compute reliable aggregations of the users’ reports in both the settings of observing a stationary quantity (Max- Trust and BACE) and a spatially distributed phenomenon (TrustGP and TrustLGCP). The key feature of all these algorithm is the ability of (i) learning the trustworthiness of each individual who provide the data and (ii) exploit this latent user’s trustworthiness information to compute a more accurate fused estimate. In particular, this is achieved by using a probabilistic framework that allows our methods to simultaneously learn the fused estimate and the users’ trustworthiness from the crowd reports. We validate our algorithms in four key application areas (cell tower mapping, WiFi networks mapping, nuclear radiation monitoring and disaster response) that demonstrate the practical impact of our framework to achieve substantially more accurate and informative predictions compared to the existing fusion methods. We expect that results of this thesis will allow to build more reliable data fusion algorithms for the broad class of human–centred information systems (e.g., recommendation systems, peer reviewing systems, student grading tools) that are based on making decisions upon subjective opinions provided by their users
Trust-Based Fusion of Untrustworthy Information in Crowdsourcing Applications
No full textIn this paper, we address the problem of fusing untrustworthy reports provided from a crowd of observers, while simultaneously learning the trustworthiness of individuals. To achieve this, we construct a likelihood model of the userss trustworthiness by scaling the uncertainty of its multiple estimates with trustworthiness parameters. We incorporate our trust model into a fusion method that merges estimates based on the trust parameters and we provide an inference algorithm that jointly computes the fused output and the individual trustworthiness of the users based on the maximum likelihood framework. We apply our algorithm to cell tower localisation using real-world data from the OpenSignal project and we show that it outperforms the state-of-the-art methods in both accuracy, by up to 21%, and consistency, by up to 50% of its predictions. Copyright © 2013, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved
Sentiment popularity - Amazon Mechanical Turk dataset
No full textDataset re-collected from an original dataset collected by Pang, B., and Lee, L. 2004. "A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts". In Proceedings of the 42nd annual meeting on Association for Computational Linguistics. The dataset presents a binary classification problem, with workers asked to select either positive (1) or negative (0) for a 500 sentences extracted from movie reviews, with gold labels assigned by the website. It contains 10,000 sentiment judgements collected from 143 using the Amazon Mechanical Turk platform. Each row is in the format WorkerID, TaskID, Worker label, Gold label, time spent on the judgement by the worker</span
Weather Sentiment - Amazon Mechanical Turk dataset
No full textDataset re-collected using Amazon Mechanical Turk from an original dataset provided by CrowdFlower as part of the 2013 Crowdsourcing at Scale shared task challenge. The dataset contains 6000 classifications of the sentiment of 300 tweets, with gold-standard sentiment labels, provided by 110 workers. The sentiment judgements are provided in the following categories: negative (0), neutral (1), positive (2), tweet not related to weather (3) and can't tell (4). Each row contains workerID, taskID, Worker label, gold label, time spent by the worker to produce the judgment</span
Facing Openness with Socio Cognitive Trust and Categories.
No full textTypical solutions for agents assessing trust relies on the circulation of information on the individual level, i.e. reputational images, subjective experiences, statistical analysis, etc. This work presents an alternative approach, inspired to the cognitive heuristics enabling humans to reason at a categorial level. The approach is envisaged as a crucial ability for agents in order to: (1) estimate trustworthiness of unknown trustees based on an ascribed membership to categories; (2) learn a series of emergent relations between trustees observable properties and their effective abilities to fulfill tasks in situated conditions. On such a basis, categorization is provided to recognize signs (Manifesta) through which hidden capabilities (Kripta) can be inferred. Learning is provided to refine reasoning attitudes needed to ascribe tasks to categories. A series of architectures combining categorization abilities, individual experiences and context awareness are evaluated and compared in simulated experiments
Community-Based Bayesian Aggregation Models for Crowdsourcing
No full textThis paper addresses the problem of extracting accurate labels from crowdsourced datasets, a key challenge in crowdsourcing. Prior work has focused on modeling the reliability of individual workers, for instance, by way of confusion matrices, and using these latent traits to estimate the true labels more accurately. However, this strategy becomes ineffective when there are too few labels per worker to reliably estimate their quality. To mitigate this issue, we propose a novel community-based Bayesian label aggregation model, CommunityBCC, which assumes that crowd workers conform to a few different types, where each type represents a group of workers with similar confusion matrices. We assume that each worker belongs to a certain community, where the worker’s confusion matrix is similar to (a perturbation of) the community’s confusion matrix. Our model can then learn a set of key latent features: (i) the confusion matrix of each community, (ii) the community membership of each user, and (iii) the aggregated label of each item. We compare the performance of our model against established aggregation methods on a number of large-scale, real-world crowdsourcing datasets. Our experimental results show that our CommunityBCC model consistently outperforms state-of-the-art label aggregation methods, gaining, on average, 8% more accuracy with the same amount of labels
From Manifesta to Krypta: The Relevance of Categories for Trusting Others
No full textIn this paper we consider the special abilities needed by agents for assessing trust based on inference and reasoning. We analyze the case in which it is possible to infer trust towards unknown counterparts by reasoning on abstract classes or categories of agents shaped in a concrete application domain. We present a scenario of interacting agents providing a computational model implementing different strategies to assess trust. Assuming a medical domain, categories, including both competencies and dispositions of possible trustees, are exploited to infer trust towards possibly unknown counterparts. The proposed approach for the cognitive assessment of trust relies on agents' abilities to analyze heterogeneous information sources along different dimensions. Trust is inferred based on specific observable properties (Manifesta), namely explicitly readable signals indicating internal features (Krypta) regulating agents' behavior and effectiveness on specific tasks. Simulative experiments evaluate the performance of trusting agents adopting different strategies to delegate tasks to possibly unknown trustees, while experimental results show the relevance of this kind of cognitive ability in the case of open Multi Agent Systems
- …
