100 research outputs found
Characterising and Mitigating Aggregation-Bias in Crowdsourced Toxicity Annotations
No full textTraining machine learning (ML) models for natural language processing usually requires large amount of data, often acquired through crowdsourcing. The way this data is collected and aggregated can have an effect on the outputs of the trained model such as ignoring the labels which differ from the majority. In this paper we investigate how label aggregation can bias the ML results towards certain data samples and propose a methodology to highlight and mitigate this bias. Although our work is applicable to any kind of label aggregation for data subject to multiple interpretations, we focus on the effects of the bias introduced by majority voting on toxicity prediction over sentences. Our preliminary results point out that we can mitigate the majority-bias and get increased prediction accuracy for the minority opinions if we take into account the different labels from annotators when training adapted models, rather than rely on the aggregated labels.Accepted Author ManuscriptWeb Information System
On developers’ practices for hazard diagnosis in machine learning systems
No full textMachine learning (ML) is an artificial intelligence technology that has a great potential for being adopted in various sectors of activities. Yet, it is now also increasingly recognized as a hazardous technology. Failures in the outputs of an ML system might cause physical or social harms. Besides, the development and deployment of an ML system itself are also argued to be harmful in certain contexts. Surprisingly, these hazards persist in applications where ML technology has been deployed, despite the increasing amount of research performed by the ML research community. In this thesis, we task ourselves with the challenges of understanding the reasons for the subsistence of hazardous system’s output failures and of hazardous development and deployment processes in practice, and of developing solutions to further diagnose these hazardous failures (especially in the system’s outputs). For that, we investigate further the nature of the potential gap between research and the practices of those developers who build and deploy the systems. To do so, we survey major related ML research directions, surface developers practices and challenges, and search for types of (mis)alignment between theory and practices. There, among others, we find a lack of technical support for ML developers to identify the potential failures of their systems. Hence, we then tackle the development and evaluation of a human-in-the-loop, explainability-based, failure diagnosis method and user-interface for computer vision systems...Web Information System
Noise and vibration on board large pleasure crafts: Literature Research
No full textSound and vibrations are important aspects when designing a ship. Most of all on board passenger ships and pleasure crafts a "silent" vessel is essential. To realise this, it has to be known where the waves, because that is what sound and vibration are about, come from, how strong they are and how propagation through the vessel takes place. With that purpose an extensive literature research was performed, using amongst others measurement data that were made available by Oceanco Shipyards. The research was performed with respect to the comfort of people on board. For humans the audible range is the frequency range between 20-20 000[Hzi. Within this range the frequencies between 1000-8 000[Hz] are most important, since they encompass speech. Frequencies below 80[Hz] are observed as vibration rather than sound. On board ships frequencies between, roughly estimated, 0-10 000[Hz] are present. Two kinds of noise are important: structure borne and airborne. Structure borne will have the most influence, because it can spread over a full ship's length, where airborne noise only has a local effect. As was expected in advance, the most dominating sources are found in the engine room and just outside: the propeller, main engines, gearboxes and auxiliary engines. To prevent or reduce the noise from propagating, the machinery is fitted with flexible mounts and heavy foundations and where possible acoustic enclosures. The accommodations, crew cabins, work spaces etcetera are fitted with appropriate wall and floor isolation: the deck just above the engine room is mostly treated with a special high density material for extra reduction. Double glazed windows keep out outside noise. Still, with all these measures taken, the measurement results sometimes are astonishingly different from the predictions that were based on the design data. An explanation is not always found. No definite conclusions can be drawn yet, but it seems likely that either the accuracy of the predictions is not good enough or that design and practice are too far apart from each other to reach the predicted results. Further investigation will thus be performed to find the cause of the differences and to think about how the situation could be improved.Mechanical, Maritime and Materials EngineeringMarine and Transport TechnologyShip Design, Production and OperationOvS 01/1
A Novel Scanning Land Mine Detector Based on the Technique of Neutron Back Scattering Imaging
No full textThe neutron back-scattering (NBS) technique is a well established method to find hydrogen in objects. It can be applied in land mine detection taking advantage of the fact that land mines are abundant in hydrogen. The NBS technique is suitable for land mine scanning e.g., seeking for land mines with a moving detector system, because of the high speed of operation. Scan speeds up to 800 mm/s are reported here depending on the intensity of the neutron source, the mine size and the depth at which the mine is buriedRadiation, Radionuclides and ReactorsApplied Science
Managing bias and unfairness in data for decision support: a survey of machine learning and data engineering approaches to identify and mitigate bias and unfairness within data management and analytics systems
No full textThe increasing use of data-driven decision support systems in industry and governments is accompanied by the discovery of a plethora of bias and unfairness issues in the outputs of these systems. Multiple computer science communities, and especially machine learning, have started to tackle this problem, often developing algorithmic solutions to mitigate biases to obtain fairer outputs. However, one of the core underlying causes for unfairness is bias in training data which is not fully covered by such approaches. Especially, bias in data is not yet a central topic in data engineering and management research. We survey research on bias and unfairness in several computer science domains, distinguishing between data management publications and other domains. This covers the creation of fairness metrics, fairness identification, and mitigation methods, software engineering approaches and biases in crowdsourcing activities. We identify relevant research gaps and show which data management activities could be repurposed to handle biases and which ones might reinforce such biases. In the second part, we argue for a novel data-centered approach overcoming the limitations of current algorithmic-centered methods. This approach focuses on eliciting and enforcing fairness requirements and constraints on data that systems are trained, validated, and used on. We argue for the need to extend database management systems to handle such constraints and mitigation methods. We discuss the associated future research directions regarding algorithms, formalization, modelling, users, and systems.Web Information System
On modified boundary conditions for the free edge of a shell
No full textMechanical, Maritime and Materials Engineerin
Explainability in AI Policies: A Critical Review of Communications, Reports, Regulations, and Standards in the EU, US, and UK
No full textPublic attention towards explainability of artificial intelligence (AI) systems has been rising in recent years to offer methodologies for human oversight. This has translated into the proliferation of research outputs, such as from Explainable AI, to enhance transparency and control for system debugging and monitoring, and intelligibility of system process and output for user services. Yet, such outputs are difficult to adopt on a practical level due to a lack of a common regulatory baseline, and the contextual nature of explanations. Governmental policies are now attempting to tackle such exigence, however it remains unclear to what extent published communications, regulations, and standards adopt an informed perspective to support research, industry, and civil interests. In this study, we perform the first thematic and gap analysis of this plethora of policies and standards on explainability in the EU, US, and UK. Through a rigorous survey of policy documents, we first contribute an overview of governmental regulatory trajectories within AI explainability and its sociotechnical impacts. We find that policies are often informed by coarse notions and requirements for explanations. This might be due to the willingness to conciliate explanations foremost as a risk management tool for AI oversight, but also due to the lack of a consensus on what constitutes a valid algorithmic explanation, and how feasible the implementation and deployment of such explanations are across stakeholders of an organization. Informed by AI explainability research, we then conduct a gap analysis of existing policies, which leads us to formulate a set of recommendations on how to address explainability in regulations for AI systems, especially discussing the definition, feasibility, and usability of explanations, as well as allocating accountability to explanation providers.Organisation & GovernanceWeb Information System
Designing a single player textual GWAP for validating tacit knowledge elicitation from crowds
No full textMachine learning can still make harmful mistakes. A solution would be tacit knowledge. Machine learning needs this type of knowledge to improve. An example of such knowledge that can help make the system draw better logical conclusions would be: if presented with an open fridge, then it could deduct that the food will go bad. Tacit knowledge or common-sense knowledge refers to the type of knowledge which is acquired through experience, the kind only humans can create. GWAPs (game with a purpose) have shown quite promising results for acquiring such knowledge. Unfortunately, it could still contain errors due to users who only want to harm the game data, etc. and there is no method for validating such knowledge without involving humans somehow. Therefore, from the previously stated problem, our goal has emerged - develop a method for validating an existing data set and for later training machine learning models using a GWAP. There has been work done before using GWAPs to elicit such information, yet they are limited in the sense that their main focus is set on data collection, not validation. Since very few projects looked into it, we decided to investigate a new GWAP which has as main purpose tacit knowledge validation. The main question which we aim to answer is "How can we elicit and validate tacit knowledge using a game with the following settings: single-player, textual concepts, goal: associate words with their concepts." The game presents hints to the users and they have to guess, as fast as possible and with the least amount of tries as possible, which answer is correct from the 6 options that are provided. The evaluation of the game will be made using standard metrics such as games played, time spent playing, number of users, etc. The conclusion is that the GWAP, even with the lack of data, was quite capable of analyzing the quality of the data set and reached a conclusion that is easily confirmed by a mere look over the initial data set.CSE3000 Research ProjectComputer Science and Engineerin
- …
