10,421 research outputs found
Metadata Representations for Queryable ML Model Zoos
No full textMachine learning (ML) practitioners and organizations are building model zoos of pre-trained models, containing metadata describing properties of the ML models and datasets that are useful for reporting, auditing, reproducibility, and interpretability purposes. The metatada is currently not standardised; its expressivity is limited; and there is no interoperable way to store and query it. Consequently, model search, reuse, comparison, and composition are hindered. In this paper, we advocate for standardized ML model metadata representation and management, proposing a toolkit supported to help practitioners manage and query that metadata.Web Information SystemsHuman-Centred Artificial Intelligenc
A Manifesto of Nodalism
No full textThis paper proposes the notion of Nodalism as a means describing contemporary culture and of understanding my own creative practice in electronic music composition. It draws on theories and ideas from Kirby, Bauman, Bourriaud, Deleuze, Guatarri, and Gochenour, to demonstrate how networks of ideas or connectionist neural models of cognitive behaviour can be used to contextualize, understand and become a creative tool for the creation of contemporary electronic music
Optimizing ML Inference Queries Under Constraints
No full textThe proliferation of pre-trained ML models in public Web-based model zoos facilitates the engineering of ML pipelines to address complex inference queries over datasets and streams of unstructured content. Constructing optimal plan for a query is hard, especially when constraints (e.g. accuracy or execution time) must be taken into consideration, and the complexity of the inference query increases. To address this issue, we propose a method for optimizing ML inference queries that selects the most suitable ML models to use, as well as the order in which those models are executed. We formally define the constraint-based ML inference query optimization problem, formulate it as a Mixed Integer Programming (MIP) problem, and develop an optimizer that maximizes accuracy given constraints. This optimizer is capable of navigating a large search space to identify optimal query plans on various model zoos.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Web Information SystemsHuman-Centred Artificial Intelligenc
Building a generalisable ML pipeline at ING
No full textAdvances in data science have caused an increase in the use of Artificial Intelligence (AI), specifically Machine Learning (ML), throughout various fields. Not only in research but in the industry as well, has ML been receiving increasing amounts of interest. Many companies rely on ML models to increase the efficiency of existing processes or offer new services and products. The industry, however, is facing several additional challenges compared to the academic context. One of those challenges is applying the Development Operations (DevOps) model to an ML application, also referred to as MLOps. This thesis sets out to find the specific challenges that practitioners encounter while operationalising ML models. To do so, we perform a single-case case study on an ML pipeline built by the Trade & Communication Surveillance team at the ING bank. This case study consists of conducting a set of interviews and performing a manual code inspection of the pipeline. The team faces challenges ranging from having insufficient time for operationalising each ML project individually to operating in the highlyregulated fintech context. Their pipeline is able to deploy a single ML model but it does not generalise well to other projects. We present the first version of an application that mitigates these challenges. The application is able to deploy ML models to the development environment at ING and can be operated by data scientists to reduce the effort of operationalising an ML model. Computer Science | Software Technolog
'Project smells' - Experiences in Analysing the Software Quality of ML Projects with mllint
No full textMachine Learning (ML) projects incur novel challenges in their development and productionisation over traditional software applications, though established principles and best practices in ensuring the project's software quality still apply. While using static analysis to catch code smells has been shown to improve software quality attributes, it is only a small piece of the software quality puzzle, especially in the case of ML projects given their additional challenges and lower degree of Software Engineering (SE) experience in the data scientists that develop them. We introduce the novel concept of project smells which consider deficits in project management as a more holistic perspective on software quality in ML projects. An open-source static analysis tool mllint was also implemented to help detect and mitigate these. Our research evaluates this novel concept of project smells in the industrial context of ING, a global bank and large software- and data-intensive organisation. We also investigate the perceived importance of these project smells for proof-of-concept versus production-ready ML projects, as well as the perceived obstructions and benefits to using static analysis tools such as mllint. Our findings indicate a need for context-aware static analysis tools, that fit the needs of the project at its current stage of development, while requiring minimal configuration effort from the user. Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Software EngineeringSoftware Technolog
Audiomobiles, Sculptures and Conundrums
No full textRoberto Gerhard was a pioneer of electronic music in England creating a number of substantial concert, theatre and radio works from as early as 1954. Gerhard’s electronic music is one of the richest repositories for understanding the development of the composer’s late compositional technique. Apart from the Symphony no.3, ‘Collages’, none of Gerhard’s electronic music is published. This paper will discuss aspects of Gerhard’s electronic music, focusing on Audiomobiles (1958-59) and Sculptures (1963)
Music for classical guitar by South African composers : a historical survey, notes on selected works and a general catalogue
No full textIncludes abstract.Includes bibliographical references (leaves 296-309).This is the first comprehensive investigation of music for, or including, the classical guitar by South African composers. The focus of this research has been, firstly, to uncover as much of the repertoire as possible, and, secondly, to collate, study, catalogue and report on the information. A brief historical survey of the guitar in South Africa provides the context within which this study was conducted. The primary sources of quantitative data collection were through the archival catalogues of the South African Music Rights Organisation and through personal contact with guitarists, composers and guitar teachers. Other sources consulted were publishers, broadcasting corporations, recording companies, libraries and the internet. The body of the dissertation comprises biographical sketches, background notes, analyses and technical notes on 17 selected solo and chamber works dating from 1947 to 2007 by some of South Africa's most prominent composers and guitaristcomposers. The repertoire ranges in style from the traditional and ethnically inspired to the experimental and abstract. As this is an empirical survey, each selected entry includes details on instrumentation, duration, level of difficulty, number of pages, scordatura, commissions or requests, sources or publishers, premières and recordings. A biography of each composer is provided as well as background notes which offer an overview of the selected work. The notes discuss historical, cultural, musical and extra-musical influences, and frequently include references to interview material. The commentaries on the selected works, with musical examples, include an analytical component describing structure, form, stylistic and compositional elements, while the technical observations include performance suggestions and a grading for each work
Assessing Robustness of ML-Based Program Analysis Tools using Metamorphic Program Transformations
No full textMetamorphic testing is a well-established testing technique that has been successfully applied in various domains, including testing deep learning models to assess their robustness against data noise or malicious input. Currently, metamorphic testing approaches for machine learning (ML) models focused on image processing and object recognition tasks. Hence, these approaches cannot be ap- plied to ML targeting program analysis tasks. In this paper, we extend metamorphic testing approaches for ML models targeting software programs. We present Lampion, a novel testing frame- work that applies (semantics preserving) metamorphic transforma- tions on the test datasets. Lampion produces new code snippets equivalent to the original test set but different in their identifiers or syntactic structure. We evaluate Lampion against CodeBERT, a state-of-the-art ML model for Code-To-Text tasks that creates Javadoc summaries for given Java methods. Our results show that simple transformations significantly impact the target model be- havior, providing additional information on the models reasoning apart from the classic performance metric.Software EngineeringSoftware Technolog
Improving the ML Tradeoff Curve: Efficiencies in Computation, Memory, and Tuning
No full text473 pagesA fundamental idea in machine learning (ML) has been the scaling of resources to increase model capability. More training iterations, hyperparameter tuning, model parameters, or data are generally beneficial. For a computational or memory resource, and a capability metric, there is a a Pareto frontier. Navigating this tradeoff curve, i.e. scaling resources to scale model capabilities, has powered much of the advancements in the modern ML age of the past 15 years. However, there is another way: the tradeoff curve can be improved. Scaling resources is not always viable, be it on edge devices or in data centers. My PhD has focused on this approach. Part I presents my work in post training compression. Given an already trained model, reduce the memory and/or compute footprint with minimal degradation. I propose methods in pruning and quantization. This problem formulation has become especially compelling with the prohibitive cost of creating LLMs. Part II presents my work in reducing hyperparameter tuning load. More tuning generally improves a model, but can be non-trivial. I use meta-learning to estimate optimal hyperparameters, and develop a more hyperparameter efficient statistical inference method. Part III goes beyond this tradeoff framework, which assumes a fixed objective. Improvements in recommender safety require reformulating the problem statement
- …
