1,720,981 research outputs found
Data Lifecycle Challenges in Production Machine Learning: A Survey
No full textMachine learning has become an essential tool for gleaning knowledge from data and tackling a diverse set of computationally hard tasks. However, the accuracy of a machine learned model is deeply tied to the data that it is trained on. Designing and building robust processes and tools that make it easier to analyze, validate, and transform data that is fed into large-scale machine learning systems poses data management challenges. Drawn from our experience in developing data-centric infrastructure for a production machine learning platform at Google, we summarize some of the interesting research challenges that we encountered, and survey some of the relevant literature from the data management and machine learning communities. Specifically, we explore challenges in three main areas of focus data understanding, data validation and cleaning, and data preparation. In each of these areas, we try to explore how different constraints are imposed on the solutions depending on where in the lifecycle of a model the problems are encountered and who encounters them.
Automated Data Slicing for Model Validation: A Big data - AI Integration Approach
No full text© 1989-2012 IEEE. As machine learning systems become democratized, it becomes increasingly important to help users easily debug their models. However, current data tools are still primitive when it comes to helping users trace model performance problems all the way to the data. We focus on the particular problem of slicing data to identify subsets of the validation data where the model performs poorly. This is an important problem in model validation because the overall model performance can fail to reflect that of the smaller subsets, and slicing allows users to analyze the model performance on a more granular-level. Unlike general techniques (e.g., clustering) that can find arbitrary slices, our goal is to find interpretable slices (which are easier to take action compared to arbitrary subsets) that are problematic and large. We propose mathsf{Slice Finder}SliceFinder, which is an interactive framework for identifying such slices using statistical techniques. Applications include diagnosing model fairness and fraud detection, where identifying slices that are interpretable to humans is crucial. This research is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research
Recommended from our members
Physical Design Tuning Methods for Emerging System Architectures
Get PDFPhysical design tuning (i.e., configuring the physical data model and secondary data structures) is key to obtaining good performance for a database management system. Recently, there have been disruptions to the types of data, queries, and systems used for analytical data management and processing. The advent of the cloud and database-as-a-service along with these recent disruptions have changed the analytical landscape significantly. Physical design tuning methods that were developed to address traditional RDBMS architectures are inadequate for the emerging system architectures and analytics. While tuning the physical design remains crucial for good performance, the problem acquires interesting dimensions in these new contexts.This dissertation introduces new physical design tuning methods for emerging system architectures. Our methods exploit the unique characteristics of several different architectures, leveraging them toward good physical design. For replicated database architectures, we introduce the concept of divergent design that exploits database replication. Our method specializes the design of replicas to efficiently process subsets of the workload while still affording the opportunity to load balance the workload across replicas. For MapReduce architectures, we introduce the concept of opportunistic design that exploits the by-products of query processing in MapReduce. Our method includes a semantic model for user-defined functions (UDFs) and a novel query rewriting algorithm that together enable the effective reuse of previous results. For hybrid MapReduce--RDBMS architectures, we introduce the concept of multistore design that exploits the unique strengths of both stores. Our method periodically reorganizes the data in each store by transferring data between them, adapting their physical designs as the workload changes dynamically. Lastly, based on our experiences with the changing analytical landscape, we examine big data exploratory queries in current real-world scenarios and present a workload modeled upon our findings. We then present a benchmark along with a set of system performance metrics that highlight the various design tradeoffs made by different architectures. By better understanding these tradeoffs, the benchmark can be used to help guide the design of future hybrid architectures
Searching Shared Content in Communities with the Data Ring
Get PDFInternational audienceInformation ubiquity has created a large crowd of users (most notably scientists), who could employ DBMS technology to process and share their data more effectively. Still, this user base prefers to keep its data in files that can be easily managed by applications such as spreadsheets, rather than deal with the complexity and rigidity of modern database systems. In this paper, we propose a vision for enabling non-experts, such as scientists, to build content sharing communities in a true database fashion: declaratively. The proposed infrastructure, called the data ring, enables users to manage and share their data with minimal effort; the user points to the data that should be shared, and the data ring becomes responsible for automatically indexing the data (to make it accessible), replicating it (for availability), and reorganizing its physical storage (for better query performance). We outline the salient features of our proposal, and outline research challenges that must be addressed in order to realize this vision
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Get PDFWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
A Scalable, Predictable Join Operator for Highly Concurrent Data Warehouses
Get PDFConventional data warehouses employ the query-at-a-time model, which maps each query to a distinct physical plan. When several queries execute concurrently, this model introduces contention, because the physical plans—unaware of each other—compete for access to the underlying I/O and computation resources. As a result, while modern systems can efficiently optimize and evaluate a single complex data analysis query, their performance suffers significantly when multiple complex queries run at the same time. We describe an augmentation of traditional query engines that improves join throughput in large-scale concurrent data warehouses. In contrast to the conventional query-at-a-time model, our approach employs a single physical plan that can share I/O, computation, and tuple storage across all in-flight join queries. We use an "always-on" pipeline of non-blocking operators, coupled with a controller that continuously examines the current query mix and performs run-time optimizations. Our design allows the query engine to scale gracefully to large data sets, provide predictable execution times, and reduce contention. In our empirical evaluation, we found that our prototype outperforms conventional commercial systems by an order of magnitude for tens to hundreds of concurrent queries.DSLA
- …
