26838 research outputs found
Sort by
The dutch draw: Constructing a universal baseline for binary classification problems
Full text linkNovel prediction methods should always be compared to a baseline to determine their performance. Without this frame of reference, the performance score of a model is basically meaningless. What does it mean when a model achieves an F1 of 0.8 on a test set? A proper baseline is, therefore, required to evaluate the 'goodness' of a performance score. Comparing results with the latest state-of-the-art model is usually insightful. However, being state-of-the-art is dynamic, as newer models are continuously developed. Contrary to an advanced model, it is also possible to use a simple dummy classifier. However, the latter model could be beaten too easily, making the comparison less valuable. Furthermore, most existing baselines are stochastic and need to be computed repeatedly to get a reliable expected performance, which could be computationally expensive. We present a universal baseline method for all binary classification models, named the Dutch Draw (DD). This approach weighs simple classifiers and determines the best classifier to use as a baseline. Theoretically, we derive the DD baseline for many commonly used evaluation measures and show that in most situations it reduces to (almost) always predicting either zero or one. Summarizing, the DD baseline is general, as it is applicable to any binary classification problem; simple, as it can be quickly determined without training or parameter tuning; and informative, as insightful conclusions can be drawn from the results. The DD baseline serves two purposes. First, it is a robust and universal baseline that enables comparisons across research papers. Second, it provides a sanity check during the prediction model's development process. When a model does not outperform the DD baseline, it is a major warning sign
Adaptive quantum computers : decoding and state preparation
Full text linkSince the first concepts of computers emerged in the late 19th century, significant advances have been made. Lately, interest in so-called quantum computers is rising, due to their potential in solving specific problems significantly faster than current methods can. Future quantum computers will have to interact with a standard computer to operate effectively. Even though current quantum computers are still under development and have limited capabilities, the interaction with a standard computer can already enhance their functionality, particularly by offloading certain computations to the standard computer. Quantum computers that interact with standard computers to perform computations are called adaptive quantum computers.
This work shows that adaptive quantum computers are more powerful than standard computers by showing that the former is better at retrieving information from corrupted digital data provided only a fixed number of computation steps is allowed. The proof uses a structure-versus-randomness approach from additive combinatorics that splits the problem in a structured and a random-like component and gives an explicit adaptive quantum circuit that retrieves the information. Additionally, this work shows that adaptive quantum computers are more efficient than non-adaptive quantum computers with respect to preparing specific quantum states. Specifically, this work gives explicit adaptive constructions for the uniform superposition state, the GHZ state, the W-state and the Dicke state. These states are often used in other quantum algorithms, so having efficient routines for preparing them also enhance the efficiency of other algorithms. This work concludes by comparing these adaptive quantum computations with non-adaptive ones, analyzing their performance both theoretically and through quantum hardware implementations
The Dutch scaler performance indicator: How much did my model actually learn?
Full text linkEvaluation metrics provide a means for quantifying and comparing performances of supervised learning models, but drawing meaningful conclusions from acquired scores requires a contextual framework. Our paper addresses this by introducing the Dutch scaler (DS), a novel performance indicator for binary classification models. It quantifies a model’s learning by contextualizing empirical metric scores with a baseline (Dutch draw) and a new instrument (Dutch oracle) representing the prediction quality of an “optimal” classifier. The DS performance indicator expresses the relative contribution of these components to obtain a model’s score, specifying the actual learning quality. We derived closed-form expressions to map metric scores to DS scores for common evaluation metrics and categorized them by their functional form and second derivative. The DS enhances the assessment of classifiers and facilitates a framework to compare prediction quality differences between models with varying metric scores
Positive streamer discharge simulations in humid air: uncertainty in input data and sensitivity analysis
Full text linkWe study how the choice of input data affects simulations of positive streamers in humid air, focusing on H2O cross sections, photoionization models, and chemistry sets. Simulations are performed in air with a mole fraction of 0%, 3% or 10% H2O using an axisymmetric fluid model. Five H2O cross section sets are considered, which lead to significant differences in the resulting electron attachment coefficient. As a result, the streamer velocity can vary by up to about 50% with 10% H2O. We compare results with three photoionization models: the Naidis model for humid air, the Aints model for humid air, and the standard Zheleznyak model for dry air. With the Naidis and in particular the Aints model, there is a significant reduction in photoionization with higher humidities. This results in higher streamer velocities and maximal electric fields, and it can also cause streamer branching in our axisymmetric simulations. Three humid air chemistry sets are considered. Differences between these sets, particularly in the formation of water clusters around positive ions, cause the streamer velocity to vary by up to about 50% with 10% H2O. A sensitivity analysis is performed to identify the most important chemical reactions in these chemistries
Benchmarking learned algorithms for computed tomography image reconstruction tasks
No full textComputed tomography (CT) is a widely used non-invasive diagnostic method in various fields, and recent advances in deep learning have led to significant progress in CT image reconstruction. However, the lack of large-scale, open-access datasets has hindered the comparison of different types of learned methods. To address this gap, we use the 2DeteCT dataset, a real-world experimental computed tomography dataset, for benchmarking machine learning based CT image reconstruction algorithms. We categorize these methods into post-processing methods, learned/unrolled iterative methods, learned regularizer methods, and plug-and-play methods, and provide a pipeline for easy implementation and evaluation. Using key performance metrics, including SSIM and PSNR, our benchmarking results showcase the effectiveness of various algorithms on tasks such as full data reconstruction, limited-angle reconstruction, sparse-angle reconstruction, low-dose reconstruction, and beam-hardening corrected reconstruction. With this benchmarking study, we provide an evaluation of a range of algorithms representative for different categories of learned reconstruction methods on a recently published dataset of real-world experimental CT measurements. The reproducible setup of methods and CT image reconstruction tasks in an open-source toolbox enables straightforward addition and comparison of new methods later on. The toolbox also provides the option to load the 2DeteCT dataset differently for extensions to other problems and different CT reconstruction tasks
Gaussian mixture identifiability from degree-6 moments
No full textWe resolve most cases of identifiability from sixth-order moments for Gaussian mixtures on spaces of large dimensions. Our results imply that for a mixture of Gaussians on an -dimensional space, the means and covariances can be uniquely recovered from the mixture moments of degree 6, as long as is bounded by some function in . The constant hidden in the -notation is optimal and equals the one in the upper bound from counting parameters. We give an argument that degree- moments never suffice in any nontrivial case, and we conduct some numerical experiments indicating that degree is minimal for identifiability
Three ways of proving termination of loops
Full text linkWe investigate three proof rules for proving termination of while programs and show their proof-theoretic equivalence. This involves a proof-theoretic analysis of various auxiliary proof rules in Hoare’s logic. By discussing representations of proofs in the form of proof outlines, we reveal differences between these equivalent proof rules when used in practice. We also address applications in the context of the paradigm of design by contract
Maintenance in procedural level design: Lessons from ludoscope
Full text linkProcedural level generation empowers level designers with tools
for generating many levels from a single specification, while engi-
neers maintain the level generator. Despite advances in procedural
techniques, little is known about their impact on long-term system
maintenance. We explore how Domain-Specific Languages (DSLs)
can help improve procedural level design processes, and support
maintenance by integrating level design sketches into generator-
agnostic tools. This short paper examines the evolution of Ludo-
scope, a state-of-the-art level generator used in the games Unex-
plored 1 and 2. In over a decade, it has grown in complexity, with
Unexplored 2’s generator now containing over 20K rewrite rules.
We investigate how Ludomotion addressed maintenance chal-
lenges, and how this impacts procedural level design. Our approach
combines: 1) a bottom-up analysis of Ludoscope; and 2) a top-down
exploration of a generic DSL for “level blueprints”. This paper con-
tributes the first step and discusses ongoing work on a reusable
framework for procedural level design. Our work takes a promising
first step towards industrial-strength maintenance solutions
Elastic-degenerate string comparison
Full text linkAn elastic-degenerate (ED) string T is a sequence of n sets T[1],…,T[n] containing m strings in total whose cumulative length is N. We call n, m, and N the length, the cardinality and the size of T, respectively. The language of T is defined as L(T)={S1⋯Sn:Si∈T[i] for all i∈[1,n]}. Given two ED strings, how fast can we check whether the two languages they represent have a nonempty intersection? We call this problem the ED STRING INTERSECTION (EDSI) problem. For two ED strings T1 and T2 of lengths n1 and n2, cardinalities m1 and m2, and sizes N1 and N2, respectively, we show the following: • There is no O((N1N2)1−ϵ)-time algorithm, for any ϵ>0, for EDSI even if T1 and T2 are over a binary alphabet, unless the Strong Exponential-Time Hypothesis is false. • There is no combinatorial O((N1+N2)1.2−ϵf(n1,n2))-time algorithm, for any ϵ>0 and any function f, for EDSI even if T1 and T2 are over a binary alphabet, unless the Boolean Matrix Multiplication conjecture is false. • An O(N1logN1logn1+N2logN2logn2)-time algorithm for outputting a compact representation of the intersection language of two unary ED strings. When T1 and T2 are given in a compact representation, we show that the problem is NP-complete. • An O(N1m2+N2m1)-time algorithm for EDSI. • An O˜(N1ω−1n2+N2ω−1n1)-time algorithm for EDSI, where ω is the matrix multiplication exponent; the O˜ notation suppresses factors that are polylogarithmic in the input size
The neurobench framework for benchmarking neuromorphic computing algorithms and systems
Full text linkNeuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. This article presents NeuroBench, a benchmark framework for neuromorphic algorithms and systems, which is collaboratively designed from an open community of researchers across industry and academia. NeuroBench introduces a common set of tools and systematic methodology for inclusive benchmark measurement, delivering an objective reference framework for quantifying neuromorphic approaches in both hardware-independent and hardware-dependent settings. For latest project updates, visit the project website ( neurobench.ai )