150813 research outputs found
Sort by
The Robust Malware Detection Challenge and Greedy Random Accelerated Multi-Bit Search
No full textAISec’20, November 13, 2020, Virtual Event, USATraining classifiers that are robust against adversarially modified examples is becoming increasingly important in practice. In the field of malware detection, adversaries modify malicious binary files to seem benign while preserving their malicious behavior. We report on the results of a recently held robust malware detection challenge. There were two tracks in which teams could participate: the attack track asked for adversarially modified malware samples and the defend track asked for trained neural network classifiers that are robust to such modifications. The teams were unaware of the attacks/defenses they had to detect/evade. Although only 9 teams participated, this unique setting allowed us to make several interesting observations.
We also present the challenge winner: GRAMS, a family of novel techniques to train adversarially robust networks that preserve the intended (malicious) functionality and yield high-quality adversarial samples. These samples are used to iteratively train a robust classifier. We show that our techniques, based on discrete optimization techniques, beat purely gradient-based methods. GRAMS obtained first place in both the attack and defend tracks of the competition
Identifying Novel Emotions and Wellbeing of Horses from Videos Through Unsupervised Learning
No full textfirst_pageDownload PDFsettingsOrder Article Reprints
Open AccessArticle
Identifying Novel Emotions and Wellbeing of Horses from Videos Through Unsupervised Learning
by Aarya Bhave 1ORCID,Emily Kieson 2ORCID,Alina Hafner 3 andPeter A. Gloor 1,*ORCID
1
Massachusetts Institute of Technology, System Design & Management, Cambridge, MA 02142, USA
2
Equine International, Cambridge CB22 5LD, UK
3
TUM School of Computation, Information and Technology, Technical University of Munich, Arcisstraße 21, 80333 Munich, Germany
*
Author to whom correspondence should be addressed.
Sensors 2025, 25(3), 859; https://doi.org/10.3390/s25030859
Submission received: 5 January 2025 / Revised: 22 January 2025 / Accepted: 30 January 2025 / Published: 31 January 2025
(This article belongs to the Special Issue Emotion Recognition and Cognitive Behavior Analysis Based on Sensors)
Downloadkeyboard_arrow_down Browse Figures Review Reports Versions Notes
Abstract
This research applies unsupervised learning on a large original dataset of horses in the wild to identify previously unidentified horse emotions. We construct a novel, high-quality, diverse dataset of 3929 images consisting of five wild horse breeds worldwide at different geographical locations. We base our analysis on the seven Panksepp emotions of mammals “Exploring”, “Sadness”, “Playing”, “Rage”, “Fear”, “Affectionate” and “Lust”, along with one additional emotion “Pain” which has been shown to be highly relevant for horses. We apply the contrastive learning framework MoCo (Momentum Contrast for Unsupervised Visual Representation Learning) on our dataset to predict the seven Panksepp emotions and “Pain” using unsupervised learning. We significantly modify the MoCo framework, building a custom downstream classifier network that connects with a frozen CNN encoder that is pretrained using MoCo. Our method allows the encoder network to learn similarities and differences within image groups on its own without labels. The clusters thus formed are indicative of deeper nuances and complexities within a horse’s mood, which can possibly hint towards the existence of novel and complex equine emotions
Quantifying the progress of artificial intelligence subdomains using the patent citation network
No full textEven though Artificial Intelligence (AI) has been having a transformative effect on human life, there is currently no precise quantitative method for measuring and comparing the performance of different AI methods. Technology Improvement Rate (TIR) is a measure that describes a technology’s rate of performance improvement, and is represented in a generalization of Moore’s Law. Estimating TIR is important for R&D purposes to forecast which competing technologies have a higher chance of success in the future. The present contribution estimates the TIR for different subdomains of applied and industrial AI by quantifying each subdomain’s centrality in the global flow of technology, as modeled by the Patent Citation Network and shown in previous work. The estimated TIR enables us to quantify and compare the performance improvement of different AI methods. We also discuss the influencing factors behind slower or faster improvement rates. Our results highlight the importance of Rule-based Machine Learning (not to be confused with Rule-based Systems), Multi-task Learning, Meta-Learning, and Knowledge Representation in the future advancement of AI and particularly in Deep Learning
Quantum Economic Advantage Calculator: An Extension of the Quantum Tortoise and Classical Hare Framework
No full textFor some algorithmic problems, quantum computation has the potential to provide enormous speedups over classical computers. However, the drastic slowdowns associated with running error-free quantum hardware make achieving these theoretical advantages challenging. Researchers and industry leaders planning for the future would benefit from understanding when it will be both feasible and advantageous to switch to quantum computing platforms. This thesis builds on the framework by Choi, Moses, and Thompson (2023) to evaluate the feasibility and timeline for achieving Quantum Economic Advantage (QEA)—the point at which quantum hardware can outperform comparably-priced classical machines for specific computational tasks. This thesis substantially extends and deepens this framework and introduces a calculator to make these analyses accessible. The model incorporates parameters from quantum hardware vendors, such as physical-logical qubit ratios and overall connectivity, alongside the computational complexities of specific problems, to estimate the year of QEA. Most of the parameters in the tool are freely adjustable, allowing users to explore how varying assumptions about quantum improvement and technological advancement influence the projected timeline for QEA.M.Eng
Sparse and Structured Tensor Programming
No full textFrom FORTRAN to NumPy, tensors have revolutionized how we express computation. However, tensors in these, and almost all prominent systems, can only handle dense rectilinear grids of values. Real-world tensors are often structured, containing patterns which allow us to optimize storage or computation, such as sparsity (mostly zero), runs of repeated values, or symmetry. Specializing implementations for structure yields significant speedups, but support for structured tensors is fragmented and incomplete. The heart of the problem is coiteration, simultaneously iterating over multiple tensors in a program, where each tensor format may have different internal structure. As each combination of structures requires a unique coiteration algorithm, existing frameworks struggle to abstract over the design space, instead hard-coding support for a few programs and/or a few structures. In this thesis, we build an abstraction for coiteration, enabling us to support both a wide range of programs and diverse tensor structures. We use a language, looplets, to describe the structure of tensors in tensor programs. Looplets allow the compiler to generate code to coiterate over any combination of structured tensor formats. The looplets language decomposes loops over sparse and structured formats hierarchically. This decomposition simplifies compilation, allowing us to capture key mathematical properties (such as x∗0 = 0, which motivates sparsity) with simple term rewriting. Building on looplets, we introduce a new language, Finch, for general structured tensor programming. Finch makes it easier to compute with structured tensors by combining program control flow and tensor structures into a common representation where they can be co-optimized. Finch automatically specializes control flow to data so that performance engineers can focus on experimenting with many algorithms. Finch supports a familiar programming language of loops, statements, ifs, breaks, etc., over a wide variety of tensor structures, such as sparsity, run-length-encoding, symmetry, triangles, padding, or blocks. Finch reliably utilizes the key properties of each structure, making it easier to write and optimize structured tensor programs. In our case studies, we show that this leads to dramatic speedups in diverse applications, including linear algebra, image processing, and graph analytics. Our abstracted design makes it easier to extend Finch to new tensor structures and programming models. Finch has been separately extended to support a DSL for symmetry-aware tensor programs and to support real-valued indexing.Ph.D
On Hing Travel Agency Fictional Archive of Disappearing Hong Kong
No full textHong Kong, shaped by rapid transformation and precarious land ownership, is a city where erasure defines its urban landscape. Amid this flux, a place I once called home was demolished, prompting the question: “How can one return to a place that no longer exists?” This thesis explores the transformative potential of disappearance, reframing it as a generative force that creates space for imagination, resistance, and continuity. Through On Hing Travel Agency (OHTA), demolished buildings "travel" into fictional worlds, becoming vessels of memory and imagination. Rooted in Hong Kong’s literary tradition—where fiction resists erasure and archives aspirations—the project employs fiction as both a tool of preservation and a site for belonging. Fictional destinations, inspired by Hong Kong novels, such as The Permanent City (1959), The Floating City (1986), and The Vanished Cities (2010), reflect pivotal historical moments while offering pathways to reconcile personal loss and master alternative spatial logics. The project culminates in the Lost Traveler’s Guide to Hong Kong, a publication curating maps, brochures, and layered narratives to immerse travelers in speculative thinking. By bridging the past and future, real and imagined, OHTA is a attempt to demonstrates how fiction can reclaim agency within the politics of disappearance, transforming loss into a catalyst for new narratives and creative engagement. Even in absence, Hong Kong’s disappearing spaces retain their resonance, generating new narratives and underscoring the creative potential of loss.M.Arch
Addressing Challenges in Object-Based Robot Navigation and Mapping
No full textDeveloping fully autonomous systems that can safely traverse and interact with the environment has been a long-term objective in robotics. Many relevant tasks, such as planning and mobile manipulation, require the robot to possess an object-level understanding of the ambient world. In particular, it would be crucial to maintain a globally consistent objectbased map of the environment for these operations. Without external assistance – such as a prior map or a motion capture system – the robot needs to navigate and map the environment using an object-based SLAM system. This thesis is dedicated to addressing several key challenges in developing object SLAM systems. The first challenge arises from the ambiguity of object poses in single-view observations. When an object is observed from a single vantage point, it can often have multiple probable poses due to symmetry, occlusion, or perceptual failures. It would be difficult for an object SLAM system to incorporate such ambiguous measurements. To address this issue, we introduce an ambiguity-aware object SLAM method. We use Gaussian max-mixture models to represent and efficiently track the multiple object pose hypotheses, and gradually disambiguate the poses to construct a globally consistent object-level map. The second challenge is the performance degradation of neural networks when deployed in novel robot operating environments, commonly known as the domain gap problem. Specifically, when a pre-trained 6DoF object pose estimator is used in a novel environment, its pose predictions are often corrupted by outliers, and quantifying their uncertainties becomes difficult. Using these noisy predictions with unmodeled uncertainties as measurements in an object SLAM system can lead to significant estimation errors. To mitigate the problem, we propose a SLAM-supported self-training pipeline for domain adaptation of 6DoF object pose estimators. We exploit robust pose graph optimization (PGO) results to pseudo-label robot-collected images and fine-tune 6D object pose estimators. In particular, we develop an Automatic Covariance Tuning (ACT) method to model pose prediction uncertainties automatically during the PGO process. The third challenge is environmental changes. As changes occur in the scene, such as object insertion, removal, or rearrangement, the robot needs to efficiently detect these changes and update the map accordingly. While detecting and reflecting scene changes is relatively straightforward with handcrafted map representations like point clouds or voxels, it becomes significantly more difficult with learned radiance-field-based scene representations, such as Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) models. In this thesis, we develop a radiance-field-based 3D change detection method to identify 3D object-level scene changes. Our approach can rapidly detect object changes in cluttered environments represented with radiance field models from as few as a single post-change image observation. We also develop efficient update methods for NeRF and 3DGS models to reflect physical object rearrangements, guided by sparse post-change images. By addressing these challenges, this thesis advances the robustness and adaptability of object SLAM systems in real-world environments, paving the way for more reliable and autonomous robotic systems capable of complex interactions with the environment.Ph.D
Engineering Scalable Quantum Systems From First-Principles to Large-Scale Control
No full textColor centers in solids are promising platforms for quantum communication, sensing, and computing, featuring highly coherent optical transitions, as well as native electron and nuclear spins that can be used as quantum memories. Existing state-of-the-art demonstrations have shown that multi-qubit control, spin-photon entanglement, and heralded entanglement are possible with devices consisting of a few color centers. However, the path to scaling the number of color centers integrated in these devices to the thousands or millions needed for advanced quantum networking and computing applications remains unclear. In particular, the requirement for highly coherent quantum operations both necessitates operation at cryogenic temperatures, and precise classical control signals delivered to each color center. Precise qubit control greatly increases the system complexity, while the cryogenic operation limits the amount of power that the system can dissipate. Both factors severely limit the number of color centers that can realistically be included in a single device using existing methods. This work will tackle the scaling problem from a system-level perspective from two directions. Firstly, I will quantify performance trade-offs between coherence, temperature, and optical properties of the group-IV color centers. A novel color center system, the ¹¹⁷SnV⁻ hyperfine color center, will be presented and its advantages compared to traditional group-IV color centers will be explored. Secondly, a method to integrate color centers with application specific integrated circuit (ASICs) will be demonstrated. The ASICs provides multiplexed control signals and increased control field efficiency, thus decreasing both the wiring complexity and thermal load per qubit. This work will thus pave the way to color center-based devices in which the number of qubits is not limited by the complexity or power dissipation of the control system.Ph.D
Design-technology Co-optimization for Sub-2 nm Technology Node Based on 2D Materials
No full textEmerging disruptive technologies such as Artificial Intelligence (AI) and 6G communications have driven stringent demands for hardware components that enable faster and more energy-efficient computation. With the diminishing returns of traditional silicon-based scaling and the escalating complexity of advanced semiconductor processes, two-dimensional (2D) materials offer promising opportunities when developed through Design-Technology Co-Optimization (DTCO). This thesis presents a comprehensive study of DTCO with a novel framework tailored for 2D material-based electronics that addresses critical challenges in material synthesis, device design, and circuit integration. In this framework, experimental material and device data are integrated into the design and optimization of MoS₂-based multichannel transistors (MCTs). With the help of DTCO, we have achieved record performance for double-gate, single-channel MoS₂ transistors as well as the first demonstration of high-performance, functional double channel MoS₂ transistors. Based on the results of MCTs, a Process Design Kit (PDK) is developed to facilitate circuit-level integration. These advancements constitute a promising foundation for the development of next-generation electronics beyond sub-2 nm technology node.S.M
Causal Inference with Survival Outcomes via Orthogonal Statistical Learning
No full textThe field of causal inference has recently made great strides in incorporating machine learning into confounding adjustment and estimation of heterogeneous treatment effects (HTE). However, there were some gaps regarding survival outcomes.
First, overlap-weighted effect estimators based on machine learning nuisance models were not available for such outcomes. Thus, researchers wishing to mitigate bias and variance from poor overlap had to accept potential bias from nuisance model misspecification in its place. In Chapter 2, we fill this gap by proposing a class of one-step cross-fitted double/debiased machine learning estimators for cumulative weighted average treatment effects for both survival outcomes and competing risk outcomes. Our approach combines importance sampling, semiparametric theory, and Neyman orthogonality to resolve both model misspecification and lack of covariate overlap between treatment arms in observational studies with censored outcomes. We give regularity conditions for the consistency, asymptotic linearity, and semiparametric efficiency bounds of the proposed estimators. Through simulation, it is shown that the proposed estimators do not require oracle parametric nuisance models. We apply the proposed estimators to compare the effects of two first-line anti-diabetic drugs on cancer outcomes.
Second, a wide range of machine learning methods (or ”learners”) for estimating heterogeneous treatment effects were not applicable to estimating effects on survival outcomes, particularly in the presence of competing risks. In Chapter 3, we fill this gap by developing several once-for-all (orthogonal) censoring unbiased transformations which convert time-to-event data into continuous outcomes, such that all HTE learners and oracle rates for continuous outcomes can be borrowed. Our approach not only reduces the pressing need to develop various HTE learners for censored outcomes and especially competing risks, but also fully leverage the state of the art of existing schemes. Through direct application of HTE learners to these transformed continuous outcomes, we obtain consistent estimates of heterogeneous cumulative incidence effects, total effects, and separable direct effects. We provide generic model-free learner-specific oracle inequalities bounding the finite-sample excess risk. The oracle efficiency results depend on the oracle selector and estimated nuisance functions from all steps involved in the transformation. We demonstrate the empirical performance of the proposed methods in simulation studies.
An important application area for causal inference methods, and one which originally motivated my interest in the field, is drug repurposing. In Chapter 4, we apply the methods of Chapter 2 to investigate whether metformin, a diabetes medication, might also have unexpected beneficial effects on cancer. The analysis encountered three major challenges: poor overlap between treatment groups, model misspecification, and pre-cancer death as competing risks for cancer incidence. To resolve these issues simultaneously, we take balancingweighted total cause-specific effects, controlled direct effect, and separable effects as causal estimands and develop balancing-weighted double/debiased machine learning estimators for both cumulative incidence functions and restricted mean time lost, with all estimators satisfying Neyman orthogonality. Using the Clinical Practice Research Datalink (CPRD) data, we find that metformin revealed a preventive direct effect on cancer incidence over sulfonylureas. The results also demonstrate the advantage of choosing the average treatment effect for the overlap population as the target quantity.
Finally, just as machine learning helps to automate nuisance model estimation for confounding adjustment and modeling effect heterogeneity, causally informed artificial intelligence (AI) and large language models (LLMs) might help to automate hypothesis generation for drug repurposing and surveillance opportunities. In Chapter 5, we explore this potential by developing a high-throughput screening approach to evaluate available drugs across multiple diseases. The screening methodology aims to identify drug-disease pairs with significant positive signals that could represent promising repurposing candidates, while also detecting pairs with negative signals that might indicate potential safety concerns–both being critical aspects for pharmacoepidemiology research. This systematic approach leverages the convergence of expanding healthcare data sources and modern data science advances to establish a data-driven framework for drug repurposing discovery and pharmacovigilance.
To conclude, we discuss the limitations of the proposed methods and provide possible future research directions.Ph.D