ISTA Research Explorer (Institute of Science and Technology Austria)
Not a member yet
6140 research outputs found
Sort by
ResiDual transformer alignment with spectral decomposition
No full textWhen examined through the lens of their residual streams, a puzzling property emerges in transformer networks: residual contributions (e.g., attention heads) sometimes specialize in specific tasks or input attributes. In this paper, we analyze this phenomenon in vision transformers, focusing on the spectral geometry of residuals, and explore its implications for modality alignment in vision-language models. First, we link it to the intrinsically low-dimensional structure of visual head representations, zooming into their principal components and showing that they encode specialized roles across a wide variety of input data distributions. Then, we analyze the effect of head specialization in multimodal models, focusing on how improved alignment between text and specialized heads impacts zero-shot classification performance. This specialization-performance link consistently holds across diverse pre-training data, network sizes, and objectives, demonstrating a powerful new mechanism for boosting zero-shot classification through targeted alignment. Ultimately, we translate these insights into actionable terms by introducing ResiDual, a technique for spectral alignment of the residual stream. Much like panning for gold, it lets the noise from irrelevant unit principal components (i.e., attributes) wash away to amplify task-relevant ones. Remarkably, this dual perspective on modality alignment yields fine-tuning level performance on different data distributions while modelling an extremely interpretable and parameter-efficient transformation, as we extensively show on 70 pre-trained network-dataset combinations (7 models, 10 datasets)
LNCS
Full text linkCooperative verification is gaining momentum in recent years. The usual setup in cooperative verification is that a verifier A is run with some pre-defined resources, and if it is not able to verify the program, the verification task is passed to a verifier B together with information learned about the program by verifier A, then the chain can continue to a verifier C, and so on. This scheme is static: tools run one after another in a fixed pre-defined order and fixed parameters and resource limits (the scheme may differ for properties to be analyzed, though).
Bubaak is a program analysis tool that allows to run multiple program verifiers in a dynamically changing combination of parallel and sequential portfolios. Bubaak starts the verification process by invoking an initial set of tasks; every task, when it is done (e.g., because of hitting a time limit or finishing its job), rewrites itself into one or more successor tasks. New tasks can be also spawned upon events generated by other tasks. This all happens dynamically based on the information gathered by finished and running tasks. During their execution, tasks that run in parallel can exchange (partial) verification artifacts, either directly or with Bubaak as an intermediary
Subgap transport in superconductor-semiconductor hybrid islands: Weak and strong coupling regimes
Full text linkSuperconductor–semiconductor hybrid systems play a crucial role in realizing nanoscale quantum devices, including hybrid qubits, Majorana bound states, and Kitaev chains. For such hybrid devices, subgap states play a prominent role in their operation. In this paper, we study these subgap states via Coulomb and tunneling spectroscopy through a superconducting island defined in a semiconductor nanowire fully coated by a superconductor. We systematically explore regimes ranging from an almost decoupled island to the open configuration. In the weak-coupling regime, the experimental observations are very similar in the absence of a magnetic field and when one flux quantum pierces the superconducting shell. Conversely, in the strong-coupling regime, significant distinctions emerge between the two cases. We attribute this distinct behavior to the existence of subgap states at one flux quantum, which become observable only for sufficiently strong coupling to the leads. We support our interpretation using a simple model to describe transport through the island. Our study highlights the importance of studying a broad range of tunnel couplings for understanding the rich physics of hybrid devices
LIPIcs
Full text linkIn numerous fields, dynamic time series data require continuous updates, necessitating efficient data processing techniques for accurate analysis. This paper examines the banana tree data structure, specifically designed to efficiently maintain the multi-scale topological descriptor commonly known as persistent homology for dynamically changing time series data. We implement this data structure and conduct an experimental study to assess its properties and runtime for update operations. Our findings indicate that banana trees are highly effective with unbiased random data, outperforming state-of-the-art static algorithms in these scenarios. Additionally, our results show that real-world time series share structural properties with unbiased random walks, suggesting potential practical utility for our implementation
Sustainable development key to limiting climate change-driven wildfire damages
Full text linkClimate change is causing wildfires to become more frequent and intense. While predicting burned areas using bioclimatic and anthropogenic factors is an active research area, few studies have examined what drives the economic damages of wildfires. Our study aims to fill this gap by analyzing key factors influencing global economic wildfire damages and projecting future damages under three shared socioeconomic pathways (SSPs). We apply regression analyses to identify significant predictors of economic wildfire damages at country levels and use the fitted model to project future damages under SSP126, SSP245, and SSP370. Results show that the human vulnerability index (HVI), reflecting socioeconomic conditions, is the strongest predictor of historical wildfire damages, followed by water vapor pressure deficit during the fire season and population density around forested areas. We found high population density to be associated with lower damages. These findings contrast with studies of burned areas, where climate factors are more dominant. Our model projects that by 2070, average global economic wildfire damages will be three times higher under SSP370 than SSP126. Our model also shows that following SSP126 not only reduces wildfire damages but also lessens the inequalities in damage distribution across countries. This pathway’s dual focus on equitable socioeconomic progress and climate action potentially enhances a country’s resilience that helps mitigate wildfire damages. Our analyses also indicate that strong socioeconomic development can offset wildfire damages associated with climate hazards, although this is less certain under SSP370. SSP126’s integrated approach improves both socioeconomic conditions and limits global warming, providing substantial benefits to less developed countries while still reducing damages in developed nations, despite their already low HVI scores. Our work complements existing research on burned areas and underscores the importance of sustainable development and international collaboration in reducing the economic damages of wildfires
High-dimensional analysis of knowledge distillation: Weak-to-Strong generalization and scaling laws
Full text linkA growing number of machine learning scenarios rely on knowledge distillation where one uses the output of a surrogate model as labels to supervise the training of a target model. In this work, we provide a sharp characterization of this process for ridgeless, high-dimensional regression, under two settings: (i) model shift, where the surrogate model is arbitrary, and (ii) distribution shift, where the surrogate model is the solution of empirical risk minimization with out-of-distribution data. In both cases, we characterize the precise risk of the target model through non-asymptotic bounds in terms of sample size and data distribution under mild conditions. As a consequence, we identify the form of the optimal surrogate model, which reveals the benefits and limitations of discarding weak features in a data-dependent fashion. In the context of weak-to-strong (W2S) generalization, this has the interpretation that (i) W2S training, with the surrogate as the weak model, can provably outperform training with strong labels under the same data budget, but (ii) it is unable to improve the data scaling law. We validate our results on numerical experiments both on ridgeless regression and on neural network architectures
Data of: "Quantifying the carbon footprint of conference travel: the case of NMR meetings"
Full text linkThis repository contains calculations of carbon footprints of NMR conferences, as described in the article by
Lucky N. Kapoor, Natalia Ruzickova, Predrag Živadinović, Valentin Leitner, Maria Anna Sisak, Cecelia Mweka, Jeroen Dobbelaere, Georgios Katsaros, and Paul Schanda
Published in Magnetic Resonance, 2025
Image-based 3D active sample stabilization on the nanometer scale for optical microscopy
Full text linkSuper-resolution microscopy often entails long acquisition times of minutes to hours. Since drifts during the acquisition adversely affect data quality, active sample stabilization is commonly used for some of these techniques to reach their full potential. Although drifts in the lateral plane can often be corrected after acquisition, this is not always possible or may come with drawbacks. Therefore, it is appealing to stabilize sample position in three dimensions (3D) during acquisition. Various schemes for active sample stabilization have been demonstrated previously, with some reaching sub-nanometer stability in 3D. Here, we present a scheme for active drift correction that delivers the nanometer-scale 3D stability demanded by state-of-the-art super-resolution techniques and is straightforward to implement compared to previous schemes capable of reaching this level of stabilization precision. Using a refined algorithm that can handle various types of reference structure, without sparse signal peaks being mandatory, we stabilized sample position to ∼1 nm in 3D using objective lenses both with high and low numerical aperture. Our implementation requires only the addition of a simple widefield imaging path and we provide an open-source control software with graphical user interface to facilitate easy adoption of the module. Finally, we demonstrate how this has the potential to enhance data collection for diffraction-limited and super-resolution imaging techniques using single-molecule localization microscopy and cryo-confocal imaging as showcases
ISTA Thesis
Full text linkThe internal structure of biomolecules and their organization in higher-order arrangements are key factors governing the working principles of biological systems. Bioimaging has successfully revealed arrangements across relevant spatial scales. For example, cryo-electron tomography has become widely used for analyzing biomolecular structures in situ due to its comprehensive structural visualization of near-natively preserved samples, and its capability of sub-nm resolution via averaging. However, the identification of molecules within crowded cellular environments is often hindered by low contrast. Fluorescence microscopy, on the other hand, routinely visualizes specifically labeled targets at single-molecule contrast against essentially zero background. Moreover, it provides comparatively high throughput and is amenable to multiplexing. Due to this complementarity, combining datasets from both modalities acquired on the same region via correlative light and electron microscopy can reveal novel types of information.
The spatial scale at which information can be extracted depends on imaging resolution and correlation accuracy. Since diffraction of light limits the resolution of conventional fluorescence microscopy to few hundreds of nanometers, reaching the full potential of correlative imaging requires super-resolution approaches. Performing imaging at cryogenic temperature preserves structures in a near-native state and minimizes distortions between the fluorescence and the electron microscopy datasets. Implementations of this concept have achieved correlation on the scale of cellular organelles or bacterial domains.
We have worked towards pushing correlative imaging to the single-molecule scale by improving cryo-super-resolution microscopy, and devising a refined image correlation workflow. As part of this project, I constructed a microscopy setup and adopted it for super-resolution fluorescence microscopy at room temperature and cryogenic conditions. I explored different cryo-stages and acquisition strategies. Specifically, I developed a new scheme for correcting sample drift, thus increasing mechanical stability during microscopy acquisitions