Helmholtz Center for Information Security
CISPA – Helmholtz-Zentrum für InformationssicherheitNot a member yet
3406 research outputs found
Sort by
When Machine Unlearning Jeopardizes Privacy
Get PDFThe right to be forgotten states that a data owner has the right to erase their data from an entity storing it. In the context of machine learning (ML), the right to be forgotten requires an ML model owner to remove the data owner's data from the training set used to build the ML model, a process known as machine unlearning. While originally designed to protect the privacy of the data owner, we argue that machine unlearning may leave some imprint of the data in the ML model and thus create unintended privacy risks. In this paper, we perform the first study on investigating the unintended information leakage caused by machine unlearning. We propose a novel membership inference attack that leverages the different outputs of an ML model's two versions to infer whether a target sample is part of the training set of the original model but out of the training set of the corresponding unlearned model. Our experiments demonstrate that the proposed membership inference attack achieves strong performance. More importantly, we show that our attack in multiple cases outperforms the classical membership inference attack on the original ML model, which indicates that machine unlearning can have counterproductive effects on privacy. We notice that the privacy degradation is especially significant for well-generalized ML models where classical membership inference does not perform well. We further investigate four mechanisms to mitigate the newly discovered privacy risks and show that releasing the predicted label only, temperature scaling, and differential privacy are effective. We believe that our results can help improve privacy protection in practical implementations of machine unlearning. Our code is available at this https URL
Quantifying and Mitigating Privacy Risks of Contrastive Learning
Get PDFData is the key factor to drive the development of machine learning (ML) during the past decade. However, high-quality data, in particular labeled data, is often hard and expensive to collect. To leverage large-scale unlabeled data, self-supervised learning, represented by contrastive learning, is introduced. The objective of contrastive learning is to map different views derived from a training sample (e.g., through data augmentation) closer in their representation space, while different views derived from different samples more distant. In this way, a contrastive model learns to generate informative representations for data samples, which are then used to perform downstream ML tasks. Recent research has shown that machine learning models are vulnerable to various privacy attacks. However, most of the current efforts concentrate on models trained with supervised learning. Meanwhile, data samples' informative representations learned with contrastive learning may cause severe privacy risks as well.
In this paper, we perform the first privacy analysis of contrastive learning through the lens of membership inference and attribute inference. Our experimental results show that contrastive models trained on image datasets are less vulnerable to membership inference attacks but more vulnerable to attribute inference attacks compared to supervised models. The former is due to the fact that contrastive models are less prone to overfitting, while the latter is caused by contrastive models' capability of representing data samples expressively. To remedy this situation, we propose the first privacy-preserving contrastive learning mechanism, Talos, relying on adversarial training. Empirical results show that Talos can successfully mitigate attribute inference risks for contrastive models while maintaining their membership privacy and model utility
Where We Stand (or Fall): An Analysis of CSRF Defenses in Web Frameworks
Get PDFCross-Site Request Forgery (CSRF) is among the oldest web vulnerabilities that, despite its popularity and severity, it is still an understudied security problem.
In this paper, we undertake one of the first security evaluations of CSRF defense as implemented by popular web frameworks, with the overarching goal to identify additional explanations to the occurrences of such an old vulnerability.
Starting from a review of existing literature, we identify 16 CSRF defenses and 18 potential threats agains them. Then, we evaluate the source code of the 44 most popular web frameworks across five languages (i.e., JavaScript, Python, Java, PHP, and C#) covering about 5.5 million LoCs, intending to determine the implemented defenses and their exposure to the identified threats. We also quantify the quality of web frameworks' documentation, looking for incomplete, misleading, or insufficient information required by developers to use the implemented CSRF defenses correctly.
Our study uncovers a rather complex landscape, suggesting that while implementations of CSRF defenses exist, their correct and secure use depends on developers' awareness and expertise about CSRF attacks. More than a third of the frameworks require developers to write code to use the defense, modify the configuration to enable CSRF defenses, or look for an external library as CSRF defenses are not built-in. Even when using defenses, developers need to be aware and address a diversity of additional security risks. In total, we identified 157 security risks in 37 frameworks, of which 17 are directly exploitable to mount a CSRF attack, leveraging implementation mistakes, cryptography-related flaws, cookie integrity, and leakage of CSRF tokens---including three critical vulnerabilities in CakePHP, Vert.x-Web, and Play. The developers' feedback indicate that, for a significant fraction of risks, frameworks have divergent expectations about who is responsible for addressing them. Finally, the documentation analysis reveals several inadequacies, including not mentioning the implemented defense, and not showing code examples for correct use
Boosting the Security of Blind Signature Schemes
Get PDFExisting blind signature schemes that are secure for poly- nomially many concurrent executions of the signing protocol are either inefficient or rely on non-standard assumptions (even in the random-oracle model). We show the first efficient blind signature schemes achieving this level of security based on the RSA, factoring, or discrete-logarithm assumptions (in the random-oracle model). Our core technique involves an extension and generalization of a transform due to Pointcheval (Eurocrypt ’98) that allows us to boost the number of signatures supported by a certain class of blind signature schemes
Automatically Detecting Bystanders in Photos to Reduce Privacy Risks
Get PDFPhotographs taken in public places often contain
bystanders~-- people who are not the main subject of a photo. These photos, when shared online,
can
reach a large number of viewers and potentially undermine the bystanders'
privacy. Furthermore, recent developments in computer vision and
machine learning can be used by online platforms to identify and track individuals. To combat this problem, researchers have proposed
technical solutions that require bystanders to be proactive and use
specific devices and/or applications to broadcast their privacy policy
and identifying information while being located in an image.
We explore the prospect of a different approach~--
identifying bystanders solely based on the visual information present
in an image. Through an online user study, we catalog the rationale
humans use to classify subjects and bystanders in an image, and
systematically validate a set of intuitive concepts (such as
intentionally posing for a photo) that can be used to
automatically identify bystanders.
Using image data, we infer those
concepts and then use them to train several classifier models. We
extensively evaluate the models and compare them with human raters. On
our training data set, which features a 10-fold cross validation, our best model
achieves a mean detection accuracy of 93% for images when human
raters have 100% agreement on the class label and 80% when the
agreement is only 67%. We validate this model on a completely different test data set and achieve similar results, demonstrating that our model generalizes well
Debugging Inputs
Get PDFWhen a program fails to process an input, it need not be the program code that is at fault. It can also be that the input data is faulty, for instance as result of data corruption. To get the data processed, one then has to debug the input data—that is, (1) identify which parts of the input data prevent processing, and (2) recover as much of the (valuable) input data as possible. In this paper, we present a general-purpose algorithm called ddmax that addresses these problems automatically. Through experiments, ddmax maximizes the subset of the input that can still be processed by the program, thus recovering and repairing as much data as possible; the difference between the original failing input and the “maximized” passing input includes all input fragments that could not be processed. To the best of our knowledge, ddmax is the first approach that fixes faults in the input data without requiring program analysis. In our evaluation, ddmax repaired about 69% of input files and recovered about 78% of data within one minute per input
Poster: Debugging Inputs
Get PDFProgram failures are often caused by invalid inputs, for instance due to input corruption. To obtain the passing input, one needs to debug the data. In this paper we present a generic technique called ddmax that (1) identifies which parts of the input data prevent processing, and (2) recovers as much of the (valuable) input data as possible. To the best of our knowledge, ddmax is the first approach that fixes faults in the input data without requiring program analysis. In our evaluation, ddmax repaired about 69% of input files and recovered about 78% of data within one minute per input
Abstracting Failure-Inducing Inputs
Get PDFA program fails. Under which circumstances does the failure occur? Starting with a single failure-inducing input "The input '((4))' fails") and an input grammar, the DDSET algorithm uses systematic tests to automatically generalize the input to an _abstract failure-inducing input_ that contains both (concrete) terminal symbols and (abstract) nonterminal symbols from the grammar - for instance, "(())", which represents any expression in double parentheses. Such an abstract failure-inducing input can be used
1. as a *debugging diagnostic*, characterizing the circumstances under which a failure occurs ("The error occurs whenever an expression is enclosed in double parentheses");
2. as a *producer* of additional failure-inducing tests to help design and validate fixes and repair candidates ("The inputs '((1))', '((3 * 4))', and many more also fail").
In its evaluation on real-world bugs in JavaScript, Clojure, Lua, and Coreutils, DDSET's abstract failure-inducing inputs provided to-the-point diagnostics, and precise producers for further failure inducing inputs
User Mental Models of Cryptocurrency Systems - A Grounded Theory Approach
Get PDFFrequent reports of monetary loss, fraud, and user-caused security incidents in the context of cryptocurrencies emphasize the need for human-centered research in this domain.
We contribute the first qualitative user study (N=29) on user mental models of cryptocurrency systems and the associated threat landscape. Using Grounded Theory, we reveal misconceptions affecting users' security and privacy.
Our results suggest that current cryptocurrency tools (e.g., wallets and exchanges) are not capable of counteracting threats caused by these misconceptions. Hence, users frequently fail to securely manage their private keys or assume to be anonymous when they are not. Based on our findings, we contribute actionable advice, grounded in the mental models of users, to improve the usability and secure usage of cryptocurrency systems
Towards Automated Testing and Robustification by Semantic Adversarial Data Generation
No full textWidespread application of computer vision systems in real world tasks is currently hindered by their unexpected behavior on unseen examples.
This occurs due to limitations of empirical testing on finite test sets and lack of systematic methods to identify the breaking points of a trained model.
In this work we propose semantic adversarial editing, a method to synthesize plausible but difficult data points on which our target model breaks down.
We achieve this with a differentiable object synthesizer allowing to modify appearances of an object instance while maintaining its original pose.
Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object instance which fool the target object detector.
Experiments show that our approach is effective in synthesizing difficult test data, dropping the performance of YoloV3 detector by more than 20 mAP points by changing the appearance of a single object and discovering failure modes of the model.
We also demonstrate that the generated semantic adversarial data can be used to robustify the detector through data
augmentation, consistently improving its performance in both standard and out-of-dataset-distribution test sets, across three different datasets