32 research outputs found
Glacier : guided locally constrained counterfactual explanations for time series classification
No full textIn machine learning applications, there is a need to obtain predictive models of high performance and, most importantly, to allow end-users and practitioners to understand and act on their predictions. One way to obtain such understanding is via counterfactuals, that provide sample-based explanations in the form of recommendations on which features need to be modified from a test example so that the classification outcome of a given classifier changes from an undesired outcome to a desired one. This paper focuses on the domain of time series classification, more specifically, on defining counterfactual explanations for univariate time series. We propose Glacier, a model-agnostic method for generating locally-constrained counterfactual explanations for time series classification using gradient search either on the original space or on a latent space that is learned through an auto-encoder. An additional flexibility of our method is the inclusion of constraints on the counterfactual generation process that favour applying changes to particular time series points or segments while discouraging changing others. The main purpose of these constraints is to ensure more reliable counterfactuals, while increasing the efficiency of the counterfactual generation process. Two particular types of constraints are considered, i.e., example-specific constraints and global constraints. We conduct extensive experiments on 40 datasets from the UCR archive, comparing different instantiations of Glacier against three competitors. Our findings suggest that Glacier outperforms the three competitors in terms of two common metrics for counterfactuals, i.e., proximity and compactness. Moreover, Glacier obtains comparable counterfactual validity compared to the best of the three competitors. Finally, when comparing the unconstrained variant of Glacier to the constraint-based variants, we conclude that the inclusion of example-specific and global constraints yields a good performance while demonstrating the trade-off between the different metrics. © The Author(s) 2024.This work was funded in part by the Digital Futures cross-disciplinary research centre in Sweden, and the EXTREMUM collaborative project ( https://datascience.dsv.su.se/projects/extremum.html ).</p
isaksamsten/wildboar: wildboar
No full textwildboar is a fast package for time series classification with Python
The package can be installed from PyPi
pip install wildboa
Assessment of double materiality
No full textSustainability reporting standards, e.g. the Global Reporting Initiative, require a broader definition of materiality than is traditionally used in financial reporting. Double materiality expands the material information concept to include information about companies' environmental and social impact relevant to society at large. A problem for reporting companies as well as auditors (even though accounting firms invest resources in establishing themselves as reliable service providers) is that the assessment of double materiality is uncertain. The chapter utilises machine learning methods to suggest a method to determine double materiality in sustainability reporting by examining what type of information can predict environmental issues resulting from companies' operations. It represents a proposal to use a structured and quantitative approach for sustainability auditors to determine double materiality, thereby potentially facilitating sustainability reporting and assurance in accordance with future regulation.</p
Code quality assessment using transformers
No full textAutomatically evaluate the correctness of programming assignments is rather
straightforward using unit and integration tests. However, programming tasks
can be solved in multiple ways, many of which, although correct, are inelegant.
For instance, excessive branching, poor naming or repetitiveness make the code
hard to understand and maintain. These subjective qualities of code are hard to
automatically assess using current techniques. In this work we investigate the
use of CodeBERT to automatically assign quality score to Java code. We
experiment with different models and training paradigms. We explore the
accuracy of the models on a novel dataset for code quality assessment. Finally,
we assess the quality of the predictions using saliency maps. We find that code
quality to some extent is predictable and that transformer based models using
task adapted pre-training can solve the task more efficiently than other
techniques
Surveillance of communicable diseases using social media: A systematic review
No full textBackground Communicable diseases pose a severe threat to public health and economic growth. The traditional methods that are used for public health surveillance, however, involve many drawbacks, such as being labor intensive to operate and resulting in a lag between data collection and reporting. To effectively address the limitations of these traditional methods and to mitigate the adverse effects of these diseases, a proactive and real-time public health surveillance system is needed. Previous studies have indicated the usefulness of performing text mining on social media. Objective To conduct a systematic review of the literature that used textual content published to social media for the purpose of the surveillance and prediction of communicable diseases.MethodologyBroad search queries were formulated and performed in four databases. Both journal articles and conference materials were included. The quality of the studies, operationalized as reliability and validity, was assessed. This qualitative systematic review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Results Twenty-three publications were included in this systematic review. All studies reported positive results for using textual social media content to surveille communicable diseases. Most studies used Twitter as a source for these data. Influenza was studied most frequently, while other communicable diseases received far less attention. Journal articles had a higher quality (reliability and validity) than conference papers. However, studies often failed to provide important information about procedures and implementation. Conclusion Text mining of health-related content published on social media can serve as a novel and powerful tool for the automated, real-time, and remote monitoring of public health and for the surveillance and prediction of communicable diseases in particular. This tool can address limitations related to traditional surveillance methods, and it has the potential to supplement traditional methods for public health surveillance.Validerad;2023;Nivå 2;2023-03-07 (joosat);Licens fulltext: CC BY License</p
Example-Based Feature Tweaking Using Random Forests
No full textIn certain application areas when using predictive models, it is not enough to make an accurate prediction for an example, instead it might be more important to change a prediction from an undesired class into a desired class. In this paper we investigate methods for changing predictions of examples. To this end, we introduce a novel algorithm for changing predictions of examples and we compare this novel method to an existing method and a baseline method. In an empirical evaluation we compare the three methods on a total of 22 datasets. The results show that the novel method and the baseline method can change an example from an undesired class into a desired class in more cases than the competitor method (and in some cases this difference is statistically significant). We also show that the distance, as measured by the euclidean norm, is higher for the novel and baseline methods (and in some cases this difference is statistically significantly) than for state-of-the-art. The methods and their proposed changes are also evaluated subjectively in a medical domain with interesting results.</p
Learning Time Series Counterfactuals via Latent Space Representations
No full textCounterfactual explanations can provide sample-based explanations of features required to modify from the original sample to change the classification result from an undesired state to a desired state; hence it provides interpretability of the model. Previous work of LatentCF presents an algorithm for image data that employs auto-encoder models to directly transform original samples into counterfactuals in a latent space representation. In our paper, we adapt the approach to time series classification and propose an improved algorithm named LatentCF++ which introduces additional constraints in the counterfactual generation process. We conduct an extensive experiment on a total of 40 datasets from the UCR archive, comparing to current state-of-the-art methods. Based on our evaluation metrics, we show that the LatentCF++ framework can with high probability generate valid counterfactuals and achieve comparable explanations to current state-of-the-art. Our proposed approach can also generate counterfactuals that are considerably closer to the decision boundary in terms of margin difference.</p
Assessing the Clinical Validity of Attention-based and SHAP Temporal Explanations for Adverse Drug Event Predictions
No full textAttention mechanisms form the basis of providing temporal explanations for a variety of state-of-the-art recurrent neural network (RNN) based architectures. However, evidence is lacking that attention mechanisms are capable of providing sufficiently valid medical explanations. In this study we focus on the quality of temporal explanations for the medical problem of adverse drug event (ADE) prediction by comparing explanations globally and locally provided by an attention-based RNN architecture against those provided by more a more basic RNN using the post-hoc SHAP framework, a popular alternative option which adheres to several desirable explainability properties. The validity of this comparison is supported by medical expert knowledge gathered for the purpose of this study. This investigation has uncovered that these explanation methods both possess appropriateness for ADE explanations and may be used complementarily, due to SHAP providing more clinically appropriate global explanations and attention mechanisms capturing more clinically appropriate local explanations. Additional feedback from medical experts reveal that SHAP may be more applicable to real-time clinical encounters, in which efficiency must be prioritised, over attention explanations which possess properties more appropriate for offline analyses.</p
Counterfactual Explanations for Survival Prediction of Cardiovascular ICU Patients
No full textIn recent years, machine learning methods have been rapidly implemented in the medical domain. However, current state-of-the-art methods usually produce opaque, black-box models. To address the lack of model transparency, substantial attention has been given to develop interpretable machine learning methods. In the medical domain, counterfactuals can provide example-based explanations for predictions, and show practitioners the modifications required to change a prediction from an undesired to a desired state. In this paper, we propose a counterfactual explanation solution for predicting the survival of cardiovascular ICU patients, by representing their electronic health record as a sequence of medical events, and generating counterfactuals by adopting and employing a text style-transfer technique. Experimental results on the MIMIC-III dataset strongly suggest that text style-transfer methods can be effectively adapted for the problem of counterfactual explanations in healthcare applications and can achieve competitive performance in terms of counterfactual validity, BLEU-4 and local outlier metrics. </p
Distributional Data Augmentation Methods for Low Resource Language
No full textText augmentation is a technique for constructing synthetic data from an
under-resourced corpus to improve predictive performance. Synthetic data
generation is common in numerous domains. However, recently text augmentation
has emerged in natural language processing (NLP) to improve downstream tasks.
One of the current state-of-the-art text augmentation techniques is easy data
augmentation (EDA), which augments the training data by injecting and replacing
synonyms and randomly permuting sentences. One major obstacle with EDA is the
need for versatile and complete synonym dictionaries, which cannot be easily
found in low-resource languages. To improve the utility of EDA, we propose two
extensions, easy distributional data augmentation (EDDA) and type specific
similar word replacement (TSSR), which uses semantic word context information
and part-of-speech tags for word replacement and augmentation. In an extensive
empirical evaluation, we show the utility of the proposed methods, measured by
F1 score, on two representative datasets in Swedish as an example of a
low-resource language. With the proposed methods, we show that augmented data
improve classification performances in low-resource settings.Comment: AAAI 2023 Workshop on Knowledge Augmented Methods for NL
