Linköping Electronic Conference Proceedings
Not a member yet
1113 research outputs found
Sort by
On Population Fidelity as an Estimator for the Utility of Synthetic Training Data
Get PDFSynthetic data promises to address several challenges in training machine learning models, such as data scarcity, privacy concerns, and efforts for data collection and annotation. In order to actually benefit from synthetic data, its utility for the intended purpose has to be ensured and, ideally, estimated before it is used to produce possibly poorly performing models. Population fidelity metrics are potential candidates to provide such an estimation. However, evidence of how well they are suited as estimators of the utility of synthetic data is scarce.
In this study, we present the results of an experiment in which we investigated whether population fidelity as measured with nine different metrics correlates with the predictive performance of classification models trained on synthetic data.
Cluster Analysis and Cross-Classification show the most consistent results w.r.t. correlation with F1-performance but do not exceed moderate levels.The degree of correlation, and hence the potential suitability for estimating utility, varies considerably across the inspected datasets. Overall, the results suggest that the inspected population fidelity metrics are not a reliable and accurate tool to estimate the utility of synthetic training data for classification tasks. They may be precise enough though to indicate trends for different synthetic datasets based on the same original data.
Further research should shed light on how different data properties affect the ability of population fidelity metrics to estimate utility and make recommendations on how to use these metrics for different scenarios and types of datasets
Sailing through multiword expression identification with Wiktionary and Linguse: A case study of language learning
Get PDFMultiword expressions (MWEs), due to their idiomatic nature, pose particular challenges in comprehension tasks and vocabulary acquisition for language learners. Current NLP tools fall short off comprehensively aiding language learners when encountering MWEs. While proficient in identifying MWEs seen during training, current systems are constrained by limited training data. To address the specific needs of language learners, this research integrates expansive MWE lexicons and NLP methodologies as championed by Savary et al. (2019a). Outcomes encompass a specialized MWE corpus from Wiktionary, the enhancement of Linguse, a reading application for language learners, with MWE annotations, and empirical validation with French language students. The culmination is an MWE identifier optimally designed for language learner requirements
Evolutionary Optimization of Artificial Neural Networks and Tree-Based Ensemble Models for Diagnosing Deep Vein Thrombosis
Get PDFMachine learning algorithms, particularly artificial neural networks, have shown promise in healthcare for disease classification, including diagnosing conditions like deep vein thrombosis. However, the performance of artificial neural networks in medical diagnosis heavily depends on their architecture and hyperparameter configuration, which presents virtually unlimited variations. This work employs evolutionary algorithms to optimize hyperparameters for three classic feed-forward artificial neural networks of pre-determined depths. The objective is to enhance the diagnostic accuracy of the classic neural networks in classifying deep vein thrombosis using electronic health records sourced from a Norwegian hospital. The work compares the predictive performance of conventional feed-forward artificial neural networks with standard tree-based ensemble methods previously successful in disease prediction on the same dataset. Results indicate that while classic neural networks perform comparably to tree-based methods, they do not surpass them in diagnosing thrombosis on this specific dataset. The efficacy of evolutionary algorithms in tuning hyperparameters is highlighted, emphasizing the importance of choosing the optimization technique to maximize machine learning models' diagnostic accuracy
Digital History and Immaterial Infrastructure: A Bottom-Up Approach
Get PDFThis paper argues for an expanded view of research infrastructure. Drawing on our experiences leading the research platform DigitalHistory@Lund, it shows how research capacity can be unlocked “bottom-up”, by providing scholars with comparatively cheap—yet often inaccessible— technological support. By engaging researchers in digitally enabled scholarly practices, the platform yielded a multiplying effect that has seen participants produce highly competitive grant applications and eventually bring home external funding currently worth eight times the platform’s original costs. The platform thus demonstrates the importance of “immaterial” infrastructure in the sense of basic organisational structures that facilitate collaboration and communication
Developing a Web-Based Intelligent Language Assessment Platform Powered by Natural Language Processing Technologies
Get PDFWe introduce System, an intelligent language assessment platform and reusable module that streamlines the creation, administration and scoring of language proficiency tests supported by Natural Language Processing (NLP) technologies. As a first implementation, we realized an automatic pipeline for the Elicited Imitation Test (EIT), a popular test format that has been widely adopted in language learning research for general proficiency and formative assessments. The platform can be extended to other test formats and assessment types. System is a valuable tool for standardizing data collection in Second Language Acquisition (SLA) and Intelligent Computer Assisted Language Learning (ICALL) research as well as serving as an application for classroom assessment. In this paper, we present the design of the system and a preliminary evaluation of LLMs for generating language errors in EIT items. We conclude with a future outlook as well as limitations. 
Exploring demonstration pre-training with improved Deep Q-learning
Get PDFThis study explores the effects of incorporating demonstrations as pre-training of an improved Deep Q-Network (DQN). Inspiration is taken from methods such as Deep Q-learning from Demonstrations (DQfD), but instead of retaining the demonstrations throughout the training, the performance and behavioral effects of the policy when using demonstrations solely as pre-training are studied. A comparative experiment is performed on two game environments, Gymnasium's Car Racing and Atari Space Invaders. While demonstration pre-training in Car Racing shows improved learning efficacy, as indicated by higher evaluation and training rewards, these improvements do not show in Space Invaders, where it instead under-performed. This divergence suggests that the nature of a game's reward structure influences the effectiveness of demonstration pre-training. Interestingly, despite less pronounced quantitative differences, qualitative observations suggested distinctive strategic behaviors, notably in target elimination patterns in Space Invaders. These retained behaviors seem to get forgotten during extended training. The results show that we need to investigate further how exploration functions affect the effectiveness of demonstration pre-training, how behaviors can be retained without explicitly making the agent mimic demonstrations, and how non-optimal demonstrations can be incorporated for more stable learning with demonstrations
Weight Rescaling: Applying Initialization Strategies During Training
Get PDFThe training success of deep learning is known to depend on the initial statistics of neural network parameters. Various strategies have been developed to determine suitable mean and standard deviation for weight distributions based on network architecture. However, during training, weights often diverge from their initial scale. This paper introduces the novel concept of weight rescaling, which enforces weights to remain within their initial regime throughout the training process. It is demonstrated that weight rescaling serves as an effective regularization method, reducing overfitting and stabilizing training while improving neural network performance. The approach rescales weight vector magnitudes to match the initialization methods’ conditions without altering their direction. It exhibits minimal memory usage, is lightweight on computational resources and demonstrates comparable results to weight decay, but without introducing additional hyperparameters as it leverages architectural information. Empirical testing shows improved performance across various architectures, even when combined with additional regularization methods like dropout in AlexNet and batch normalization in ResNet-50. The effectiveness of weight rescaling is further supported by a thorough statistical evaluation
Local Interpretable Model-Agnostic Explanations for Neural Ranking Models
Get PDFNeural Ranking Models have shown state-of-the-art performance in Learning-To-Rank (LTR) tasks. However, they are considered black-box models. Understanding the logic behind the predictions of such black-box models is paramount for their adaptability in the real-world and high-stake decision-making domains. Local explanation techniques can help us understand the importance of features in the dataset relative to the predicted output of these black-box models. This study investigates new adaptations of Local Interpretable Model-Agnostic Explanation (LIME) explanation for explaining Neural ranking models. To evaluate our proposed explanation, we explain Neural GAM models. Since these models are intrinsically interpretable Neural Ranking Models, we can directly extract their ground truth importance scores. We show that our explanation of Neural GAM models is more faithful than explanation techniques developed for LTR applications such as LIRME and EXS and non-LTR explanation techniques for regression models such as LIME and KernelSHAP using measures such as Rank Biased Overlap (RBO) and Overlap AUC. Our analysis is performed on the Yahoo! Learning-To-Rank Challenge dataset
Machine Learning for Lithology Analysis using a Multi-Modal Approach of Integrating XRF and XCT data
Get PDFWe explore the use of various machine learning (ML) models for classifying lithologies utilizing data from X-ray fluorescence (XRF) and X-ray computed tomography (XCT). Typically, lithologies are identified over several meters, which restricts the use of ML models due to limited training data. To address this issue, we augment the original interval dataset, where lithologies are marked over extensive sections, into finer segments of 10cm, to produce a high resolution dataset with vastly increased sample size. Additionally, we examine the impact of adjacent lithologies on building a more generalized ML model. We also demonstrate that combining XRF and XCT data leads to an improved classification accuracy compared to using only XRF data, which is the common practice in current studies, or solely relying on XCT data
Konsten att bedriva svensk ordforskning utan att kränka upphovsrätten
Get PDFVi beskriver KB-labb och Språkbanken Texts samarbete för att underlätta ordforskning på de upphovsrätts-skyddade korpusar som finns i Kungliga bibliotekets samlingar. Satsningen har hittils lett till två öppna datasamlingar, Kubord 1 och 2, som ger tillgång till ordstatistik och ordsamförekomststatistik. Vi beskriver även Kubord-fastText, en samling vektormodeller som är baserade på samma korpusar, som är under utveckling