1,720,988 research outputs found
HPC IO and seismic data performance optimization using ANNs prediction based auto-tuning
No full textHPC or super-computing clusters are designed for executing computationally intensive operations that typically involve large scale IO operations. This most commonly involves using a standard MPI library implemented in C/C++. The MPI-IO performance in HPC clusters tends to vary significantly over a range of configuration parameters that are generally not considered by the algorithm. It is commonly left to individual practitioners to optimise IO on a case-by-case basis at code level. This can often lead to a range of unforeseen outcomes, specifically when it comes to the manual tuning of the configuration parameters. The ExSeisDat utility is built on top of the native MPI-IO library comprising of Parallel IO and Workflow Libraries to process seismic data encapsulated in SEG-Y file format. The SEG-Y File data structure is complex in nature, due to the alternative arrangement of trace header and trace data. Its size scales to petabytes and the chances of IO performance degradation are further increased by ExSeisDat. The aim of this research is to auto-tune the Parallel IO configurations which determine the gain in bandwidth performance, without requiring the user programmer’s interven tion. This research thesis presents a novel study of the changing IO performance in terms of bandwidth, with the use of parallel plots against various MPI-IO, Lustre (Parallel) File System and SEG-Y File configuration parameters. Another novel aspect of this research is the predictive modelling of parallel (MPI) IO, and ExSeisDat’s SEG-Y IO and file sorting bandwidth performance behaviour using Artificial Neural Networks (ANNs). In continuation to this, the auto-tuning parameters strategy is designed, which is based on the ANN models predictions. This is an innovative approach to optimize Parallel IO bandwidth performance for MPI-IO, seismic (SEG-Y) data IO and file sorting operations. The results presented in the thesis, show significant performance gain through sta tistical analysis, in the Parallel IO bandwidth within the HPC cluster. Additionally, this research has highlighted the common most useful configuration settings which give the highest probability of improving the IO performance, in the event ML models are unavailable for bandwidth predictions. Furthermore, the different ANN hidden layer node configurations have also been dis cussed with respect to identified SEG-Y operations, for separate and combined READ- /WRITE executions, which show maximum average bandwidth performances
Leveraging nature-inspired techniques to semi-automate the design of multi-agent systems
No full textMulti-agent systems (MAS) applications are used in many fields, from telecommu nications network management to traffic simulation. Their utilisation is increasing with emerging technologies such as Edge computing and the Internet of Things. This research aims to enhance the use of nature-inspired techniques that semi-automate the development of decentralised heuristics for resolving complex systems. It explores a series of frontiers in these areas focusing on different techniques and problem domains. The Gossip Contracts (GC) protocol is a novel communication protocol for multi agent systems to facilitate decentralised cooperation strategies. It is a generic protocol that can be applied to many problems. Hence, it was used to develop a novel de centralised dynamic virtual machine consolidation (DVMC) strategy. This strategy was compared to two well-known DVMC strategies, Sercon and ecoCloud, using a cloud data centre model derived from a real-world dataset. The assessment considered various cri teria and found that the GC-based DVMC had lower service level agreement violations than the other strategies. The Evolved Gossip Contracts (EGC) framework is a novel evolutionary computing based framework for designing decentralised heuristics. Using EGC, a decentralised heuristic for the NP-hard bin packing problem (BPP) was developed and shown to be better than two popular BPP heuristics, Best Fit and First Fit, for the datasets on which it was trained. Based on a literature review, EGC is the first evolutionary computing based framework evaluated on an NP-hard problem. The QD(λ) learning algorithm is a novel value-based MARL algorithm that leverages networked agents. It incorporates the advantages of transfer learning from the MARL QD learning algorithm and faster learning when rewards are delayed from the RL Q(λ) learning algorithm. It attained the lowest accumulative error for larger environments that require agents to transition through several states before receiving a reward. Thus, QD(λ) learning is suited to sizeable multi-agent systems learning a communication pro tocol, an emerging research area that has seen significant activity in the last few years
Machine learning for De Novo peptide identification
No full textProteomics involves the identification and analysis of proteins, therefore providing valuable insight into ecosystem functioning. In this methodology, protein sequences are typically identified using a bottom-up approach whereby short subsequences called peptides are matched to experimental mass spectra using a database search. However, it is reported that on average, 75% of the spectra recovered from experiments remain unidentified. De novo peptide identification is an alternative approach to database searching that uses only the spectrum to identify the peptide sequence. This method has undergone significant recent improvements, in part due to the integration of machine learning models into the algorithms. This thesis explores the strengths and weaknesses of many of the current state-ofthe-art de novo peptide identification algorithms through an extensive evaluation. As understanding the underlying data is key to this analysis, a comprehensive survey of the characteristics of tandem mass spectra is included alongside the performance of the algorithms. An alternative machine learning architecture is then proposed to address the weaknesses found. The proposed novel CNN-GNN peptide ion encoding module was able to identify more peptide ions than the encoding modules used by state-of-the-art de novo peptide identification algorithms in all datasets tested. Finally, the utility of artificial data in the context of de novo peptide identification is explored. Artificial spectra were found to be missing critical noise that was present in real data. However, the quantification and introduction of this noise into to artificial spectra increased their similarity to real spectra, significantly improving their potential for use in the training and testing of models. Based on the results of this thesis we recommend specific research avenues for the design and development of the next generation of de novo peptide identification algorithms. This thesis not only demonstrates the challenges facing de novo peptide identification, but also takes the critical first steps toward overcoming them
Adaptive reinforcement learning: From time series anomaly detection to continual learning
No full textThe complexity and scale of IT systems are increasing dramatically, posing many challenges to real-world time series anomaly detection. Many existing methods emphasize feature learning and anomaly scoring but give limited attention to thresholding. Static or manually defined thresholds are commonly used, but they lack adaptability to complex data. Reinforcement learning (RL) is well known for providing optimal control in complex, dynamic systems. This thesis investigates its application for dynamic thresholding control in anomaly detection, and validates the proposed methods in real-world cyber-physical systems (CPS).
First, an agent-based dynamic thresholding (ADT) method is proposed, which models thresholding in time series anomaly detection as a Markov decision process (MDP). ADT employs a deep Q-network and integrates it with an autoencoder-based anomaly scorer to provide dynamic thresholds adaptively. ADT consistently outperforms traditional static and statistical thresholding methods, achieving significantly improved detection performance and robustness even when trained with limited data.
Next, ADT is extended and evaluated in real-world CPS with additional challenges. Experimental results show that ADT significantly outperforms baseline anomaly detection methods, maintaining superior anomaly detection performance, dynamic thresholding capability, data efficiency, and fast training. It shows strong robustness even when the environmental feedback is noisy, partial, or delayed. Moreover, when integrated as a dynamic thresholding controller into existing anomaly detection methods, ADT significantly enhances their detection performance.
Traditional RL methods typically assume a static MDP that rarely holds in practice. This thesis further investigates the continual reinforcement learning (CRL) problem, where the RL agents continually learn and adapt to evolving tasks without catastrophic forgetting. Unlike existing methods that apply prior knowledge mainly through optimization with limited direct guidance on the agent’s behavior, this thesis introduces a novel demonstration-guided continual reinforcement learning (DGCRL) framework, which stores prior knowledge in an external, self-evolving demonstration repository and uses selected demonstrations to directly guide the agent’s exploration and adaptation across tasks, following a dynamic curriculum-based strategy. DGCRL demonstrates superior average performance, knowledge transfer, mitigation of forgetting, and learning efficiency
Multi-objective reinforcement learning and planning for the expected scalarised returns
No full textMany problems in the real world have multiple, often conflicting, objectives. To solve such problems a multi-objective approach to decision making must be taken. In the multi-objective decision making (MODeM) literature, the utility-based approach is followed where a utility function is used to model the preferences over the objectives of a human decision maker (or user). If the utility function is known a priori a single optimal solution can be computed. However, if the utility function is unknown or uncertain, a set of optimal solutions must be computed. When following the utility-based approach, multiple optimality criteria can arise. In scenarios where the utility function of a user is derived from multiple executions of a policy, the scalarised expected returns (SER) must be optimised. In scenarios where the utility of a user is derived from a single execution of a policy, the expected scalarised returns (ESR) criterion must be optimised. In the MODeM literature, the SER criterion has been studied extensively, while the ESR criterion has largely been ignored. In the real world, a user may only have a single opportunity to make a decision. For example, in a medical setting, a patient may only have one chance to select a treatment. Therefore, in order to effectively apply MODeM algorithms to a range of practical applications, the ESR criterion must be further investigated. This thesis contains a number of important contributions. It is demonstrated by example that for ESR settings where the utility function is known and nonlinear, multi-objective methods that compute policies must be explicitly designed for the ESR criterion. For settings where the utility function of a user is unknown, it is shown that expected value vectors are not sufficient to determine optimality under the ESR criterion. Therefore, to determine a partial ordering over policies, new methods to compute sets of optimal policies are proposed. Finally, this thesis proposes a number of new multi-objective algorithms that can compute sets of optimal policies for the ESR criterion in various MODeM settings
Applying machine learning towards automating resource management in cloud computing environments
No full textOver the last number of years there has been high acceleration and widespread adoption of cloud based services by many different industries to improve the scalability and reliability of their services while also reducing costs. The popularity of the cloud has invoked major concern for the high energy consumption and carbon emissions evident in the operation of data centers today. One of the key causes of high energy consumption is inefficient resource management. While much effort in recent years has been devoted to achieving improved energy efficient resource management strategies, performance uncertainty has also become a major hindrance for cloud resource management systems. Performance uncertainties present significant challenges for planning and provisioning Virtual Machine (VM) resource allocation and scheduling in the cloud while having negative impacts overall in achieving greater energy efficiencies and improved Quality of Service (QoS). Unlike traditional resource management strategies that rely on heuristics this work presents novel forecasting based approaches for scheduling and allocating the computational resources of public cloud infrastructure. Our goal is to extend and make advancements upon existing research by applying and comparing the performance of state of the art learning algorithms using real world data. We explore how these advanced approaches can be incorporated to give rise to intelligent resource management systems with the capacity to make improved decisions under uncertainty. In particular, we propose and analyse several interesting variations and extensions to problems such as workflow scheduling, dynamic VM consolidation and also VM migration. The results obtained from this research indicate the immense potential of adopting learning based methodologies to optimise resource usage, improve performance and overall attempt to reach new frontiers in energy efficiency
Knowledge-based multi-objective multi-agent reinforcement learning
No full textMulti-Agent Reinforcement Learning (MARL) is a powerful Machine Learning paradigm, where multiple autonomous agents can learn to improve the performance of a system through experience. The majority of MARL implementations aim to optimise systems with respect to a single objective, despite the fact that many real world problems are inherently multi-objective in nature. Examples of multi-objective problems where MARL may be applied include water resource management, traffic signal control, electricity generator scheduling and robot coordination tasks. Compromises between conflicting objectives may be defined using the concept of Pareto dominance. The Pareto optimal or non-dominated set consists of solutions that are incomparable, where each solution in the set is not dominated by any of the others on every objective. Reward shaping has been proposed as a means to address the credit assignment problem in single-objective MARL, however it has been shown to alter the intended goals of the domain if misused, leading to unintended behaviour. Potential-Based Reward Shaping (PBRS) and difference rewards (D) are commonly used shaping methods for MARL, both of which have been repeatedly shown to improve learning speed and the quality of joint policies learned by agents in single-objective problems. Research into multi-objective MARL is still in its infancy, and very few studies have dealt with the issue of credit assignment in this context. This thesis explores the possibility of using reward shaping to improve agent coordination in multi-objective MARL domains. The implications of using either D or PBRS are evaluated from a theoretical perspective, and the results of several empirical studies support the conclusion that these shaping techniques do not alter the true Pareto optimal solutions in multi-objective MARL domains. Therefore, the benefits of reward shaping can now be leveraged in a broader range of application domains, without the risk of altering the agents\u27 intended goals
Applying machine learning towards automating resource management in cloud computing environments
No full textOver the last number of years there has been high acceleration and widespread adoption of cloud based services by many different industries to improve the scalability and reliability of their services while also reducing costs. The popularity of the cloud has invoked major concern for the high energy consumption and carbon emissions evident in the operation of data centers today. One of the key causes of high energy consumption is inefficient resource management. While much effort in recent years has been devoted to achieving improved energy efficient resource management strategies, performance uncertainty has also become a major hindrance for cloud resource management systems. Performance uncertainties present significant challenges for planning and provisioning Virtual Machine (VM) resource allocation and scheduling in the cloud while having negative impacts overall in achieving greater energy efficiencies and improved Quality of Service (QoS). Unlike traditional resource management strategies that rely on heuristics this work presents novel forecasting based approaches for scheduling and allocating the computational resources of public cloud infrastructure. Our goal is to extend and make advancements upon existing research by applying and comparing the performance of state of the art learning algorithms using real world data. We explore how these advanced approaches can be incorporated to give rise to intelligent resource management systems with the capacity to make improved decisions under uncertainty. In particular, we propose and analyse several interesting variations and extensions to problems such as workflow scheduling, dynamic VM consolidation and also VM migration. The results obtained from this research indicate the immense potential of adopting learning based methodologies to optimise resource usage, improve performance and overall attempt to reach new frontiers in energy efficiency
Exploring the Pareto front of multi-objective COVID-19 mitigation policies using reinforcement learning
No full textInfectious disease outbreaks can have a disruptive impact on public health and societal processes. As decisionmaking in the context of epidemic mitigation is multi -dimensional hence complex, reinforcement learning in combination with complex epidemic models provides a methodology to design refined prevention strategies. Current research focuses on optimizing policies with respect to a single objective, such as the pathogen's attack rate. However, as the mitigation of epidemics involves distinct, and possibly conflicting, criteria (i.a., mortality, morbidity, economic cost, well-being), a multi -objective decision approach is warranted to obtain balanced policies. To enhance future decision -making, we propose a deep multi -objective reinforcement learning approach by building upon a state-of-the-art algorithm called Pareto Conditioned Networks (PCN) to obtain a set of solutions for distinct outcomes of the decision problem. We consider different deconfinement strategies after the first Belgian lockdown within the COVID-19 pandemic and aim to minimize both COVID-19 cases (i.e., infections and hospitalizations) and the societal burden induced by the mitigation measures. As such, we connected a multi -objective Markov decision process with a stochastic compartment model designed to approximate the Belgian COVID-19 waves and explore reactive strategies. As these social mitigation measures are implemented in a continuous action space that modulates the contact matrix of the age -structured epidemic model, we extend PCN to this setting. We evaluate the solution set that PCN returns, and observe that it explored the whole range of possible social restrictions, leading to high -quality trade-offs, as it captured the problem dynamics. In this work, we demonstrate that multi -objective reinforcement learning adds value to epidemiological modeling and provides essential insights to balance mitigation policies.C.F.H. is funded by the University of Galway Hardiman Scholarship, Belgium. This research was supported by funding from the Flemish Government under the ‘‘Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen’’ program. This work also received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant number 101003688 – EpiPose project). P.J.K.L. gratefully acknowledges
support from FWO via postdoctoral fellowship, Belgium 1242021N and the Research council of the Vrije Universiteit Brussel (OZR-VUB via grant number OZR3863BOF). N.H. acknowledges support from the Scientific Chair of Evidence-based Vaccinology under the umbrella of the Methusalem framework at the University of Antwerp. N.H. and A.N. acknowledge funding from the iBOF DESCARTES project (reference: iBOF-21-027). L.W. gratefully acknowledges support from FWO postdoctoral fellowship 1234620N. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. A.N. acknowledges the support by the FWO COVID19 research project G0H0420N. P.L. and L.W. acknowledge support from FWO grant G059423N
- …
