1,721,165 research outputs found
Bayesian networks with applications in reliability analysis
Full text linkA common goal of the papers in this thesis is to propose, formalize and exemplify the use of Bayesian networks as a modelling tool in reliability analysis. The papers span work in which Bayesian networks are merely used as a modelling tool (Paper I), work where models are specially designed to utilize the inference algorithms of Bayesian networks (Paper II and Paper III), and work where the focus has been on extending the applicability of Bayesian networks to very large domains (Paper IV and Paper V).
Paper I is in this respect an application paper, where model building, estimation and inference in a complex time-evolving model is simplified by focusing on the conditional independence statements embedded in the model; it is written with the reliability data analyst in mind. We investigate the mathematical modelling of maintenance and repair of components that can fail due to a variety of failure mechanisms. Our motivation is to build a model, which can be used to unveil aspects of the “quality” of the maintenance performed. This “quality” is measured by two groups of model parameters: The first measures “eagerness”, the maintenance crew’s ability to perform maintenance at the right time to try to stop an evolving failure; the second measures “thoroughness”, the crew’s ability to actually stop the failure development. The model we propose is motivated by the imperfect repair model of Brown and Proschan (1983), but extended to model preventive maintenance as one of several competing risks (David and Moeschberger 1978). The competing risk model we use is based on random signs censoring (Cooke 1996). The explicit maintenance model helps us to avoid problems of identifiability in connection with imperfect repair models previously reported by Whitaker and Samaniego (1989). The main contribution of this paper is a simple yet flexible reliability model for components that are subject to several failure mechanisms, and which are not always given perfect repair. Reliability models that involve repairable systems with non perfect repair, and a variety of failure mechanisms often become very complex, and they may be difficult to build using traditional reliability models. The analysis are typically performed to optimize the maintenance regime, and the complexity problems can, in the worst case, lead to sub-optimal decisions regarding maintenance strategies. Our model is represented by a Bayesian network, and we use the conditional independence relations encoded in the network structure in the calculation scheme employed to generate parameter estimates.
In Paper II we target the problem of fault diagnosis, i.e., to efficiently generate an inspection strategy to detect and repair a complex system. Troubleshooting has long traditions in reliability analysis, see e.g. (Vesely 1970; Zhang and Mei 1987; Xiaozhong and Cooke 1992; Norstrøm et al. 1999). However, traditional troubleshooting systems are built using a very restrictive representation language: One typically assumes that all attempts to inspect or repair components are successful, a repair action is related to one component only, and the user cannot supply any information to the troubleshooting system except for the outcome of repair actions and inspections. A recent trend in fault diagnosis is to use Bayesian networks to represent the troubleshooting domain (Breese and Heckerman 1996; Jensen et al. 2001). This allows a more flexible representation, where we, e.g., can model non-perfect repair actions and questions. Questions are troubleshooting steps that do not aim at repairing the device, but merely are performed to capture information about the failed equipment, and thereby ease the identification and repair of the fault. Breese and Heckerman (1996) and Jensen et al. (2001) focus on fault finding in serial systems. In Paper II we relax this assumption and extend the results to any coherent system (Barlow and Proschan 1975). General troubleshooting is NP-hard (Sochorov´a and Vomlel 2000); we therefore focus on giving an approximate algorithm which generates a “good” troubleshooting strategy, and discuss how to incorporate questions into this strategy. Finally, we utilize certain properties of the domain to propose a fast calculation scheme.
Classification is the task of predicting the class of an instance from as set of attributes describing it, i.e., to apply a mapping from the attribute space to a predefined set of classes. In the context of this thesis one may for instance decide whether a component requires thorough maintenance or not based on its usage pattern and environmental conditions. Classifier learning, which is the theme of Paper III, is to automatically generate such a mapping based on a database of labelled instances. Classifier learning has a rich literature in statistics under the name of supervised pattern recognition, see e.g. (McLachlan 1992; Ripley 1996). Classifier learning can be seen as a model selection process, where the task is to find the model from a class of models with highest classification accuracy. With this perspective it is obvious that the model class we select the classifier from is crucial for classification accuracy. We use the class of Hierarchical Na¨ıve Bayes (HNB) models (Zhang 2002) to generate a classifier from data. HNBs constitute a relatively new model class which extends the modelling flexibility of Näive Bayes (NB) models (Duda and Hart 1973). The NB models is a class of particularly simple classifier models, which has shown to offer very good classification accuracy as measured by the 0/1-loss. However, NB models assume that all attributes are conditionally independent given the class, and this assumption is clearly violated in many real world problems. In such situations overlapping information is counted twice by the classifier. To resolve this problem, finding methods for handling the conditional dependence between the attributes has become a lively research area; these methods are typically grouped into three categories: Feature selection, feature grouping, and correlation modelling. HNB classifiers fall in the last category, as HNB models are made by introducing latent variables to relax the independence statements encoded in an NB model. The main contribution of this paper is a fast algorithm to generate HNB classifiers. We give a set of experimental results which show that the HNB classifiers can significantly improve the classification accuracy of the NB models, and also outperform other often-used classification systems.
In Paper IV and Paper V we work with a framework for modelling large domains. Using small and “easy-to-read” pieces as building blocks to create a complex model is an often applied technique when constructing large Bayesian networks. For instance, Pradhan et al. (1994) introduce the concept of sub-networks which can be viewed and edited separately, and frameworks for modelling object oriented domains have been proposed in, e.g., (Koller and Pfeffer 1997; Bangsø and Wuillemin 2000). In domains that can approx priately be described using an object oriented language (Mahoney and Laskey 1996) we typically find repetitive substructures or substructures that can naturally be ordered in a superclass/subclass hierarchy. For such domains, the expert is usually able to provide information about these properties. The basic building blocks available from domain experts examining such domains are information about random variables that are grouped into substructures with high internal coupling and low external coupling. These substructures naturally correspond to instantiations in an object-oriented BN (OOBN). For instance, an instantiation may correspond to a physical object or it may describe a set of entities that occur at the same instant of time (a dynamic Bayesian network (Kjærulff 1992) is a special case of an OOBN). Moreover, analogously to the grouping of similar substructures into categories, instantiations of the same type are grouped into classes. As an example, several variables describing a specific pump may be said to make up an instantiation. All instantiations describing the same type of pump are said to be instantiations of the same class. OOBNs offer an easy way of defining BNs in such object-oriented domains s.t. the object-oriented properties of the domain are taken advantage of during model building, and also explicitly encoded in the model. Although these object oriented frameworks relieve some of the problems when modelling large domains, it may still prove difficult to elicit the parameters and the structure of the model. In Paper IV and Paper V we work with learning of parameters and specifying the structure in the OOBN definition of Bangsø and Wuillemin (2000).
Paper IV describes a method for parameter learning in OOBNs. The contributions in this paper are three-fold: Firstly, we propose a method for learning parameters in OOBNs based on the EM-algorithm (Dempster et al. 1977), and prove that maintaining the object orientation imposed by the prior model will increase the learning speed in object oriented domains. Secondly, we propose a method to efficiently estimate the probability parameters in domains that are not strictly object oriented. More specifically, we show how Bayesian model averaging (Hoeting et al. 1999) offers well-founded tradeoff between model complexity and model fit in this setting. Finally, we attack the situation where the domain expert is unable to classify an instantiation to a given class or a set of instantiations to classes (Pfeffer (2000) calls this type uncertainty; a case of model uncertainty typical to object oriented domains). We show how our algorithm can be extended to work with OOBNs that are only partly specified.
In Paper V we estimate the OOBN structure. When constructing a Bayesian network, it can be advantageous to employ structural learning algorithms (Cooper and Herskovits 1992; Heckerman et al. 1995) to combine knowledge captured in databases with prior information provided by domain experts. Unfortunately, conventional learning algorithms do not easily incorporate prior information, if this information is too vague to be encoded as properties that are local to families of variables (this is for instance the case for prior information about repetitive structures). The main contribution of Paper V is a method for doing structural learning in object oriented domains. We argue that the method supports a natural approach for expressing and incorporating prior information provided by domain experts and show how this type of prior information can be exploited during structural learning. Our method is built on the Structural EM-algorithm (Friedman 1998), and we prove our algorithm to be asymptotically consistent. Empirical results demonstrate that the proposed learning algorithm is more efficient than conventional learning algorithms in object oriented domains. We also consider structural learning under type uncertainty, and find through a discrete optimization technique a candidate OOBN structure that describes the data well.dr.ing.dr.ing
Bayesian networks with applications in reliability analysis
No full textA common goal of the papers in this thesis is to propose, formalize and exemplify the use of Bayesian networks as a modelling tool in reliability analysis. The papers span work in which Bayesian networks are merely used as a modelling tool (Paper I), work where models are specially designed to utilize the inference algorithms of Bayesian networks (Paper II and Paper III), and work where the focus has been on extending the applicability of Bayesian networks to very large domains (Paper IV and Paper V). <b>Paper I </b>is in this respect an application paper, where model building, estimation and inference in a complex time-evolving model is simplified by focusing on the conditional independence statements embedded in the model; it is written with the reliability data analyst in mind. We investigate the mathematical modelling of maintenance and repair of components that can fail due to a variety of failure mechanisms. Our motivation is to build a model, which can be used to unveil aspects of the “quality” of the maintenance performed. This “quality” is measured by two groups of model parameters: The first measures “eagerness”, the maintenance crew’s ability to perform maintenance at the right time to try to stop an evolving failure; the second measures “thoroughness”, the crew’s ability to actually stop the failure development. The model we propose is motivated by the imperfect repair model of Brown and Proschan (1983), but extended to model preventive maintenance as one of several competing risks (David and Moeschberger 1978). The competing risk model we use is based on random signs censoring (Cooke 1996). The explicit maintenance model helps us to avoid problems of identifiability in connection with imperfect repair models previously reported by Whitaker and Samaniego (1989). The main contribution of this paper is a simple yet flexible reliability model for components that are subject to several failure mechanisms, and which are not always given perfect repair. Reliability models that involve repairable systems with non perfect repair, and a variety of failure mechanisms often become very complex, and they may be difficult to build using traditional reliability models. The analysis are typically performed to optimize the maintenance regime, and the complexity problems can, in the worst case, lead to sub-optimal decisions regarding maintenance strategies. Our model is represented by a Bayesian network, and we use the conditional independence relations encoded in the network structure in the calculation scheme employed to generate parameter estimates. In <b>Paper II </b>we target the problem of fault diagnosis, i.e., to efficiently generate an inspection strategy to detect and repair a complex system. Troubleshooting has long traditions in reliability analysis, see e.g. (Vesely 1970; Zhang and Mei 1987; Xiaozhong and Cooke 1992; Norstrøm et al. 1999). However, traditional troubleshooting systems are built using a very restrictive representation language: One typically assumes that all attempts to inspect or repair components are successful, a repair action is related to one component only, and the user cannot supply any information to the troubleshooting system except for the outcome of repair actions and inspections. A recent trend in fault diagnosis is to use Bayesian networks to represent the troubleshooting domain (Breese and Heckerman 1996; Jensen et al. 2001). This allows a more flexible representation, where we, e.g., can model non-perfect repair actions and questions. Questions are troubleshooting steps that do not aim at repairing the device, but merely are performed to capture information about the failed equipment, and thereby ease the identification and repair of the fault. Breese and Heckerman (1996) and Jensen et al. (2001) focus on fault finding in serial systems. In Paper II we relax this assumption and extend the results to any coherent system (Barlow and Proschan 1975). General troubleshooting is NP-hard (Sochorov´a and Vomlel 2000); we therefore focus on giving an approximate algorithm which generates a “good” troubleshooting strategy, and discuss how to incorporate questions into this strategy. Finally, we utilize certain properties of the domain to propose a fast calculation scheme. Classification is the task of predicting the class of an instance from as set of attributes describing it, i.e., to apply a mapping from the attribute space to a predefined set of classes. In the context of this thesis one may for instance decide whether a component requires thorough maintenance or not based on its usage pattern and environmental conditions. Classifier learning, which is the theme of<b> Paper III</b>, is to automatically generate such a mapping based on a database of labelled instances. Classifier learning has a rich literature in statistics under the name of supervised pattern recognition, see e.g. (McLachlan 1992; Ripley 1996). Classifier learning can be seen as a model selection process, where the task is to find the model from a class of models with highest classification accuracy. With this perspective it is obvious that the model class we select the classifier from is crucial for classification accuracy. We use the class of Hierarchical Na¨ıve Bayes (HNB) models (Zhang 2002) to generate a classifier from data. HNBs constitute a relatively new model class which extends the modelling flexibility of Näive Bayes (NB) models (Duda and Hart 1973). The NB models is a class of particularly simple classifier models, which has shown to offer very good classification accuracy as measured by the 0/1-loss. However, NB models assume that all attributes are conditionally independent given the class, and this assumption is clearly violated in many real world problems. In such situations overlapping information is counted twice by the classifier. To resolve this problem, finding methods for handling the conditional dependence between the attributes has become a lively research area; these methods are typically grouped into three categories: Feature selection, feature grouping, and correlation modelling. HNB classifiers fall in the last category, as HNB models are made by introducing latent variables to relax the independence statements encoded in an NB model. The main contribution of this paper is a fast algorithm to generate HNB classifiers. We give a set of experimental results which show that the HNB classifiers can significantly improve the classification accuracy of the NB models, and also outperform other often-used classification systems. In<b> Paper IV </b>and<b> Paper V </b>we work with a framework for modelling large domains. Using small and “easy-to-read” pieces as building blocks to create a complex model is an often applied technique when constructing large Bayesian networks. For instance, Pradhan et al. (1994) introduce the concept of sub-networks which can be viewed and edited separately, and frameworks for modelling object oriented domains have been proposed in, e.g., (Koller and Pfeffer 1997; Bangsø and Wuillemin 2000). In domains that can approx priately be described using an object oriented language (Mahoney and Laskey 1996) we typically find repetitive substructures or substructures that can naturally be ordered in a superclass/subclass hierarchy. For such domains, the expert is usually able to provide information about these properties. The basic building blocks available from domain experts examining such domains are information about random variables that are grouped into substructures with high internal coupling and low external coupling. These substructures naturally correspond to instantiations in an object-oriented BN (OOBN). For instance, an instantiation may correspond to a physical object or it may describe a set of entities that occur at the same instant of time (a dynamic Bayesian network (Kjærulff 1992) is a special case of an OOBN). Moreover, analogously to the grouping of similar substructures into categories, instantiations of the same type are grouped into classes. As an example, several variables describing a specific pump may be said to make up an instantiation. All instantiations describing the same type of pump are said to be instantiations of the same class. OOBNs offer an easy way of defining BNs in such object-oriented domains s.t. the object-oriented properties of the domain are taken advantage of during model building, and also explicitly encoded in the model. Although these object oriented frameworks relieve some of the problems when modelling large domains, it may still prove difficult to elicit the parameters and the structure of the model. In Paper IV and Paper V we work with learning of parameters and specifying the structure in the OOBN definition of Bangsø and Wuillemin (2000). Paper IV describes a method for parameter learning in OOBNs. The contributions in this paper are three-fold: Firstly, we propose a method for learning parameters in OOBNs based on the EM-algorithm (Dempster et al. 1977), and prove that maintaining the object orientation imposed by the prior model will increase the learning speed in object oriented domains. Secondly, we propose a method to efficiently estimate the probability parameters in domains that are not strictly object oriented. More specifically, we show how Bayesian model averaging (Hoeting et al. 1999) offers well-founded tradeoff between model complexity and model fit in this setting. Finally, we attack the situation where the domain expert is unable to classify an instantiation to a given class or a set of instantiations to classes (Pfeffer (2000) calls this type uncertainty; a case of model uncertainty typical to object oriented domains). We show how our algorithm can be extended to work with OOBNs that are only partly specified. In Paper V we estimate the OOBN structure. When constructing a Bayesian network, it can be advantageous to employ structural learning algorithms (Cooper and Herskovits 1992; Heckerman et al. 1995) to combine knowledge captured in databases with prior information provided by domain experts. Unfortunately, conventional learning algorithms do not easily incorporate prior information, if this information is too vague to be encoded as properties that are local to families of variables (this is for instance the case for prior information about repetitive structures). The main contribution of Paper V is a method for doing structural learning in object oriented domains. We argue that the method supports a natural approach for expressing and incorporating prior information provided by domain experts and show how this type of prior information can be exploited during structural learning. Our method is built on the Structural EM-algorithm (Friedman 1998), and we prove our algorithm to be asymptotically consistent. Empirical results demonstrate that the proposed learning algorithm is more efficient than conventional learning algorithms in object oriented domains. We also consider structural learning under type uncertainty, and find through a discrete optimization technique a candidate OOBN structure that describes the data well
Explainable Reinforcement Learning (XRL): Simplifying Agent Behavior
Full text linkReinforcement learning (RL) is a framework for learning intelligent agents in sequential decision-making. The agents in RL have shown impressive results in many domains, especially in games like Go, chess, Atari, and poker, where the environments are stationary and can be simulated. These achievements are possible because of neural networks, which perform well in many machine learning tasks due to their flexible structure supported by the universal approximation theorem and subsequent work. Additionally, structures like convolution and self-attention positively affect the performance gain. However, understanding how neural networks achieve these results is notoriously difficult. We understand how they work computationally but struggle to distill the computational details into high-level insights. For example, is a group of neurons looking for a humanrecognizable concept, do neurons work together to find concepts, and are neurons specialized to do one thing or more? Without a better understanding, humans cannot extract scientific knowledge, trust that these agents will function as expected in unseen situations, and easily debug them.
New research looking at explainability in RL to solve these challenges is known as explainable reinforcement learning (XRL). The XRL research field started gaining traction when the Defense Advanced Research Projects Agency launched its explainable artificial intelligence program in 2017. Early XRL works focused on leveraging explainability methods from supervised learning. They did not consider RL-specific challenges, such as delayed rewards, stochastic environments, and large state spaces. While these XRL methods were developed, numerous surveys on XRL were published concurrently but they missed much of the new XRL research.
Focusing on these two issues, we conducted a systematic literature review of the XRL research field. Based on our findings on what exists and the current challenges, we developed three new XRL methods. The first one focused on state abstraction, where we reduced the state space complexity by merging states with similar feature importance. The second study focused on agent distillation, in which we tried to simplify the agent by making its representation less complex. In the third study, we proposed outcome-based semifactual explanations, where we looked at the necessity of action switching to simplify the behavior. The thesis provides an overview of the research field and new methods for understanding RL agents. From the developed methods, we learned that the behavior of RL agents can often be simplified without affecting their performance negatively. Future XRL research should move towards developing agents that are interpretable out of the box. With fully interpretable agents, we can more easily extract insights and trust that they will function as intended in unseen situations
Using similarity learning to enable decision support in aquaculture
Full text linkAquaculture (AQ) is an industry that cultivates food in water. This includes many types of seafood such as salmon, trout, and whitefish, as well as shellfish and algae. Farms for seafood production are typically described as sites by the industry. In Norway, the site locations are normally regulated and allocated by the government. Artificial intelligence (AI) and machine learning (ML) has not yet been widely adopted in the industry. AI/ML would potentially be able to support the industry in automation, operation and decision support.
The aquaculture industry is expanding across the globe. This is a result of technological development and the need for more food production to feed a growing population. In 2012, the Norwegian seafood industry was expected to grow five-fold from 2007 until 2050 [1]. According to industry representatives and the government1,2, this is still the case today. As a result of this expansion, the industry needs to increase the number of production sites. While expanding, the industry needs to keep the environmental impact of such production sites to a minimum. As production sites pollute their immediate surroundings, they should ideally not be in constant production over a long period of time. Additionally, the production sites cannot be too concentrated geographically to minimize the environmental impact and risk of spreading diseases such as sea lice.
As a result, the number of available sites is decreasing, and the industry now looks to increasingly more exposed locations for their aquaculture operations. Exposed aquaculture sites are subject to rough conditions and are often inaccessible. Typical aquaculture sites are well sheltered. To ensure the same level of safety, aquaculture sites that are more exposed would require more resources and a more robust physical infrastructure. Also, the level of exposure often leads to more downtime, where personnel is waiting for the weather to clear up to perform their tasks.
The aquaculture industry is a conservative industry and has not progressed far in terms of digitalization and instrumentation compared to many other comparable industries such as oil and gas. The push towards more exposed aquaculture operations is now changing this, where increasing the level of automation and remote work would significantly contribute to decreasing the risks to personnel. Such automated operations require the application of digital technologies both for operations and decision-support. This development is supported by the availability of more operational data from the aquaculture industry in recent years. As a result, the connectivity and data availability allows for data-driven services and utilization of ML.
Data-driven models and ML support in the aquaculture industry include both operational use cases and decision support systems (DSSs). Operational use cases for aquaculture include 1) computer vision for situation recognition needed for automatic fish feeding, and 2) robotics that can perform necessary operations such as cage cleaning or extracting fish. As such, operational use cases are use cases where ML models are used in real-time or close to real-time. In contrast, DSSs are typically used as a planning tool. DSSs use data-driven models in the context of supporting decision-making or operational planning. Such systems are designed to help operators by predicting operational properties, such as production, structure movements, or waves.
Most decision-makers, especially from conservative industries, prefer an understandable and explainable DSS. When the DSS explains the recommendation it produces, it increases the trust in that recommendation, and as a result, the usefulness of the DSS. Many machine learning methods and their resulting models are not easy to explain to most users. One way of alleviating this is to use case-based reasoning (CBR)[2]. CBR captures previous experiences or situations in the form of cases that consist of a problem description and the corresponding solution. As part of a DSS, CBR would store previous situations where the DSS was used and the resulting action or solution. In this way, the DSS user can be presented with the previous situation most similar to the current situation and the resulting action for that situation. The input of a DSS can be the current state. In the case of using CBR for planning in a DSS, the CBR input can be a prediction (e.g., a predicted situation for which the CBR can retrieve a solution). Presenting an actual recorded situational experience and resulting action along with the prediction provides an indirect explanation and strengthens the user’s confidence in the DSS.
The work described in this thesis investigates the use of machine learning to increase the level of automation in aquaculture operations, focusing on decision support. A general framework for designing a DSS is introduced, from data gathering to the user interface. This framework outlines the steps from sensors readings, preprocessing of the data, combining the data with knowledge and experience from the users of the DSS, using the data to feed machine learning, knowledge models, and numerical models to then predict a future state which can be used to make informed decisions. In addition, a CBR-based DSS can store previously recorded situations where the DSS was applied (cases). The DSS can then use this repository to retrieve and present the user with the previously recorded cases that are most similar to the predicted state. To do this, the DSS must retrieve the case most relevant (similar) to the one predicted by the DSS or input by the DSS user (query case). Retrieving the most similar case requires the DSS to compute the similarity between the query case and the cases in the repository.
Measuring similarity between cases is a focus of research within machine learning and case-based reasoning. Manual modeling this similarity can be challenging. Building on previous state-of-the-art machine learning methods, we propose a new method for learning such similarity measures from data (similarity learning), which can be used for retrieving cases: Extended Siamese Neural Networks (ESNN). ESNN is a similarity learning (SL) method that outperforms the accuracy and training speed of state-of-the-art methods across domains. Extending the testing of ESNN, we developed a dataset for describing situations in aquaculture operations. We demonstrated that ESNN also outperformed state-of-the-art methods for retrieving the most similar operational situations
Hydropower optimization using model-based Reinforcement Learning
Full text linkEn av de mest effektive måtene å lagre energi på er ved å planlegge produksjonen av vannkraft slik at kraft produseres når det er mest gunstig og at vannet i andre perioder reserveres. I det nordiske kraftmarkedet prøver vannkraftoperatører å optimalisere profitten.
Dette er et utfordrende problem som innebærer ikke-lineær dynamikk, usikkerhet og driftsbegrensninger. Tradisjonelle metoder bruker vanligvis varianter av lineær programmering for å løse problemet, men disse metodene kan ha lang kjøretid. Det nordiske kraftmarkedet er i ferd med å bli gå over til 15 minutters tidsoppløsning, noe som øker beregningsbyrden for tradisjonelle metoder. I denne avhandlingen prøver vi å finne alternative metoder med en annen avveining mellom løsningskvalitet og kjøretid. Kjøretiden må være lavere, og ideelt sett bør ikke løsningskvaliteten påvirkes negativt.
Vi observerer at modellen vår for profitten til kraftverket er differensierbar med noen ikke-kontinuerlige punkter, noe som gjør det mulig å beregne gradienten av profitten med hensyn til vannføringen i kraftverket. Vi kan deretter bruke gradient ascent for å optimalisere vannføringen i en episode. For å håndtere de ikke-kontinuerlige punktene bruker vi først ulike teknikker som Monte Carlo tree search og cross-entropy method i et forsøk på å komme nær det globale maksimumet.
Vi finner at denne metoden har en kjøretid innenfor de gitte tidsbegrensningene og bedre løsningskvalitet enn noen state-of-the-art metoder. Imidlertid var vi ikke i stand til å sammenligne ytelsen med produksjonssystemer eller det globale optimumet. Følgelig er det nøyaktige omfanget av løsningens kvalitet ukjent.One of the most effective ways to store energy is by scheduling hydropower plants to produce when it is most beneficial and reserve the water in other periods. In the Nordic power market, hydropower operators try to optimize profits.
This is a challenging problem involving non-linear dynamics, uncertainty and operating constraints. Traditional methods typically use variants of linear programming to solve the problem, but these methods can have high running times. The Nordic power market is transitioning to become closer to real-time, which increases the computational burden on traditional methods. In this thesis, we try to find alternative methods with a different trade-off between solution quality and running time. The running time needs to be lower, and ideally, the solution quality should not be adversely affected.
We make the observation that our model of the environment is differentiable with some non-continuous points, which allows us to compute the gradient of the profit with regard to the actions. We can then do gradient ascent to optimize actions for an episode. To deal with the non-continuous points we first use various techniques such as Monte Carlo tree search and the cross-entropy method in an attempt to get close to the global maximum.
We find that this method has a running time within the given time constraint and a better solution quality than some state-of-the-art methods. However, we were unable to compare the performance to production systems or the global optimum. Consequently, the exact extent of the solution's quality remains unknown
Profiting from Football Betting using Artificial Neural Networks
Full text linkMarkedet for fotballtipping er enormt, og mange tippeselskaper lar håpefulle spillere
plassere veddemål omtrent alle aspekter av en kamp. Det mest populære veddemålet er å spille på kampens sluttresultat.
Å forutsi disse utfallene er en vanskelig oppgave på grunn av spillets kompleksitet, men ved å konsekvent produsere bedre forutsigelser enn tippeselskapene kan muligheten for profitt vise seg.
I denne oppgaven blir to kamppredikeringsmodeller basert på nevrale nettverk utviklet, hvor en er en fullstendig datadreven tilpasning av en vellykket Bayesiansk nettverks-modell avhengig av kunnskapen til en domeneekspert. Effekten av å utnytte domenekunnskap i strukturen av nevrale nettverk er utforsket ved å gruppere relaterte inputvariable i separate sub-nettverk.
Lønnsomheten til modellene er evaluert over to sesonger av den engelske Premier League med ulike spillstrategier. Eksperimenter med å bruke metoder fra
reinforcement learning for å trene en spillstrategi-agent blir utført for å utforske muligheten til å bygge et komplett ende-til-ende spillsystem.
Enkelte modell-strategi-kombinasjoner klarte å generere profitt over begge testsesongene, som viser at det er mulig å profittere fra fotballtipping med kunstige nevrale nettverkThe football betting market is an enormous market, with numerous bookmakers
allowing hopeful punters to place bets on almost every aspect of a match. The most popular aspect of football betting is placing bets on the final match outcome.
Predicting these outcomes is a difficult task given the complexity of the game, but by consistently producing better predictions than the bookmaker, an opportunity to make a profit on the market may arise.
In this thesis, two match prediction models using artificial neural networks are developed, where one is a fully data-driven adaptation of a successful Bayesian network model utilizing domain expert knowledge. The effect of utilizing domain knowledge in the structure of neural networks is explored by grouping related input
features into separate sub-networks.
The profitability of the models is evaluated over two seasons of the English Premier League using different money management strategies. Experiments in using reinforcement learning methods to train a money management agent are performed to explore the possibility of building a complete end-to-end betting
system.
Some model-strategy combinations were able to generate a profit over both
test seasons, showing that it is indeed possible to profit from football betting
using artificial neural networks
Recognizing Text Signatures Using Neural Machine Translation
Full text linkOptical character recognition (OCR) is a technology used to convert scanned text into searchable data. OCR systems have achieved up to 99% recognition rates when working with clean and well-formatted documents under optimal conditions. However, the results are less promising under suboptimal conditions, for example when faced with damaged or obfuscated text.
We propose a new method for recognizing words that are obfuscated in a particular way. This recognition is accomplished by utilizing their signature, a small portion of the original text. Our approach to this problem is to consider it as a translation problem, and we attempt to solve it by using state-of-the-art methods in the field of machine translation. Three models were developed as a result of the research conducted in this thesis. Two of these were based on the encoder-decoder framework for sequence-to-sequence prediction. The best performing model had an accuracy of over 98% when recognizing text written in a single font and close to 90% when recognizing text written in five different fonts under 10% noise
Factorization models with relational and contextual information
No full textThe increasing availability of interconnected multi-modal information sources motivates the development of novel probabilistic models for recommender system that can leverage context present in relational data. Thus, we seek to integrate contextual information that can be relevant for determining the users’ information needs. In this thesis we focus on a set of techniques for modeling contextual information to factorization models, in particular models that uses implicit feedback such as event counts. Furthermore we propose analytical tools for those models, improving our capabilities with regards to find suitable hyparparameters. In order to model counts (for example, number of clicks in a page) as implicit user feedback, we chose to utilize the Poisson factorization as a building block. Then, we develop two Poisson factorization models that include social networks, item textual content and periodic time events as contextual information, incorporated into a joint matrix and tensor factorization model (in Chapters 3 and 4). Additionally, we develop a joint hierarchical recurrent neural networks and a temporal point process model for the problem of multi-session recommendations, where we observe sequences of items grouped into sequences of sessions, and create a model capable of providing itens recommendation and next-session time prediction (Chapter 5). Finally, we utilize and develop an approach based on the prior predictive distribution that allows us to set hyperparameters for Poisson factorization models without the need to fit the model to the data, obtaining both closed-form equations and an optimization algorithm for this task (Chapter 6). One relevant result here is a closed-form equation for the dimensionality of the latent space in Poisson factorization models. In general, we position this work as a contribution to probabilistic modeling in the context of recommender system utilizing multi-relational and count data as a signal for contextual information, with contributions ranging from model design, analysis and hyperparameter selection
Traffic sign anomaly detection with unsupervised learning
No full textNåværende har Statens Vegvesen (SVV) og andre veg-etater ansatt et stort antall utlærte individer for å gjøre manuell befaring av veisystemet. Denne oppgaven utforsker bruken av uovervåket avviks-deteksjon for å redusere den manuelle arbeidsmengden dette fører til.
Data i den virkelige verden inneholder ofte støy, har feil merking, eller ingen merking i det hele tatt, som fører til mange utfordringer. Denne oppgaven takler dette problemet ved å utvikle et uovervåket pipeline som oppdager trafikkskilt, bruker de til trening, og finner avvik i disse skiltene. Dette i håp om å hjelpe utviklingen av et maskinlæring system som i framtiden kan brukes av veg-etater i Norge for å finne avvik i forskjellige veg-relaterte objekter. Modellen presentert i denne oppgaven oppnår en ROC-AUC verdi på 0.92.
Resultatet viser at å utvikle et avviks deteksjonssystem til bruk av veg-etater for å redusere manuell arbeidsbruk er mulig med høy nøyaktighet. Resultatet viser også at dette er mulig med kun umerket, ekte data, med lite menneskelig innblanding
Predicting E-commerce Consumer Behaviour Using Sparse Session Data
Full text linkThis thesis research consumer behavior in an e-commerce domain by using a data set of sparse session data collected from an anonymous European e-commerce site. The goal is to predict whether a consumer session results in a purchase, and if so, which items are purchased. The data is supplied by the ACM Recommender System Challenge, which is a yearly challenge held by the ACM Recommender System Conference.
Classification is used for predicting whether or not a session made a purchase, as well as what items it bought. Several characteristics of the data are analysed in order to discover what separates a buy-session from the rest. In addition the interactions with items will be analysed to see what items a given buy-session is likely to purchase. The data is on a rather general format containing only a session ID, an ID of the item interacted with, a timestamp, and a category of the object - meaning the analysis can be applicable to other e-commerce sites and domains. Observations from the analysis are used for extracting features and to provide other valuable information for the classification. The following algorithms for classification are evaluated: Random Forest, Logistic Regression, Decision Tree, Bayesian Network and Naive Bayes.
It is shown that one can predict a session's behaviour by using classification. Which items the session interacted with and when the interaction occurred proved to be important factors. The findings may contribute towards improving implicit ratings in recommender systems, or provide useful information for recommender systems when only session data is available
- …
