1,721,092 research outputs found
Recommended from our members
Towards Data Efficiency on Model-Based Reinforcement Learning: Model Confidence and Representation
Get PDFHumans can develop their internal model of the external world and use it for decision making. Reinforcement Learning (RL) is an optimization method to maximize the expected total reward on sequential decision-making problems. RL is divided into two approaches: a model-free approach directly learns optimal behaviors given the data, whereas a model-based approach builds the model of the environment and utilizes it for decision making. Although the Model-based approach is intuitive and appealing, it has several challenges to overcome, such as the model's inaccuracy or determining the effective model architecture. These challenges limit practical applications of the model-based RL. In this thesis, we first discuss how to integrate the model uncertainty into model-based RL and propose methods to use them. We apply the Monte Carlo dropout technique to the state transition model to estimate uncertainty. Our approach enables the algorithm to use model simulations effectively by filtering the simulation given the model uncertainty. We show that this scheme achieves speed-up of agents' policy learning in contrast to conventional ways to use model simulations without considering the uncertainty. In model-based RL, model architecture is another critical factor to consider. In this context, we then investigate variants of the Variational Autoencoder (VAE) and Generative Adversarial Networks (GANs), and then evaluate the combination of them, VAE/GAN, as the agents' state representation learning (SRL) methods. Acquiring a compact and efficient representation of the world for control is essential to help model-based RL agents overcome the curse of dimensionality. We evaluate the VAE/GAN architecture qualitatively and quantitatively, and show that the RL agent that learns a policy over the VAE/GAN embedding outperforms the one with the VAE embedding. We further discuss VAE/GAN and disentanglement. Taken together, the presented method and models provide the RL agent architecture to achieve better sample efficiency
Recommended from our members
An Analysis of Deceptive Text Using Techniques of Machine Learning, Corpus Generation, and Online Crowdsourcing
Get PDFThis research demonstrates how to use deep learning techniques alongside corpus generation and online crowdsourcing in order to better understand deceptive text. In this dissertation, I use state-of-the-art classifiers to examine the structure of deceptive text and determine what parts mark it as deceptive. I also expand knowledge of deception into new areas by adding to the knowledge base of deceptive text with large amounts of curated, realistic data. It also offers a more complete understanding by examining deceptive text through multiple lenses. My research accomplishes this through three interrelated projects: (I) The construction of a new state-of-the-art classifier, and modifying the input to the classifier to examine what the classifier considers most informative in a classification, (II) the creation of a new corpus of deceptive text, the Motivated Deception Corpus, which uses gameifying techniques to improve the quality of deceptive text samples by making them more realistic through competition, and (III) a human subject study on Amazon Mechanical Turk, where I observe what samples humans consider deceptive or truthful and use a Cultural Consensus Theory model to identify what prompts a subject to decide one way or the other
Electrochemical charge transport in organic neuromorphic device networks
No full textNeuromorphic computing aims to implement neural networks in brain-inspired hardware to enable faster computation, lower power consumption, and higher degrees of biological realism in the neural models. Artificial synapses represent unique devices that simultaneously enable information storage as a conductance state and highly parallel, analog information processing. Organic artificial synapses made from organic mixed ionic electronic conductors take another step toward biological realism. They are biocompatible and can communicate through the same electrolytes as living neurons while being ultra-low powered and linearly programmable. However, otherwise promising devices based on the conductive polymer PEDOT:PSS show write non-idealities and self-discharge so that programmed states decay over time. Both phenomena must be understood and compensated for to successfully implement even small neural models. This thesis employs high-resolution charge transport models to explain the electrochemical processes in organic synapses and identify the non-ideal behavior's root causes. To find solutions to compensate for the non-idealities, it applies algorithm-hardware co-design with full network simulations of artificial and spiking neural networks. While an optimized write scheme can improve programming reproducibility, self-discharge is shown to be caused by impurities in the device that are difficult to exclude. Simulations of artificial neural networks show that self-discharge can significantly degrade network performance over single-digit minutes. Special network design and newly proposed reminder pulses are developed to compensate for self-discharge effectively. They do, however, introduce additional complexity. On the other hand, always-on learning in spiking neural networks proves to be a much more effective way to compensate for self-discharge natively and unlock synergies with the device properties that result in higher accuracies during learning. In summary, the thesis demonstrates that algorithm-hardware co-design is a powerful method to understand and compensate for device limitations, and it highlights the synergies between always-on-spike-based learning and forgetful devices
Learning rules for CMOS-memristive co-integrated neuromorphic hardware
No full textToday, applications of artificial intelligence based on machine learning algorithms pervade various aspects of human society. The training and inference of the large artificial neural networks (ANN) is executed in server farms on conventional von-Neumann computers. This computer architecture renders unfit for the highly parallelizable workloads of machine learning and neuromorphic applications, leading to the consumption of vast amounts of energy during training and inference. Furthermore, the data evaluation far away from the actual user that could be consumer smartphone or a remote IoT application raises concerns in sectors like medical, industrial, and military applications where data privacy is crucial. Potentially a solution to both issues, innovative computer architectures and novel electronic device concepts hold the promise of local and energy-efficient training and application of artificial intelligence algorithms. Prominent among these concepts, In-Memory-Compute (IMC) computer architectures based on memristive switching devices can overcome the limitations of the von-Neumann architecture by co-localizing data storage and computation in an energy-efficient manner while saving silicon real estate. Memristive devices are an emerging non-volatile memory consisting of nanoscale resistors with an electronically adjustable resistance. However, to fully leverage the application of memristive devices in IMC architectures for machine learning and neuromorphic applications, it is necessary to develop dedicated learning rules for these hybrid systems that mitigate potential shortfalls of the memristive switching dynamics or even exploit these for the implementation of efficient algorithms. Therefore, in this thesis on the one hand, different methods for the compensation of the effect of memristive switching dynamics on the training of ANNs are investigated. An approach is developed that allows for the same training performance on a classical benchmark task as optimal floating-point weights. On the other hand, the integration of memristive devices with conventional CMOS-circuitry is investigated for neuromorphic applications. Therefore, the famous Hierarchical Temporal Memory algorithm is adapted for an application on a memristive crossbar array. This approach conserves the crucial context-sensitivity of this algorithm in a sequence learning application and renders energy-efficient in comparison to conventional storage technology. Lastly, the algorithm is validated on real hardware on a hybrid memristive CMOS-chip that is developed in the course of this thesis.\parIn this thesis a crucial issue for the integration of memristive devices into machine learning accelerators is tackled: the loss of training performance due to maleficent memristive switching dynamics. It is furthermore showcased how algorithm-hardware co-design can adapt neuromorphic algorithms for the use with energy-efficient CMOS-memristive hardware
Spiking Neural Network Learning, Benchmarking, Programming and Executing
No full textThis eBook is a collection of articles from a Frontiers Research Topic. Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office: frontiersin.org/about/contac
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Pre-training and Meta-learning for Memristor Crossbar Arrays
No full textMemristive crossbar arrays show promise as non-von Neumann computing technologies, bringing sophisticated neural network processing to the edge and facilitating real-world online learning. However, their deployment for real-world learning problems faces challenges such as non-linearities in conductance updates, variations during operation, fabrication mismatch, conductance drift, and the realities of gradient descent training.This talk will present methods to pre-train neural networks to be largely insensitive to these non-idealities during learning tasks. These methods rely on a phenomenological model of the device, obtainable experimentally, and bi-level optimization. We showcase this effect through meta-learning and a differentiable model of conductance updates on few-shot learning tasks. Since pre-training is a necessary procedure for any online learning scenario at the edge, our results may pave the way for real-world applications of memristive devices without significant adaptation overhead.Furthermore, by considering the programming of memristive devices as a learning problem in its own right, we demonstrate that the developed methods can accelerate existing write-verify techniques
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Get PDFWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
- …
