5,172 research outputs found
Teacher-apprentices RL (TARL): leveraging complex policy distribution through generative adversarial hypernetwork in reinforcement learning
No full textTypically, a Reinforcement Learning (RL) algorithm focuses in learning a single deployable policy as the end product. Depending on the initialization methods and seed randomization, learning a single policy could possibly leads to convergence to different local optima across different runs, especially when the algorithm is sensitive to hyper-parameter tuning. Motivated by the capability of Generative Adversarial Networks (GANs) in learning complex data manifold, the adversarial training procedure could be utilized to learn a population of good-performing policies instead. We extend the teacher-student methodology observed in the Knowledge Distillation field in typical deep neural network prediction tasks to RL paradigm. Instead of learning a single compressed student network, an adversarially-trained generative model (hypernetwork) is learned to output network weights of a population of good-performing policy networks, representing a school of apprentices. Our proposed framework, named Teacher-Apprentices RL (TARL), is modular and could be used in conjunction with many existing RL algorithms. We illustrate the performance gain and improved robustness by combining TARL with various types of RL algorithms, including direct policy search Cross-Entropy Method, Q-learning, Actor-Critic, and policy gradient-based methods.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Interactive Intelligenc
BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs
No full textWhile reinforcement learning (RL) has made great advances in scalability, exploration and partial observability are still active research topics. In contrast, Bayesian RL (BRL) provides a principled answer to both state estimation and the exploration-exploitation trade-off, but struggles to scale. To tackle this challenge, BRL frameworks with various prior assumptions have been proposed, with varied success. This work presents a representation-agnostic formulation of BRL under partially observability, unifying the previous models under one theoretical umbrella. To demonstrate its practical significance we also propose a novel derivation, Bayes-Adaptive Deep Dropout rl (BADDr), based on dropout networks. Under this parameterization, in contrast to previous work, the belief over the state and dynamics is a more scalable inference problem. We choose actions through Monte-Carlo tree search and empirically show that our method is competitive with state-of-the-art BRL methods on small domains while being able to solve much larger ones.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Interactive Intelligenc
Refined Risk Management in Safe Reinforcement Learning with a Distributional Safety Critic
No full textSafety is critical to broadening the real-world use of reinforcement learning (RL). Modeling the safety aspects using a safety-cost signal separate from the reward is becoming standard practice, since it avoids the problem of finding a good balance between safety and performance. However, the total safety-cost distribution of different trajectories is still largely unexplored. In this paper, we propose an actor critic method for safe RL that uses an implicit quantile network to approximate the distribution of accumulated safety-costs. Using an accurate estimate of the distribution of accumulated safetycosts, in particular of the upper tail of the distribution, greatly improves the performance of riskaverse RL agents. The empirical analysis shows that our method achieves good risk control in complex safety-constrained environments.AlgorithmicsIntelligent Electrical Power Grid
qgym: A Gym for Training and Benchmarking RL-Based Quantum Compilation
No full textCompiling a quantum circuit for specific quantum hardware is a challenging task. Moreover, current quantum computers have severe hardware limitations. To make the most use of the limited resources, the compilation process should be optimized. To improve currents methods, Reinforcement Learning (RL), a technique in which an agent interacts with an environment to learn complex policies to attain a specific goal, can be used. In this work, we present qgym, a software framework derived from the OpenAI gym, together with environments that are specifically tailored towards quantum compilation. The goal of qgym is to connect the research fields of Artificial Intelligence (AI) with quantum compilation by abstracting parts of the process that are irrelevant to either domain. It can be used to train and benchmark RL agents and algorithms in highly customizable environments.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Quantum Circuit Architectures and Technolog
Influence-Augmented Local Simulators: a Scalable Solution for Fast Deep RL in Large Networked Systems
No full textLearning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.Interactive IntelligenceAlgorithmic
VERIFIKASI LAPORAN RL 3.1 : KEGIATAN PELAYANAN RAWAT INAP DI BANGSAL PRINGGONDANI II RSJ PROF. DR. SOEROJO MAGELANG
No full textBackground: Based on the Regulation of the Minister of Health of the Republic
of Indonesia Number 1171/MENKES/PER/VI/2011 assigned byDirector General
of Health Efforts, each hospital shall implement the Hospital Information System
(SIRS), one of them is RL 3.1. in Prof. RSJ. Dr. Soerojo Magelang, RL 3.1 is
based on the data in the daily inpatient census that should be grouped based on
patient's illness. Due to the absence of the grouping, it is an obstacle in making
RL 3.1 by reporting officer.
Objective: This study aimed to overview the implementation of the inpatient daily
census and RL 3.1, 3.1 RL verify the inpatient daily census and the verification of
benefits, and not knowing the cause grouped daily inpatient census as a data
source RL 3.1.
Methods: The study was a descriptive study with a qualitative approach to the
design of a cross-sectional study. Techniques of data collection by interview,
observation, and documentation.
Results: In making RL 3.1, inpatient daily census by nurses not stratified by
case, all patients are grouped into a common disease but Pringgondani II Ward is
Ward Care for the disease and nerve. Verification of the results of the previous
RL 3.1 patients grouped into general categories, obtained in a single year in
Ward II as many as 998 patients Pringgondani Internal Medicine, Pediatrics 163
patients, 12 patients Surgery, Neurology 199 patients, and 52 patients Case
General. Verification of the results obtained the highest incidence rate Chrronic
Diarrhea (K52.9) as 1.04.6/1000 and performance figures of the highest doctor to
doctor internal medicine. Cause not grouped daily inpatient census as input to RL
3.1 is the lack of personal communication between nurses and medical records
and no information or column to the inclusion of cases of patients daily inpatient
census
Subtask-masked curriculum learning for reinforcement learning with application to UAV maneuver decision-making
No full textUnmanned Aerial Vehicle (UAV) maneuver strategy learning remains a challenge when using Reinforcement Learning (RL) in this sparse reward task. In this paper, we propose Subtask-Masked curriculum learning for RL (SUBMAS-RL), an efficient RL paradigm that implements curriculum learning and knowledge transfer for UAV maneuver scenarios involving multiple missiles. First, this study introduces a novel concept known as subtask mask to create source tasks from a target task by masking partial subtasks. Then, a subtask-masked curriculum generation method is proposed to generate a sequenced curriculum by alternately conducting task generation and task sequencing. To establish efficient knowledge transfer and avoid negative transfer, this paper employs two transfer techniques, policy distillation and policy reuse, along with an explicit transfer condition that masks irrelevant knowledge. Experimental results demonstrate that our method achieves a 94.8% success rate in the UAV maneuver scenario, where the direct use of reinforcement learning always fails. The proposed RL framework SUBMAS-RL is expected to learn an effective policy in complex tasks with sparse rewards.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Algorithmic
Reinforcement Learning for Intelligent Healthcare Systems: A Review of Challenges, Applications, and Open Research Issues
No full textThe rise of chronic disease patients and the pandemic pose immediate threats to healthcare expenditure and mortality rates. This calls for transforming healthcare systems away from one-on-one patient treatment into intelligent health systems, leveraging the recent advances of Internet of Things and smart sensors. Meanwhile, reinforcement learning (RL) has witnessed an intrinsic breakthrough in solving a variety of complex problems for distinct applications and services. Thus, this article presents a comprehensive survey of the recent models and techniques of RL that have been developed/used for supporting Intelligent-healthcare (I-health) systems. It can guide the readers to deeply understand the state-of-the-art regarding the use of RL in the context of I-health. Specifically, we first present an overview of the I-health systems' challenges, architecture, and how RL can benefit these systems. We then review the background and mathematical modeling of different RL, deep RL (DRL), and multiagent RL models. We highlight important guidelines on how to select the appropriate RL model for a given problem, and provide quantitative comparisons, showing the results of deploying key RL models in two scenarios that can be followed in monitoring applications. After that, we conduct an in-depth literature review on RL's applications in I-health systems, covering edge intelligence, smart core network, and dynamic treatment regimes. Finally, we highlight emerging challenges and future research directions to enhance RL's success in I-health systems, which opens the door for exploring some interesting and unsolved problems.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Networked System
Upper-Room Ultraviolet Light and Negative Air Ionization to Prevent Tuberculosis Transmission
No full textBackground Institutional tuberculosis (TB) transmission is an important public health problem highlighted by the HIV/AIDS pandemic and the emergence of multidrug- and extensively drug-resistant TB. Effective TB infection control measures are urgently needed. We evaluated the efficacy of upper-room ultraviolet (UV) lights and negative air ionization for preventing airborne TB transmission using a guinea pig air-sampling model to measure the TB infectiousness of ward air. Methods and Findings For 535 consecutive days, exhaust air from an HIV-TB ward in Lima, Perú, was passed through three guinea pig air-sampling enclosures each housing approximately 150 guinea pigs, using a 2-d cycle. On UV-off days, ward air passed in parallel through a control animal enclosure and a similar enclosure containing negative ionizers. On UV-on days, UV lights and mixing fans were turned on in the ward, and a third animal enclosure alone received ward air. TB infection in guinea pigs was defined by monthly tuberculin skin tests. All guinea pigs underwent autopsy to test for TB disease, defined by characteristic autopsy changes or by the culture of Mycobacterium tuberculosis from organs. 35% (106/304) of guinea pigs in the control group developed TB infection, and this was reduced to 14% (43/303) by ionizers, and to 9.5% (29/307) by UV lights (both p < 0.0001 compared with the control group). TB disease was confirmed in 8.6% (26/304) of control group animals, and this was reduced to 4.3% (13/303) by ionizers, and to 3.6% (11/307) by UV lights (both p < 0.03 compared with the control group). Time-to-event analysis demonstrated that TB infection was prevented by ionizers (log-rank 27; p < 0.0001) and by UV lights (log-rank 46; p < 0.0001). Time-to-event analysis also demonstrated that TB disease was prevented by ionizers (log-rank 3.7; p = 0.055) and by UV lights (log-rank 5.4; p = 0.02). An alternative analysis using an airborne infection model demonstrated that ionizers prevented 60% of TB infection and 51% of TB disease, and that UV lights prevented 70% of TB infection and 54% of TB disease. In all analysis strategies, UV lights tended to be more protective than ionizers. Conclusions Upper-room UV lights and negative air ionization each prevented most airborne TB transmission detectable by guinea pig air sampling. Provided there is adequate mixing of room air, upper-room UV light is an effective, low-cost intervention for use in TB infection control in high-risk clinical settings
Treatment of varicocele in pediatric patients with controindication for open surgery with transfemoral retrograde sclero-embolization under local anesthesia
No full textThe author describe an alternative approach for the treatment of varicocele by scleroembolization of spermatic vein
- …
