2,872 research outputs found
Teacher-apprentices RL (TARL): leveraging complex policy distribution through generative adversarial hypernetwork in reinforcement learning
No full textTypically, a Reinforcement Learning (RL) algorithm focuses in learning a single deployable policy as the end product. Depending on the initialization methods and seed randomization, learning a single policy could possibly leads to convergence to different local optima across different runs, especially when the algorithm is sensitive to hyper-parameter tuning. Motivated by the capability of Generative Adversarial Networks (GANs) in learning complex data manifold, the adversarial training procedure could be utilized to learn a population of good-performing policies instead. We extend the teacher-student methodology observed in the Knowledge Distillation field in typical deep neural network prediction tasks to RL paradigm. Instead of learning a single compressed student network, an adversarially-trained generative model (hypernetwork) is learned to output network weights of a population of good-performing policy networks, representing a school of apprentices. Our proposed framework, named Teacher-Apprentices RL (TARL), is modular and could be used in conjunction with many existing RL algorithms. We illustrate the performance gain and improved robustness by combining TARL with various types of RL algorithms, including direct policy search Cross-Entropy Method, Q-learning, Actor-Critic, and policy gradient-based methods.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Interactive Intelligenc
BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs
No full textWhile reinforcement learning (RL) has made great advances in scalability, exploration and partial observability are still active research topics. In contrast, Bayesian RL (BRL) provides a principled answer to both state estimation and the exploration-exploitation trade-off, but struggles to scale. To tackle this challenge, BRL frameworks with various prior assumptions have been proposed, with varied success. This work presents a representation-agnostic formulation of BRL under partially observability, unifying the previous models under one theoretical umbrella. To demonstrate its practical significance we also propose a novel derivation, Bayes-Adaptive Deep Dropout rl (BADDr), based on dropout networks. Under this parameterization, in contrast to previous work, the belief over the state and dynamics is a more scalable inference problem. We choose actions through Monte-Carlo tree search and empirically show that our method is competitive with state-of-the-art BRL methods on small domains while being able to solve much larger ones.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Interactive Intelligenc
Refined Risk Management in Safe Reinforcement Learning with a Distributional Safety Critic
No full textSafety is critical to broadening the real-world use of reinforcement learning (RL). Modeling the safety aspects using a safety-cost signal separate from the reward is becoming standard practice, since it avoids the problem of finding a good balance between safety and performance. However, the total safety-cost distribution of different trajectories is still largely unexplored. In this paper, we propose an actor critic method for safe RL that uses an implicit quantile network to approximate the distribution of accumulated safety-costs. Using an accurate estimate of the distribution of accumulated safetycosts, in particular of the upper tail of the distribution, greatly improves the performance of riskaverse RL agents. The empirical analysis shows that our method achieves good risk control in complex safety-constrained environments.AlgorithmicsIntelligent Electrical Power Grid
Paraheliotropism in Robinia pseudoacacia L. plants: an efficient strategy to optimise photosynthetic performance under natural environmental conditions.
No full textWe assessed the contribution of leaf movements to PSII photoprotection
against high light and temperature in Robinia pseudoacacia. Gas exchange
and chlorophyll a fluorescence measurements were performed during the
day at 10:00, 12:00, 15:00 and 18:00 hours on leaves where paraheliotropic
movements were restrained (restrained leaves, RL) and on control unrestrained
leaves (UL). RL showed a strong decrease of net photosynthesis
(An), stomatal conductance (gsH2O), quantum yield of electron transport
(FPSII), percentage of photosynthesis inhibited by O2 (IPO) and photochemical
quenching (qP) in the course of the day, whereas, a significant increase
in Ci ⁄Ca and NPQ was observed. Contrary to RL, UL had higher photosynthetic
performance that was maintained at elevated levels throughout the
day. In the late afternoon, An, gsH2O, FPSII and qP of RL showed a tendency
to recovery, as compared to 15:00 hours, even if the values remained lower
than those measured at 10:00 hours and in UL. In addition, contrary to UL,
no recovery was found in Fv ⁄ Fm at the end of the study period in RL. Data
presented suggest that in R. pseudoacacia, leaf movements, by reducing light
interception, represent an efficient, fast and reversible strategy to overcome
environmental stresses such as high light and temperature. Moreover, paraheliotropism
was able to protect photosystems, avoiding photoinhibitory
damage, leading to a carbon gain for the plant
qgym: A Gym for Training and Benchmarking RL-Based Quantum Compilation
No full textCompiling a quantum circuit for specific quantum hardware is a challenging task. Moreover, current quantum computers have severe hardware limitations. To make the most use of the limited resources, the compilation process should be optimized. To improve currents methods, Reinforcement Learning (RL), a technique in which an agent interacts with an environment to learn complex policies to attain a specific goal, can be used. In this work, we present qgym, a software framework derived from the OpenAI gym, together with environments that are specifically tailored towards quantum compilation. The goal of qgym is to connect the research fields of Artificial Intelligence (AI) with quantum compilation by abstracting parts of the process that are irrelevant to either domain. It can be used to train and benchmark RL agents and algorithms in highly customizable environments.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Quantum Circuit Architectures and Technolog
Influence-Augmented Local Simulators: a Scalable Solution for Fast Deep RL in Large Networked Systems
No full textLearning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.Interactive IntelligenceAlgorithmic
Effects of aging anti-aging caloric restriction on carbonyl and heart shock protein levels and expression
No full textHeat shock proteins (Hsps) are induced by stressful stimuli and have been shown
to protect cells and organs from such stresses both in vitro and in vivo, and
play a positive role in lifespan determination. An attenuated response to stress
is characteristic of senescence and no Hsp induction is observed upon exposure to
stress and no protective effect of a mild stress is observed in cells from aged
individuals. The artificial over-expression of Hsps, can produce a protective
effect against a variety of damaging stimuli in cells from aged rats or aged
humans, in whom cardiovascular disease is a major cause of morbidity in older
age. Here, we show that aging significantly decreases the levels of Hsp27, Hsp60,
Hsp72 and Hsc70 in right atrium and left ventricle of the rat heart, both at
level of protein and of mRNA. Two different caloric restriction regimens have
been found to counteract in part the decrease in the levels of Hsp expression in
the aged heart tissue as well as the tendency to an increase of the levels of
carbonyl in cardiac proteins. Our data suggest that cardiac Hsp levels may be a
determinant of longevity in rodents, and that generation of new regimens of
caloric restriction may eventually show how to improve modulation of cardiac
aging
Rapporto tecnico sui calcoli per la caratterizzazione dei vari impianti nucleari in condizioni di incidente grave
No full textAntibody degradation in tobacco plants: a predominantly apoplastic process.
No full textBACKGROUND: Interest in using plants for production of recombinant proteins such as monoclonal antibodies is growing, but proteolytic degradation, leading to a loss of functionality and complications in downstream purification, is still a serious problem.
RESULTS: In this study, we investigated the dynamics of the assembly and breakdown of a human IgG(1)κ antibody expressed in plants. Initial studies in a human IgG transgenic plant line suggested that IgG fragments were present prior to extraction. Indeed, when the proteolytic activity of non-transgenic Nicotiana tabacum leaf extracts was tested against a human IgG1 substrate, little activity was detectable in extraction buffers with pH > 5. Significant degradation was only observed when the plant extract was buffered below pH 5, but this proteolysis could be abrogated by addition of protease inhibitors. Pulse-chase analysis of IgG MAb transgenic plants also demonstrated that IgG assembly intermediates are present intracellularly and are not secreted, and indicates that the majority of proteolytic degradation occurs following secretion into the apoplastic space.
CONCLUSIONS: The results provide evidence that proteolytic fragments derived from antibodies of the IgG subtype expressed in tobacco plants do not accumulate within the cell, and are instead likely to occur in the apoplastic space. Furthermore, any proteolytic activity due to the release of proteases from subcellular compartments during tissue disruption and extraction is not a major consideration under most commonly used extraction conditions
Subtask-masked curriculum learning for reinforcement learning with application to UAV maneuver decision-making
No full textUnmanned Aerial Vehicle (UAV) maneuver strategy learning remains a challenge when using Reinforcement Learning (RL) in this sparse reward task. In this paper, we propose Subtask-Masked curriculum learning for RL (SUBMAS-RL), an efficient RL paradigm that implements curriculum learning and knowledge transfer for UAV maneuver scenarios involving multiple missiles. First, this study introduces a novel concept known as subtask mask to create source tasks from a target task by masking partial subtasks. Then, a subtask-masked curriculum generation method is proposed to generate a sequenced curriculum by alternately conducting task generation and task sequencing. To establish efficient knowledge transfer and avoid negative transfer, this paper employs two transfer techniques, policy distillation and policy reuse, along with an explicit transfer condition that masks irrelevant knowledge. Experimental results demonstrate that our method achieves a 94.8% success rate in the UAV maneuver scenario, where the direct use of reinforcement learning always fails. The proposed RL framework SUBMAS-RL is expected to learn an effective policy in complex tasks with sparse rewards.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Algorithmic
- …
