OIST Institutional Repository
Not a member yet
2744 research outputs found
Sort by
Efficient and Noise-Tolerant Reinforcement Learning Algorithms via Theoretical Analysis of Gap-Increasing and Softmax Operators
Get PDFOkinawa Institute of Science and Technology Graduate UniversityDoctor of PhilosophyModel-free deep Reinforcement Learning (RL) algorithms, a combination of deep learning and model-free RL algorithms, have attained remarkable successes in solving complex tasks such as video games. However, theoretical analyses and recent empirical results indicate its proneness to various types of value update errors including but not limited to estimation error of updates due to finite samples and function approximation error. Because real-world tasks are inherently complex and stochastic, such errors are inevitable, and thus, the development of error-tolerant RL algorithms are of great importance for applications of RL to real problems. To this end, I propose two error-tolerant algorithms for RL called Conservative Value Iteration (CVI) and Gap-increasing RetrAce for Policy Evaluation (GRAPE).
CVI unifies value-iteration-like single-stage-lookahead algorithms such as soft value iteration, advantage learning and Ψ-learning, all of which are characterized by the use of a gap-increasing operator and/or softmax operator in value updates. We provide detailed theoretical analysis of CVI that not only shows CVI's advantages but also contributes to the theory of RL in the following two points: First, it elucidates pros and cons of gap-increasing and softmax operators. Second, it provides an actual example in which performance of algorithms with max operator is worse than that of algorithms with softmax operator demonstrating the limitation of traditional greedy value updates.
GRAPE is a policy evaluation algorithm extending advantage learning (AL) and retrace, both of which have different advantages: AL is noise-tolerant as shown through our theoretical analysis of CVI, while retrace is efficient in that it is off-policy and allows the control of bias-variance trade-off. Theoretical analys is of GRAPE shows that it enjoys the merits of both algorithms. In experiments, we demonstrate the benefit of GRAPE combined with a variant of trust region policy optimization and its superiority to previous algorithms.
Through these studies, I theoretically elucidated the benefits of gap-increasing and softmax operators in both policy evaluation and control settings. While some open problems remain as explained in the final chapter, the results presented in this thesis are an important step towards a deep understanding of RL algorithms.doctoral thesi
Sedimentation of finite-size particles in quiescent wall-bounded shear-thinning and Newtonian fluids
Get PDFWe study the sedimentation of finite-size particles in quiescent wall-bounded Newtonian and shearthinning fluids by interface resolved numerical simulations. The suspended phase consists of NonBrownian rigid spherical particles with particle to fluid density ratio ρp /ρf = 1 . 5 at three different solid volume fractions [Formula: see text] = 1% , 5% and 20%. Firstly, to focus on the effect of shear-thinning on the particle dynamics and interactions, the Archimedes number is increased for a single particle to have the same settling speed in the Newtonian fluid as in the shear-thinning fluid. Secondly, we consider fixed Archimedes and vary the shear-thinning properties of the fluid. Overall, we report a twofold effect of shear thinning. First and more important, the substantial increase of the particle sedimentation velocity in the shearthinning case due to the increase of the shear rate around the particles, which reduces the local viscosity leading to a reduced particle drag. Secondly, the shear-thinning fluid reduces the level of particle interactions, causing a reduction of velocity fluctuations and resulting in particles sedimenting at approximately the same speed. Moreover, the mean settling velocities decrease with the particle concentration as a consequence of the hindering effect. Particles tend to sediment in the middle of the channel, preferentially positioning in the wake of neighbouring particles or aside them, resulting in lower levels of fluid velocity fluctuations in the gravity direction in the shear-thinning fluid.journal articl
Organelle inheritance and genome architecture variation in isogamous brown algae
Get PDFAmong the brown algal lineages, Ectocarpales species have isogamous fertilization in which male and female gametes are morphologically similar. In contrast, female gametes are much larger than male gametes in the oogamous species found in many other brown algal lineages. It has been reported that the plastids of isogamous species are biparentally inherited whereas the plastids of oogamous species are maternally inherited. In contrast, in both isogamous and oogamous species, the mitochondria are usually inherited maternally. To investigate whether there is any relationship between the modes of inheritance and organellar genome architecture, we sequenced six plastid genomes (ptDNA) and two mitochondrial genomes (mtDNA) of isogamous species from the Ectocarpales and compared them with previously sequenced organellar genomes. We found that the biparentally inherited ptDNAs of isogamous species presented distinctive structural rearrangements whereas maternally inherited ptDNAs of oogamous species showed no rearrangements. Our analysis permits the hypothesis that structural rearrangements in ptDNAs may be a consequence of the mode of inheritance.journal articl
Genomic Signature of Shifts in Selection in a Subalpine Ant and Its Physiological Adaptations
Get PDFUnderstanding how organisms adapt to extreme environments is fundamental and can provide insightful case studies for both evolutionary biology and climate-change biology. Here, we take advantage of the vast diversity of lifestyles in ants to identify genomic signatures of adaptation to extreme habitats such as high altitude. We hypothesized two parallel patterns would occur in a genome adapting to an extreme habitat: 1) strong positive selection on genes related to adaptation and 2) a relaxation of previous purifying selection. We tested this hypothesis by sequencing the high-elevation specialist Tetramorium alpestre and four other phylogenetically related species. In support of our hypothesis, we recorded a strong shift of selective forces in T. alpestre, in particular a stronger magnitude of diversifying and relaxed selection when compared with all other ants. We further disentangled candidate molecular adaptations in both gene expression and protein-coding sequence that were identified by our genome-wide analyses. In particular, we demonstrate that T. alpestre has 1) a higher level of expression for stv and other heat-shock proteins in chill-shock tests and 2) enzymatic enhancement of Hex-T1, a rate-limiting regulatory enzyme that controls the entry of glucose into the glycolytic pathway. Together, our analyses highlight the adaptive molecular changes that support colonization of high-altitude environments.journal articl
Numerical simulations of a sphere settling in simple shear flows of yield stress fluids
Get PDFWe perform three-dimensional numerical simulations to investigate the sedimentation of a single sphere in the absence and presence of a simple cross-shear flow in a yield stress fluid with weak inertia. In our simulations, the settling flow is considered to be the primary flow, whereas the linear cross-shear flow is a secondary flow with amplitude 10 % of the primary flow. To study the effects of elasticity and plasticity of the carrying fluid on the sphere drag as well as the flow dynamics, the fluid is modelled using the elastoviscoplastic constitutive laws proposed by Saramito (J. Non-Newtonian Fluid Mech., vol. 158 (1-3), 2009, pp. 154-161). The extra non-Newtonian stress tensor is fully coupled with the flow equation and the solid particle is represented by an immersed boundary method. Our results show that the fore-aft asymmetry in the velocity is less pronounced and the negative wake disappears when a linear cross-shear flow is applied. We find that the drag on a sphere settling in a sheared yield stress fluid is reduced significantly compared to an otherwise quiescent fluid. More importantly, the sphere drag in the presence of a secondary cross-shear flow cannot be derived from the pure sedimentation drag law owing to the nonlinear coupling between the simple shear flow and the uniform flow. Finally, we show that the drag on the sphere settling in a sheared yield stress fluid is reduced at higher material elasticity mainly due to the form and viscous drag reduction.journal articl
Syntheses and Characterizations of Functional Polycyclic Aromatic Hydrocarbons and Graphene Nanoribbons
Get PDFIn contrast to zero-bandgap graphene, nanostructures of graphene, such as graphene quantum dots (GQDs) and graphene nanoribbons (GNRs) have open bandgaps due to the quantum confinement effect, and are thus highly interesting for semiconductor applications, for example in nanoelectronics and optoelectronics. While conventional methods cannot provide GQDs and GNRs with chemically precise structures, large polycyclic aromatic hydrocarbon (PAH) molecules can be regarded as atomically precise GQDs. Moreover, extension of the PAH synthesis can lead to GNRs with well-defined chemical structures. In this account, we summarize our recent achievements in our synthetic exploration of PAHs and GNRs with novel structures and properties. For example, we have developed new PAHs having zigzag edges, such as dibenzo[hi,st]ovalene derivatives with strong red luminescence and stimulated emission, which are promising for light-emitting devices and bioimaging applications. We have also accomplished a synthesisof magnetic GNRs through edge functionalization with organic radicals, which can be interesting for spintronicas well as quantum computing applications. Moreover, incorporation of zigzag edges in GNR structures, through on-surface syntheses under ultrahigh (UHV) vacuum conditions, allowed for significant modulations of the electronic structures of GNRs, leading to the emergence of topological quantum phases. On the other hand, we have also explored onsurface synthesisof GNRs without UHV, namelyusing a setup for chemical vapor deposition (CVD). Scalable fabrication of GNR films could thus be achieved on goldonmica substrates, which couldbe integrated into field-effect transistor devices. These results highlight the importance of developing novel PAHs and GNRs and their potentials for various applications, including quantum technologies, energy and optoelectronic devices, and bioimaging.journal articl
The Impact of Atmosphere on Energetics of Lead Halide Perovskites
Get PDFSolar cells based on metal halide perovskites have emerged as a promising low-cost photovoltaic technology. In contrast to inert atmospheres where most of the lab-scale devices are made to date, large-area low-cost production of perovskite solar cells often involves processing of perovskites in various atmospheres including ambient air, nitrogen, and/or vacuum. Herein, the impact of atmosphere on the energy levels of methylammonium lead halide perovskite films is systematically investigated. The atmosphere is varied to simulate the typical fabrication process. Through a comprehensive analysis combining the Fermi level evolution, surface photovoltage, photoluminescence properties, photovoltaic performance, and device simulation, an overall landscape of the energy diagram of the perovskite layer is able to be determined. The findings have direct implications for real-world devices under typical atmospheres, and provide insights into the fabrication-process design and optimization. Furthermore, a universal Fermi level shift under vacuum for lead halide-based perovskites revealed in this study, urges a refreshed view on the energetics studies conducted without considering the atmospheric effect.journal articl
Many-body quantum dynamics and induced correlations of Bose polarons
Get PDFWe study the ground state properties and non-equilibrium dynamics of two spinor bosonic impurities immersed in a one-dimensional bosonic gas upon applying an interspecies interaction quench. For the ground state of two non-interacting impurities we reveal signatures of attractive induced interactions in both cases of attractive or repulsive interspecies interactions, while a weak impurity-impurity repulsion forces the impurities to stay apart. Turning to the quench dynamics we inspect the time-evolution of the contrast unveiling the existence, dynamical deformation and the orthogonality catastrophe of Bose polarons. We find that for an increasing postquench repulsion the impurities reside in a superposition of two distinct two-body configurations while at strong repulsions their corresponding two-body correlation patterns show a spatially delocalized behavior evincing the involvement of higher excited states. For attractive interspecies couplings, the impurities exhibit a tendency to localize at the origin and remarkably for strong attractions they experience a mutual attraction on the two-body level that is imprinted as a density hump on the bosonic bath.journal articl
Approaching the quantum critical point in a highly correlated all-in–all-out antiferromagnet
Get PDFContinuous quantum phase transition involving all-in–all-out (AIAO) antiferromagnetic order in strongly spin-orbit-coupled 5d compounds could give rise to various exotic electronic phases and strongly-coupled quantum critical phenomena. Here we experimentally trace the AIAO spin order in Sm2Ir2O7 using direct resonant x-ray magnetic diffraction techniques under high pressure. The magnetic order is suppressed at a critical pressure Pc=6.30GPa, while the lattice symmetry remains in the cubic Fd−3m space group across the quantum critical point. Comparing pressure tuning and the chemical series R2Ir2O7 reveals that the approach to the AIAO quantum phase transition is characterized by contrasting evolutions of the pyrochlore lattice constant a and the trigonal distortion surrounding individual Ir moments, which affects the 5d bandwidth and the Ising anisotropy, respectively. We posit that the opposite effects of pressure and chemical tuning lead to spin fluctuations with different Ising and Heisenberg character in the quantum critical region. Finally, the observed low pressure scale of the AIAO quantum phase transition in Sm2Ir2O7 identifies a circumscribed region of P-T space for investigating the putative magnetic Weyl semimetal state.journal articl
Carbon-Ion Beam Irradiation Alone or in Combination with Zoledronic acid Effectively Kills Osteosarcoma Cells
Get PDFOsteosarcoma(OSA)isthemostcommonmalignantbonetumorinchildrenandadolescents. The overall five-year survival rate for all bone cancers is below 70%; however, when the cancer has spread beyond the bone, it is about 15–30%. Herein, we evaluated the effects of carbon-ion beam irradiation alone or in combination with zoledronic acid (ZOL) on OSA cells. Carbon-ion beam irradiation in combination with ZOL significantly inhibited OSA cell proliferation by arresting cell cycle progression and initiating KHOS and U2OS cell apoptosis, compared to treatments with carbon-ion beam irradiation, X-ray irradiation, and ZOL alone. Moreover, we observed that this combination greatly inhibited OSA cell motility and invasion, accompanied by the suppression of the Pi3K/Akt and MAPK signaling pathways, which are related to cell proliferation and survival, compared to individual treatments with carbon-ion beam or X-ray irradiation, or ZOL. Furthermore, ZOL treatment upregulated microRNA (miR)-29b expression; the combination with a miR-29b mimic further decreased OSA cell viability via activation of the caspase 3 pathway. Thus, ZOL-mediated enhancement of carbon-ion beam radiosensitivity may occur via miR-29b upregulation; co-treatment with the miR-29b mimic further decreased OSA cell survival. These findings suggest that the carbon-ion beam irradiation in combination with ZOL has high potential to increase OSA cell death.journal articl