10784 research outputs found
Sort by
Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning
No full textPublisher Copyright: © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.In this work, we investigate the means of using curiosity on replay buffers to improve offline multi-task continual reinforcement learning when tasks, which are defined by the non-stationarity in the environment, are non labeled and not evenly exposed to the learner in time. In particular, we investigate the use of curiosity both as a tool for task boundary detection and as a priority metric when it comes to retaining old transition tuples, which we respectively use to propose two different buffers. Firstly, we propose a Hybrid Reservoir Buffer with Task Separation (HRBTS), where curiosity is used to detect task boundaries that are not known due to the task-agnostic nature of the problem. Secondly, by using curiosity as a priority metric when it comes to retaining old transition tuples, a Hybrid Curious Buffer (HCB) is proposed. We ultimately show that these buffers, in conjunction with regular reinforcement learning algorithms, can be used to alleviate the catastrophic forgetting issue suffered by the state of the art on replay buffers when the agent’s exposure to tasks is not equal along time. We evaluate catastrophic forgetting and the efficiency of our proposed buffers against the latest works such as the Hybrid Reservoir Buffer (HRB) and the Multi-Time Scale Replay Buffer (MTR) in three different continual reinforcement learning settings. These settings are defined based on how many times the agent encounters the same task, how long they last, and how different new tasks are when compared to the old ones (i.e., how large the task drift is). The three settings are namely, 1. prolonged task encounter with substantial task drift, and no task re-visitation, 2. frequent, short-lived task encounter with substantial task drift and task re-visitation, and 3. every timestep task encounter with small task drift and task re-visitation. Experiments were done on classical control tasks and Metaworld environment. Experiments show that our proposed replay buffers display better immunity to catastrophic forgetting compared to existing works in all but the every time step task encounter with small task drift and task re-visitation. In this scenario curiosity will always be higher, thus not being an useful measure in both proposed buffers, making them not universally better than other approaches across all types of CL settings, and thereby opening up an avenue for further research.Natalia Díaz-Rodríguez is supported by grant IJC2019-039152-I funded by MCIN/AEI /10.13039/501100011033, by “ESF Investing in your future,” Google Research Scholar Program, and by the European Union through the Marie Curie Postdoctoral Fellowship (Project: 101059332 - RRR-XAI - HORIZON-MSCA-2021-PF-01). J. Del Ser is supported by the Basque Government through the ELKARTEK program and the consolidated research group MATHMODE (IT1456-22).Peer reviewe
An efficient frequency-domain based methodology for the preliminary design of FOWT substructures
Get PDFPublisher Copyright: © 2024 Institute of Physics Publishing. All rights reserved.An efficient frequency domain numerical tool for the preliminary design of the Floating Offshore Wind (FOW) substructures is developed, and validated against one of the leading state-of-the-art time domain based numerical tools: OpenFAST. It has been proven that the present tool enables to reach to similar conclusions as the state-of-the-art tools with a substantial reduction of the computational effort. This is achieved through two novel methods that obtain the hydrodynamic loads acting on the floating substructures. One estimates the linear hydrodynamic coefficients, and the other predicts the second-order wave loads. When combined with a simplified frequency domain response model, a quick assessment of the dynamics of numerous platform designs can be performed, accelerating the floater preliminary design process. The proposed method has been validated by carrying out a comparative analysis on a semi-submersible platform design space, characterised by the column diameter, the separation between the columns and the platform draft. The response motions have been compared against OpenFAST. Furthermore, the high efficiency of the proposed frequency domain tool makes it suitable to perform several sensitivity analyses that will allow for a better understanding of the dynamics and fine tune the foundations design accordingly from the early stages of design.Peer reviewe
Fuzzy Logic Based Decision-Making for Urban Platooning on Urban Roundabout Scenarios
No full textPublisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.This paper proposes a fuzzy-based decision-making framework for urban platooning in roundabout scenarios. By utilizing fuzzy logic to handle uncertainties and imprecise inputs, the framework adapts the behavior of platoon vehicles based on real-time traffic conditions, vehicle dynamics, and safety considerations. In addition, a MPC-based platoon following controller is proposed to execute the actions defined by the decision-making approach. This approach is tested in Carla simulator with successful results, proving the proposal is feasible for platoon handling in urban roundabouts.Peer reviewe
A transdiagnostic group exercise intervention for mental health outpatients in Germany (ImPuls): results of a pragmatic, multisite, block-randomised, phase 3 controlled trial
No full textPublisher Copyright: © 2024 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 licenseBackground: Globally, mental health conditions pose a substantial burden of disease. Despite the availability of evidence-based pharmacological and psychological treatments, the symptoms of a substantial subgroup of patients do not respond to these interventions, and only a minority of patients have access to them. This study aimed to assess the efficacy of ImPuls, a 6-month transdiagnostic group exercise intervention, plus treatment-as-usual, compared with treatment-as-usual alone in outpatients with various mental disorders. Methods: In this pragmatic, two-arm, multisite, randomised controlled trial in Germany, ten outpatient rehabilitative and medical care facilities were involved as study sites. Participants were outpatients diagnosed according to ICD-10 with one or more of the following disorders based on structured clinical interviews: moderate or severe depression, primary insomnia, post-traumatic stress disorder (PTSD), panic disorder, or agoraphobia. Participants were required to be aged between 18 years and 65 years, insured by the health insurers Allgemeine Ortskrankenkasse Baden-Württemberg or Techniker Krankenkasse, fluent in German, and without medical contraindications for exercise. Blocks of six participants were randomly allocated to ImPuls plus treatment-as-usual or treatment-as-usual alone (allocation ratio: 1:1), stratified by study site. The randomisation sequence was generated by an external data manager. The team responsible for data collection and management was masked to the randomisation sequence. The ImPuls intervention comprised evidence-based outdoor exercises lasting 30 min, and aimed at achieving at least moderate intensity. It also incorporated behavioural change techniques targeting motivational and volitional determinants of exercise behaviour. Treatment-as-usual was representative of typical outpatient health care in Germany, allowing patients access to any standard treatments. The primary outcome was global symptom severity at 6 months after randomisation, measured using self-report on the Brief Symptom Inventory (BSI-18) and analysed in the intention-to-treat sample. No individuals with lived experience of mental illness were involved in conducting the study or writing the final publication. Safety was assessed in all participants. The trial was registered with the German Clinical Trials Register (DRKS00024152) with a completion date of June 30, 2024. Findings: 600 patients provided informed consent, were recruited to the study, and underwent a diagnostic interview between Jan 1, 2021, and May 31, 2022. Following this, 199 were excluded on the basis of inclusion and exclusion criteria and one withdrew consent during the baseline assessment. Of the 400 eligible participants, 284 (71%) self-identified as female, 106 (27%) self-identified as male, and nine (2%) self-identified as other. The mean age was 42·20 years (SD 13·23; range 19–65). Ethnicity data were not assessed. 287 (72%) participants met the criteria for moderate or severe depression, 81 (20%) for primary insomnia, 37 (9%) for agoraphobia, 46 (12%) for panic disorder, and 72 (18%) for PTSD. 199 participants were allocated to the intervention group of ImPuls plus treatment-as-usual and 201 to the control group of treatment-as-usual alone. 38 (19%) participants did not receive the minimum ImPuls intervention dose. ImPuls plus treatment-as-usual demonstrated superior efficacy to treatment-as-usual alone in reducing global symptom severity, with an adjusted difference on BSI-18 of 4·11 (95% CI 1·74–6·48; d=0·35 [95% CI 0·14–0·56]; p=0·0007) at 6 months. There were no significant differences in the total number of adverse events or serious adverse events between the two groups. There was one serious adverse event (male, torn ligament) related to the intervention. Interpretation: ImPuls is an efficacious transdiagnostic adjunctive treatment in outpatient mental health care. Our findings suggest that exercise therapy should be implemented in outpatient mental health care as an adjunctive transdiagnostic treatment for mental disorders such as depression, insomnia, panic disorder, agoraphobia, and PTSD. Transdiagnostic group exercise interventions might ameliorate the existing disparity in care provision between the many individuals in need of evidence-based treatment and the few who are receiving it. Funding: The German Innovation Fund of the Federal Joint Committee of Germany.Peer reviewe
Motor Intentions Decoded from fMRI Signals
Get PDFPublisher Copyright: © 2024 by the authors.Motor intention is a high-level brain function related to planning for movement. Although studies have shown that motor intentions can be decoded from brain signals before movement execution, it is unclear whether intentions relating to mental imagery of movement can be decoded. Here, we investigated whether differences in spatial and temporal patterns of brain activation were elicited by intentions to perform different types of motor imagery and whether the patterns could be used by a multivariate pattern classifier to detect such differential intentions. The results showed that it is possible to decode intentions before the onset of different types of motor imagery from functional MR signals obtained from fronto-parietal brain regions, such as the premotor cortex and posterior parietal cortex, while controlling for eye movements and for muscular activity of the hands. These results highlight the critical role played by the aforementioned brain regions in covert motor intentions. Moreover, they have substantial implications for rehabilitating patients with motor disabilities.Peer reviewe
Non-isocyanate polyurethanes derived from carbonated soybean oil: Synthesis, characterization and comparison with traditional vegetable oil-based polyurethanes
No full textPublisher Copyright: © 2024 The AuthorsThe polyaddition between carbonated vegetable oils and diamines to form non-isocyanate polyurethanes (NIPUs) has been highlighted as a safer and more sustainable alternative to traditional polyurethanes (PUs) synthesis. Although NIPUs exhibit a broad range of properties, research so far has been more focused on exploring their synthetic routes rather than on the evaluation of their performance versus traditional PUs in a given application making complicate to determine if NIPUs can be considered an efficient and plausible, in terms of performance, alternative to their fossil fuel-bases PUs counterparts. In this work, a series of carbonate soybean oil-based NIPUs (CSBO-NIPUs) have been prepared by polyaddition of CSBO with three different diamines (MBCHA, DAO and Primaine1074) yielding CSBO-NIPUs with bio-base content well above 73 %. The obtained CBSO-NIPUs exhibited glass transition temperatures between −29 °C and 21 °C, decomposition temperatures in the range of 261 °C to 304 °C, excellent impact resistance (>100 kg/cm2), soft pencil hardness (6B) and a polymeric network with up to 74 % cross-linking degree. The performance of the prepared CSBO-NIPUs was compared with PUs (found in the scientific literature) synthesized from vegetable oil-based polyols and diisocyanates by the traditional synthetic route. The comparison revealed that CSBO-NIPUs show comparable or superior potential than the vegetable oil-based traditional PUs in terms of thermal stability, impact resistance and cross-linking degree. In conclusion, the polyaddition process sustainability in line with Green Chemistry principles along with CBSO-NIPUs excellent thermal and impact resistance place these materials in a promising position a viable alternative to traditional PUs in the near future.Peer reviewe
Analysis of the Impact of New Singular Ventilation Technologies on Enhancing Indoor Air Quality in Schools
No full textPublisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.Concern about indoor air quality (IAQ) in schools has grown in recent years, particularly in the aftermath of the COVID-19 pandemic, which underscored its impact on children's health. Existing educational buildings employ simple strategies such as opening windows to ventilate classrooms. While this approach achieves the goal of air renewal, it compromises energy efficiency and user comfort. In this context, there is a need to address ventilation in schools from a new perspective, providing innovative technologies that allow quick and simple installation while guaranteeing high standards of air quality, energy efficiency, and user comfort. In response to this challenge, a new solution has been developed consisting of autonomous equipment installed inside each classroom and featuring independent intelligent control. The objective of the present study is to evaluate the applicability and the social impact that the widespread implementation of this alternative technology could have compared to conventional methods. The general characteristics and the specific peculiarities and needs of schools in the Basque Country (CAPV) in Spain were determined. The study first approached the topic theoretically through bibliographic references and statistical analysis, and subsequently, fieldwork to assess the reality of existing buildings. Additionally, an air quality monitoring campaign was carried out in pilot schools, conducted in two stages: first without ventilation and later with the new solution. The study evaluated the benefits in terms of improved air quality achieved, as well as the improvements in the implementation and operational processes. These results were extrapolated to Basque educational buildings, providing an estimation of the potential impact of this new ventilation approach. Highly positive results were yielded in terms of acceptance, feasibility, and ultimately, addressing the identified challenges.Peer reviewe
Floating Neutral Detection Using Actual Generation of Form 2S Meters
Get PDFPublisher Copyright: © 2020 IEEE.In the low-voltage distribution system of the USA, Canada and some countries of Central and South America, the most used configuration is the single-phase three-wire system (120 V/240 V) also known as the split-phase distribution system. When the neutral wire of the distribution system gets damaged or broken the current returns through the ground and a floating neutral condition arises. Service to the house continues without interruptions because no high over-currents come up. If the return path impedance is high enough, the equally balanced voltage system gets shifted, going out of boundaries and causing malfunctions in the appliances or even fire. A new classification-based detector is proposed to detect this condition, which only needs current measurements that the actual generation of form 2S meter gathers. Moreover, due to the simplicity of the algorithm, it can be embedded in the current generation of meters, which represents great potential of the detector. To that end, the low-voltage distribution system is modelled using a real database and some assumptions are made. The proposed novel detector approach shows zero false alarms in the houses tested and a detection time that allows the fault to be detected before significant damage occurs to the house.Peer reviewe
Interplay between the oxidation process and cytotoxic effects of antimonene nanomaterials
Get PDFPublisher Copyright: © 2024 The Royal Society of Chemistry.Pnictogen nanomaterials have recently attracted researchers’ attention owing to their promising properties in the field of electronic, energy storage, and nanomedicine applications. Moreover, especially in the case of heavy pnictogens, their chemistry allows for nanomaterial synthesis using both top-down and bottom-up approaches, yielding materials with remarkable differences in terms of morphology, size, yield, and properties. In this study, we carried out a comprehensive structural and spectroscopic characterization of antimony-based nanomaterials (Sb-nanomaterials) obtained by applying different production methodologies (bottom-up and top-down routes) and investigating the influence of the synthesis on their oxidation state and stability in a biological environment. Indeed, in situ XANES/EXAFS studies of Sb-nanomaterials incubated in cell culture media were carried out, unveiling a different oxidation behavior. Furthermore, we investigated the cytotoxic effects of Sb-nanomaterials on six different cell lines: two non-cancerous (FSK and HEK293) and four cancerous (HeLa, SKBR3, THP-1, and A549). The results reveal that hexagonal antimonene (Sb-H) synthesized using a colloidal approach oxidizes the most and faster in cell culture media compared to liquid phase exfoliated (LPE) antimonene, suffering acute degradation and anticipating well-differentiated toxicity from its peers. In addition, the study highlights the importance of the synthetic route for the Sb-nanomaterials as it was observed to influence the chemical evolution of Sb-H into toxic Sb oxide species, playing a critical role in its ability to rapidly eliminate tumor cells. These findings provide insights into the mechanisms underlying the dark cytotoxicity of Sb-H and other related Sb-nanomaterials, underlining the importance of developing therapies based on controlled and on-demand nanomaterial oxidation.Peer reviewe
