67639 research outputs found
Sort by
On the Convergence and Stability of Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning, and Online Decision Transformers
No full textThis article provides a rigorous analysis of convergence and stability of Episodic Upside-Down Reinforcement Learning, Goal-Conditioned Supervised Learning and Online Decision Transformers. These algorithms performed competitively across various benchmarks, from games to robotic tasks, but their theoretical understanding is limited to specific environmental conditions. This work initiates a theoretical foundation for algorithms that build on the broad paradigm of approaching reinforcement learning through supervised learning or sequence modeling. At the core of this investigation lies the analysis of conditions on the underlying environment, under which the algorithms can identify optimal solutions. We also assess whether emerging solutions remain stable in situations where the environment is subject to tiny levels of noise. Specifically, we study the continuity and asymptotic convergence of command-conditioned policies, values and the goal-reaching objective depending on the transition kernel of the underlying Markov Decision Process. We demonstrate that near-optimal behavior is achieved if the transition kernel is located in a sufficiently small neighborhood of a deterministic kernel. The mentioned quantities are continuous (with respect to a specific topology) at deterministic kernels, both asymptotically and after a finite number of learning cycles. The developed methods allow us to present the first explicit estimates on the convergence and stability of policies and values in terms of the underlying transition kernels. On the theoretical side we introduce a number of new concepts to reinforcement learning, like working in segment spaces, studying continuity in quotient topologies and the application of the fixed-point theory of dynamical systems. The theoretical study is accompanied by a detailed investigation of example environments and numerical experiments.This work was supported by the European Research Council (ERC, Advanced Grant Num-
ber 742870), the Swiss National Supercomputing Centre (CSCS, Project s1090), and by the
Swiss National Science Foundation (Grant Number 200021 192356, Project NEUSYM). We
also thank both the NVIDIA Corporation for donating a DGX-1 as part of the Pioneers of
AI Research Award and IBM for donating a Minsky machine
The influence of unsaturation modifications on the tribological characteristics of bio-based lubricants obtained from vegetable oils: a review
No full textThe current need for energy transition has driven research on the production of new fuels and lubricants from more biodegradable and renewable raw materials. In this context, the chemical synthesis of vegetable oils and animal fats—from which esterification/transesterification, epoxidation, oxirane ring opening, hydrogenation, and estolide formation stand out—is being successfully applied in generating biolubricants with physicochemical characteristics comparable to those of commercial mineral lubricants. In this article, a literature review was conducted to identify and present outlines of the formation of the lubrication mechanisms of the molecules obtained via the aforementioned chemical processes when subjected to tribological tests. Collected data suggest that biolubricants formed by the association of polar groups and with a lower unsaturation content are more efficient at forming lubricating films that reduce friction and resist wear-inducing shear forces. These chemical pathways demonstrate the possibility of obtaining lubricants with numerous characteristics that depend both on the base oil used and the reagents involved, showing that the future of biolubricants lies in improving these methods.The authors received financial support from CNPq (Conselho Nacional de Pesquisa e Desenvolvimento Científico), to Scientific and Technological Research and Development Support Foundation of Maranhão (FAPEMA) and State University of Maranhão (UEMA). The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper. Data sharing is not applicable to this article, as no new data were created or analyzed in this study. The idea of carrying out a review of the influence of changes in biolubricant molecules on tribological characteristics, presenting sketches of lubrication mechanisms, came from Professor Dr. Paulo Roberto Campos Flexa Ribeiro Filho. Leonardo dos Santos e Santos contributed to the creation of the images present in this work. The authors have read and approved the final version of the manuscript. Paulo Roberto Campos Flexa Ribeiro Filho and Leonardo dos Santos e Santos performed writing of original draft and critically reviewed the results reported in this paper
High-Performance Low-Emissivity Paints Enabled by N-Doped Poly(benzodifurandione) (n-PBDF) for Energy-Efficient Buildings
No full textLow-emissivity (low-e) paints reduce radiative heat exchange between buildings and the environment, stabilizing indoor climates and lowering air conditioning demand. However, low-cost, durable, and colored low-e paints have yet to be demonstrated. Here, an approach is proposed using n-doped poly(benzodifurandione) (n-PBDF), a transparent organic conducting polymer, coated over colored commercial paints. This achieves a low thermal emissivity of 0.19 in the mid-infrared spectrum, attributed to the efficient charge transport of delocalized π-electrons in n-PBDF structure. The reduction in thermal emissivity aids in regulating building temperatures by minimizing heat transfer between buildings and their surroundings across diverse climate zones and seasons. The n-PBDF coating preserves the underlying paint's color due to its high visible transparency, meeting aesthetic requirements. It also shows strong stability in accelerated indoor weathering tests, ensuring long-term performance. Simulations estimate annual HVAC energy savings of over 10,800 kWh in San Francisco and 5,500 kWh in Chicago for the typical mid-rise apartments. The paint's versatility, scalability, and durability make it suitable for buildings, vehicles, and greenhouses, aiding urban heat island mitigation.X.L. acknowledges the Gilbreth Postdoctoral Fellowship at Purdue University. W.\u2010J.L., L.Y., and J.M. acknowledge the support from Ambilight Inc. under contract #4000187.02. J.R. acknowledges the support from the Center for Soft PhotoElectroChemical Systems, an Energy Frontier Research funded by Department of Energy, Office of Science, Basic Energy Sciences, under award #DE\u2010SC0023411. D.W.C. and X.R. acknowledge partial support from the US National Science Foundation through award 2102645, A.F. and O.G.R.G. acknowledge support from the US National Science Foundation through a Graduate Research Fellowship. Q.G. and Y.L. are partially supported by KAUST (BAS/1/1415\u201001\u201001) and FutureWei's gift fund: radiative cooling and thermal management materials for consumer electronics (GIF/5/5705\u201001). A.A. acknowledges the support from Saudi Arabian Cultural Mission (SACM)
Real-Time Reactive Control and 6D Pose Estimation for Robotic Manipulators
No full textReal-time perception and control integration remains a critical challenge for robotic manipulators in dynamic environments. This thesis presents a novel reactive control framework that synergistically combines the model-based version of FoundationPose — a transformer-based, contrastive-learning foundation vision model trained on large-scale synthetic data — with deterministic Model Predictive Control (MPC). The perception pipeline employs a ZED 2i stereo camera in Neural Depth Mode for high-fidelity RGB-D sensing and the Segment Anything Model (SAM) for precise object segmentation, operating at 30 Hz to generate robust 6D pose estimates. Initial registration via textured CAD models is followed by continuous render-and-compare refinement, achieving an ADD of 87.7% and ADD-S of 93.2% against AprilTag ground truth. These estimates feed an acados-generated MPC controller with a one-second prediction horizon and 35 discretization steps, enforcing joint, velocity, and workspace constraints to drive a UR10e manipulator with sub-centimeter precision. Extensive static and dynamic sampling on symmetric (Jenga brick) and asymmetric (L-shaped Soma piece) objects revealed positional variances under 10 mm for low-symmetry cases and bounded spikes up to 20 mm due to symmetry induced ambiguity, while latency profiling showed an average 12 ms inference delay — against 4 ms for AprilTag — enabling stable closed-loop operation at 30 Hz. Demonstrations of a “claw-machine” pick-and-place behavior and a reactive “snake” tracking task validated agile, disturbance-tolerant manipulation in cluttered settings. Together, these results highlight the powerful synergy between advanced neural-network perception and performance guaranteed MPC for robust, real-time robotic manipulation
Optimized design of a fault-tolerant 12-slot/10-pole six-phase surface permanent magnet motor with asymmetrical winding configuration for electric vehicles
No full textThis paper presents a comprehensive methodology for optimizing the design of a 12-slot/10-pole permanent magnet (PM) motor with a six-phase winding configuration tailored for electric vehicles (EVs). The design aims to enhance motor performance under both healthy and fault conditions. While the single neutral configuration offers superior torque during faults, it also introduces zero sequence currents and additional space harmonics, which can lead to increased torque ripple that is difficult to control. This study addresses these challenges through innovative machine design optimization. The optimization process begins with sizing equations to establish an initial design. K-means clustering techniques are then employed to identify distinct loading points that accurately represent the full EV driving cycle, effectively minimizing computational power requirements. Following this, the Full Range Minimum Loss (FRML) strategy is applied to determine optimal current profiles across these loading points, significantly reducing copper losses. Finally, a multi-objective optimization approach is utilized to minimize torque ripple, enhance average torque, and optimize machine losses. The results demonstrate substantial improvements in torque and reduced ripple, validated through experiments conducted with a 2 kW lab-scale motor. This integrated approach not only ensures a robust and efficient motor design but also enhances fault tolerance, making it well-suited for advanced EV applications
Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization
No full textOnline continual learning (OCL) seeks to learn new tasks from data streams that appear only once, while retaining knowledge of previously learned tasks. Most existing methods rely on replay, focusing on enhancing memory retention through regularization or distillation. However, they often overlook the adaptability of the model, limiting the ability to learn generalizable and discriminative features incrementally from online training data. To address this, we introduce a plug-and-play module, S6MOD, which can be integrated into most existing methods and directly improve adaptability. Specifically, S6MOD introduces an extra branch after the backbone, where a mixture of discretization selectively adjusts parameters in a selective state space model, enriching selective scan patterns such that the model can adaptively select the most sensitive discretization method for current dynamics. We further design a class-conditional routing algorithm for dynamic, uncertainty-based adjustment and implement a contrastive discretization loss to optimize it. Extensive experiments combining our module with various models demonstrate that S6MOD significantly enhances model adaptability, leading to substantial performance gains and achieving the state-of-the-art results. The code is available at https://github.com/MyToumaKazusa/S6MOD
NAD <sup>+</sup> reverses Alzheimer’s neurological deficits via regulating differential alternative RNA splicing of <i>EVA1C</i>
No full textDysfunctional alternative splicing events (ASEs) in RNA are markers of aging and Alzheimer’s disease (AD). As a key neuronal resilience metabolite, the oxidized nicotinamide adenine dinucleotide (NAD+) slows down AD progression in preclinical studies with several clinical trials ongoing. However, the underlying molecular mechanisms around how NAD+ enhances neuronal resilience, especially whether it has any effect on ASEs, have remained elusive. This study shows that NAD+ augmentation corrects the ASEs of many genes via a key protein, EVA1C (epithelial V-like antigen 1 homolog C), which is involved in neuronal development and activities. EVA1C is reduced in the hippocampus in patients with AD compared to cognitively normal ones. NAD+-induced memory retention is partially dependent on EVA1C, as adeno-associated virus–based Eva1c knockdown in the hippocampal CA1 region annuls NAD+-induced memory improvement in pathological Tau–bearing mice. We propose that NAD+ reduces AD pathologies, at least partially, via amplification of the NAD+-EVA1C splicing axis, pointing to a potential splice-switching therapy for AD.E.F.F. is also supported by Cure Alzheimer’s Fund (nos. 282952 and 284930), HELSE SØR-ØST (nos. 2020001, 2021021, and 2023093), the Research Council of Norway (nos. 262175 and 334361), Molecule AG/VITADAO (no. 282942), NordForsk Foundation (no. 119986), the National Natural Science Foundation of China (no. 81971327), Akershus University Hospital (nos. 269901, 261973, and 262960), the Civitan Norges Forskningsfond for Alzheimers sykdom (no. 281931), the Czech Republic-Norway KAPPA programme (no. TO01000215, with M. Vyhnálek), the Rosa sløyfe/Norwegian Cancer Society and Norwegian Breast Cancer Society (no. 207819), and HORIZON-TMA-MSCA-DN (no. 101073251, with R. Houtkooper). G.Y. was supported in part by the ERC IMI (101005122), the H2020 (952172), the MRC (MC/PC/21013), the Royal Society (IEC\NSFC\211235), the NVIDIA Academic Hardware Grant Program, the SABER project supported by Boehringer Ingelheim Ltd., NIHR Imperial Biomedical Research Centre (RDA01), The Wellcome Leap Dynamic resilience program (cofunded by Temasek Trust), UKRI guarantee funding for Horizon Europe MSCA Postdoctoral Fellowships (EP/Z002206/1), UKRI MRC Research Grant, TFS Research Grants (MR/U506710/1), Swiss National Science Grants (MR/U506710/1), and the UKRI Future Leaders Fellowship (MR/V023799/1). R.A. is supported by Wellcome Leap’s Dynamic Resilience Program (jointly funded by Temasek Trust). R.A. and S.-q.Z. were funded by the China Scholarship Council (www.csc.edu.cn/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Y.H. is supported by the Natural Science Foundation of China (no. 82101664) and Medical Scientific Research Foundation of Guangdong Province of China (no. A2022007). Support for some of the transgenic nematode models used came from a Collaborative Innovation Award from the Howard Hughes Medical Institute to G. A. Caldwell. J.M.S. was supported by national funds through FCT Contract for Scientific Employment (2021.00204.CEECCIND) and projects UIDB/50026/2020 and UIDP/50026/2020. C.C.-M. was supported by FCT for PhD fellowship. I.S. and F.K. were supported by the Hellenic Foundation for Research and Innovation and National Recovery and Resilience plan Greece 2.0, funded by the European Union–NextGenerationEU (grants 15493 and TAEDR-0535850) and NIH grant 4R01AG069941
Rate Adaptation in Delay-Sensitive and Energy-Constrained Large-Scale IoT Networks
No full textFeedback transmissions are used to acknowledge correct packet reception, trigger erroneous packet re-transmissions, and adapt transmission parameters (e.g., rate and power). Despite the feedback paramount role in establishing reliable communication links, the majority of the literature overlooks its impact by assuming genie-aided systems with flawless and instantaneous feedback. However, this idealistic assumption is no longer valid for large-scale Internet of Things (IoT) networks, characterized by energy-constrained devices, susceptible to interference, and serving delay-sensitive applications. Furthermore, feedback-free operation is necessitated for IoT receivers with stringent energy constraints. In this context, this paper explicitly accounts for the impact of feedback in energy-constrained delay-sensitive large-scale IoT networks. We consider a time-slotted system with closed-loop and open-loop rate adaptation schemes, where packets are fragmented to operate at a reliable transmission rate satisfying packet delivery deadlines. In the closed-loop scheme, the delivery of each fragment is acknowledged through an error-prone feedback channel. The open-loop scheme has no feedback mechanism, and hence, a predetermined fragment repetition strategy is employed to improve transmission reliability. Using stochastic geometry and queueing theory, we develop a novel spatiotemporal framework for both schemes to quantify the impact of feedback on network performance in terms of transmission reliability, latency, and energy consumption
Robust Wetting and Drying with Discontinuous Galerkin Flood Model on Unstructured Triangular Meshes
No full textGodunov-based finite volume (FV) methods are widely employed to numerically solve the Shallow-Water Equations (SWEs) with application to simulate flood inundation over irregular geometries and real-field, where unstructured triangular meshing is favored. Second-order extensions have been devised, mostly on the MUSCL reconstruction and the discontinuous Galerkin (DG) approaches. In this paper, we introduce a novel second-order Runge–Kutta discontinuous Galerkin (RKDG) solver for flood modeling, specifically addressing positivity preservation and wetting and drying on unstructured triangular meshes. To enhance the RKDG model, we adapt and refine positivity-preserving and wetting and drying techniques originally developed for the MUSCL-based finite volume (FV) scheme, ensuring its effective integration within the RKDG framework. Two analytical test problems are considered first to validate the proposed model and assess its performance in comparison with the MUSCL formulation. The performance of the model is further explored in real flooding scenarios involving irregular topographies. Our findings indicate that the added complexity of the RKDG model is justified, as it delivers higher-quality results even on very coarse meshes. This reveals that there is a promise in deploying RKDG-based flood models in real-scale applications, in particular when field data are sparse or of limited resolution.GK was supported by the UK Engineering and Physical Sciences Research Council (EPSRC) grant EP/R007349/1
Data-Fused MPC With Guarantees: Application to Flying Humanoid Robots
No full textThis paper introduces a Data-Fused Model Predictive Control (DFMPC) framework that combines physics-based models with data-driven representations of unknown dynamics. Leveraging Willems’ Fundamental Lemma and an artificial equilibrium formulation, the method enables tracking of changing, potentially unreachable setpoints while explicitly handling measurement noise through slack variables and regularization. We provide guarantees of recursive feasibility and practical stability under input–output constraints for a specific class of reference signals. The approach is validated on the iRonCub flying humanoid robot, integrating analytical momentum models with data-driven turbine dynamics. Simulations show improved tracking and robustness compared to a purely model-based MPC, while maintaining real-time feasibility. Code: https://github.com/ami-iit/papergorbanielobaid2025lcssdfmpc-ironcub