1,273 research outputs found
Transfer learning to enhance the scalability of artificial intelligence-based control strategies in buildings
No full textL'abstract è presente nell'allegato / the abstract is in the attachmen
Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings
No full textIn recent years, advanced control strategies based on Deep Reinforcement Learning (DRL) proved to be effective in optimizing the management of integrated energy systems in buildings, reducing energy costs and improving indoor comfort conditions when compared to traditional reactive controllers. However, the scalability and implementation of DRL controllers are still limited since they require a considerable amount of time before converging to a near-optimal solution. This issue is currently addressed in literature through the offline pre -training of the DRL agent. However this solution results in two main critical issues: (1) the need to develop a building surrogate model to perform the training task, and (2) the need to perform a fine-tuning process over several training episodes to obtain a near-optimal control policy.In this context, this paper introduces an Online Transfer Learning (OTL) strategy that exploits two knowledge-sharing techniques, weight-initialization and imitation learning, to transfer a DRL control policy from a source office building to various target buildings in a simulation environment coupling EnergyPlus and Python.A DRL controller based on discrete Soft Actor-Critic (SAC) is trained on the source building to manage the operation of a cooling system consisting of a chiller and a thermal storage. Several target buildings are defined to benchmark the performance of the OTL strategy with that of a Rule-Based Controller (RBC) and two DRL-based control strategies, deployed in offline and online fashion. The strategy adopted for OTL emulates the real world implementation with a simulation process by implementing the transferred DRL agent for a single episode in the target buildings. Target buildings have the same geometrical features and are served by the same energy system as the source building, but differ in terms of weather conditions, electricity price schedules, occupancy patterns, and building envelope efficiency levels. The results show that the OTL strategy can reduce the cumulated sum of temperature violations on average by 50% and 80% respectively when compared to RBC and online DRL while enhancing the energy system operation with electricity cost savings ranging between 20% and 40%. The OTL agent performs slightly worse than the offline DRL controller but it does not require any modeling effort and can be implemented directly on target buildings emulating a real-world implementation
Practical deployment of reinforcement learning for building controls using an imitation learning approach
No full textThis paper addresses the critical need for more efficient and adaptive building control systems to maximise occupant comfort while reducing energy consumption. Our objective is to explore the practical application of model-free Deep Reinforcement Learning (DRL) in real-world building environments by developing a system that learns and adapts to changing conditions, beginning its operation by imitating an existing Rule-Based Control (RBC) system. This approach ensures initial reliability and performance while setting the stage for advanced learning capabilities. The methodology involves two distinct phases. Initially, the DRL controller mimics the behaviour of the RBC system, using imitation learning with behavioural cloning as a safe and efficient strategy to achieve baseline operational efficiency. Subsequently, the controller is implemented within a real building in an online learning setting. In this phase, the controller utilises real-time data to continuously refine its control policy, responding adaptively to occupant behaviours and external environmental conditions. To validate our approach, we conducted a comprehensive analysis, comparing the performance of our DRL controller against the baseline RBC controller, another RBC, and a PI (Proportional-Integral) controller implemented in a digital twin model of the real office environment. Energy consumption and temperature violations related to a temperature acceptability range are considered as metrics, providing a robust framework for assessing the effectiveness of our system. The results indicate that our DRL controller, supported by imitation learning, outperforms the two RBCs by reducing energy consumption by 40 % while reducing the cumulative sum of temperature violations by 43 % and 13 % with respect to the two RBCs. Although the PI controller ensures better performance in terms of temperature violations compared to DRL, it requires 45 % more energy than the proposed DRL controller due to its inherent inability to deal with multi-objective control problems. In conclusion, this paper demonstrates the feasibility and advantages of implementing advanced DRL techniques in real-world building control scenarios. Integrating imitation learning with a DRL controller offers a novel and effective way to enhance the scalability of DRL systems, expanding their application in buildings and driving significant improvements in energy efficiency
Comparison of two deep reinforcement learning algorithms towards an optimal policy for smart building thermal control
No full textHeating, Ventilation, and Air Conditioning (HVAC) systems are the main providers of occupant comfort, and at the same time, they represent a significant source of energy consumption. Improving their efficiency is essential for reducing the environmental impact of buildings. However, traditional rule-based and model-based strategies are often inefficient in real-world applications due to the complex building thermal dynamics and the influence of heterogeneous disturbances, such as unpredictable occupant behavior. In order to address this issue, the performance of two state-of-the-art model-free Deep Reinforcement Learning (DRL) algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC), has been compared when the percentage valve opening is managed in a thermally activated building system, modeled in a simulated environment from data collected in an existing office building in Switzerland. Results show that PPO reduced energy costs by 18% and decreased temperature violations by 33%, while SAC achieved a 14% reduction in energy costs and 64% fewer temperature violations compared to the onsite Rule-Based Controller (RBC)
Recommended from our members
An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems
No full textDeep Reinforcement Learning (DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers (RBCs), but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process. Transfer Learning (TL) is a potential solution to address this limitation. However, existing TL applications in building control have been mostly tested among buildings with similar features, not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems. This paper assesses the performance of an online heterogeneous TL strategy, comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python. The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage (TES). The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems, different building envelope features, occupancy schedule and boundary conditions (e.g., weather and price signal). The TL approach incorporates model slicing, imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings. Results show that the proposed methodology leads to a reduction of 10% in electricity cost and between 10% and 40% in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers. Moreover, online TL maximises self-sufficiency and self-consumption by 9% and 11% with respect to RBC. Conversely, online TL achieves worse performance compared to offline DRL in either transductive or inductive settings. However, offline Deep Reinforcement Learning (DRL) agents should be trained at least for 15 episodes to reach the same level of performance as the online TL. Therefore, the proposed online TL methodology is effective, completely model-free and it can be directly implemented in real buildings with satisfying performance
A scalable approach for real-world implementation of deep reinforcement learning controllers in buildings based on online transfer learning: The HiLo case study
No full textIn recent years, Transfer Learning (TL) has emerged as a promising solution to scale Deep Reinforcement Learning (DRL) controllers for building energy management, addressing challenges related to DRL implementation as high data requirements and reliance on surrogate models. Moreover, most TL applications are limited to simulations, not revealing their real performance in actual buildings. This paper explores the implementation of an online TL methodology combining imitation learning and fine-tuning to transfer a DRL controller between two real office environments. Pre-trained in simulation using a calibrated digital twin, the DRL controller reduces energy consumption and improves indoor temperature control when managing the operation of a Thermally Activated Building System in one of the two offices both in simulation and in the real field. Afterwards, the DRL controller is transferred to the other office following the online TL methodology. The proposed approach outperforms a DRL controller implemented without pre-training, and Rule-Based and Proportional-Integral controllers, achieving energy savings between 6 and 40% and improving indoor temperature control between 30 and 50%. These findings underscore the efficacy of the online TL methodology as a viable solution to enhance the scalability of DRL controllers in real buildings
Correction to: When terminology hinders research: the colloquialisms of transitions of control in automated driving (Cognition, Technology & Work, (2022), 10.1007/s10111-022-00705-3)
No full textIn the original article, author affiliation published with error. The correct affiliations are: Davide Maggi—Institute for Transport Studies, Leeds, UK. Richard Romano—Institute for Transport Studies, Leeds, UK. Oliver Carsten—Institute for Transport Studies, Leeds, UK. Joost C. F. De Winter—Faculty of Mechanical, Maritime and Materials Engineering, Delft University of Technology, Delft, The Netherlands. The original article has been corrected.Human-Robot Interactio
Admiel Kosman, Siamo giunti a Dio
No full textInternational audienceSix poems from Israeli poet Admiel Kosman translated from the Hebrew into Italian. Selection of poems, presentation of the author, translation and notes by Davide Mano
Admiel Kosman, Siamo giunti a Dio
No full textInternational audienceSix poems from Israeli poet Admiel Kosman translated from the Hebrew into Italian. Selection of poems, presentation of the author, translation and notes by Davide Mano
Real building implementation of a deep reinforcement learning controller to enhance energy efficiency and indoor temperature control
No full textISSN:0306-2619ISSN:1872-9118ISSN:1872-911
- …
