1,721,491 research outputs found
On the direct use of hydroclimatic information to better inform water reservoir operation
No full textIntegrating Inverse Reinforcement Learning and Direct Policy Search for Modeling Multipurpose Water Reservoir Systems
Full text linkSystem identification and optimal control have always contributed to water resources systems planning and management. Although water control problems are commonly formulated as multi-objective Markov Decision Processes, accurately modeling reservoir systems controlled by human operators remains challenging due to the absence of a formal definition of the objective function guiding their behavior. In this letter, we introduce a mixed Reinforcement Learning approach to model the dynamics of multipurpose reservoir systems. Specifically, our method first uses Inverse Reinforcement Learning to extract the tradeoff among competing objectives from historical observations of the reservoir system dynamics. The identified objective function is then used in the formulation of an optimal control problem returning a closed-loop policy which allows the simulation of the observed dynamics of the reservoir system. We demonstrate the potential of the proposed method in a real-world application involving the multipurpose regulation of Lake Como in northern Italy. Results show that our approach effectively infers the tradeoff between flood control and water supply adopted in the observed system's operation, and yields a control policy that closely approximates the observed system dynamics
Joint design of floating solar plant and dam operating policies for waterborne epidemics control: an assessment on the Kariba dam
No full textExploring the Climate Puzzle: A Surprising Twist in Fighting Climate Change
Full text linkSometimes, when scientists try to help people, they can end up with a surprise ending in which things do not work out as expected. Their “help” might even accidentally make the situation worse for some people. We wanted to know if this could be true for a strategy to slow down climate change: charging countries a fee when they cut down forests to create farmland. We used computers to predict what might happen if countries were charged different fees, to keep things fair. Specifically, countries with less money would only have to pay low (or no) fees, while rich countries would pay higher fees. However, our computer model showed that this plan could have unexpected negative consequences for water availability in some places that pay low fees, like certain regions in Africa. This tells us that, as we fight climate change, we must keep our eyes open for unintended consequences that could result from our attempts to help the planet
Improved reservoir operation by direct use of hydro-meteorological information
No full textIt is generally agreed that more information translates into better decisions. For instance, the availability of inflow predictions can improve reservoir operation; soil moisture data can be exploited to increase irrigation efficiency; etc. However, beyond this general statement, many theoretical and practical questions remain open. Provided that not all information sources are equally relevant, how does their value depend on the physical features of the water system and on the purposes of the system operation? What is the minimum lead time needed for anticipatory man- agement to be effective? Is the data-predictions-decision paradigm truly effective or would it be better to directly use hydroclimatic data to take optimal decisions, skipping the intermediate step of hydrological forecasting? In this work we investigate these issues by application to the management of a complex water system in Northern Viet- nam, characterized by multiple, conflicting objectives including hydropower production, flood control and water supply. First, we quantify the value of hydroclimatic information as the improvement in the system performances that could be attained under the (ideal) assumption of perfect knowledge of all future meteorological and hydro- logical input. Then, we assess and compare the relevance of different candidate information (meteorological or hydrological observations; ground or remote data; etc.) for the purpose of system operation by novel Input Vari- able Selection techniques. Finally, we evaluate the performance improvement made possible by the direct use of such information in re-designing the system operation
A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run
No full textThe operation of large-scale water resources systems often involves several conflicting and noncommensurable objectives. The full characterization of tradeoffs among them is a necessary step to inform and support decisions in the absence of a unique optimal solution. In this context, the common approach is to consider many single objective problems, resulting from different combinations of the original problem objectives, each one solved using standard optimization methods based on mathematical programming. This scalarization process is computationally very demanding as it requires one optimization run for each trade-off and often results in very sparse and poorly informative representations of the Pareto frontier. More recently, bio-inspired methods have been applied to compute an approximation of the Pareto frontier in one single run. These methods allow to acceptably cover the full extent of the Pareto frontier with a reasonable computational effort. Yet, the quality of the policy obtained might be strongly dependent on the algorithm tuning and preconditioning. In this paper we propose a novel multiobjective Reinforcement Learning algorithm that combines the advantages of the above two approaches and alleviates some of their drawbacks. The proposed algorithm is an extension of fitted Q-iteration (FQI) that enables to learn the operating policies for all the linear combinations of preferences (weights) assigned to the objectives in a single training process. The key idea of multiobjective FQI (MOFQI) is to enlarge the continuous approximation of the value function, that is performed by single objective FQI over the state-decision space, also to the weight space. The approach is demonstrated on a real-world case study concerning the optimal operation of the HoaBinh reservoir on the Da river, Vietnam. MOFQI is compared with the reiterated use of FQI and a multiobjective parameterization-simulation-optimization (MOPSO) approach. Results show that MOFQI provides a continuous approximation of the Pareto front with comparable accuracy as the reiterated use of FQI. MOFQI outperforms MOPSO when no a priori knowledge on the operating policy shape is available, while produces slightly less accurate solutions when MOPSO can exploit such knowledge
Assessing the value of cooperation and information exchange in large water re- sources systems by multi-agent optimization: the Zambezi River case.
No full textFlexible and adaptive water systems operations through more informed and dynamic decisions
No full textTimely adapting the operations of water systems to be resilient against rapid changes in both hydroclimatic and socioeconomic forcing is generally recommended as a part of planning and managing water resources under uncertain futures. A great opportunity to make the operations more flexible and adaptive is offered by the unprecedented amount of information that is becoming available to water system operators, providing a wide range of data at increasingly higher temporal and spatial resolution. Yet, many water systems are still operated using very simple information systems, typically based on basic statistical analysis and the operator’s experience. In this work, we discuss the potential offered by incorporating improved information to enhance water systems operation and increase their ability of adapting to different external conditions and resolving potential conflicts across sectors. In particular, we focus on the use of different variables associated to different dynamics of the system (slow and fast) diversely impacting the operating objectives on the short-, medium- and long-term. The multi-purpose operations of the Hoa Binh reservoir in the Red River Basin (Vietnam) is used to demonstrate our approach. Numerical results show that our procedure is able to automatically select the most valuable information for improving the Hoa Binh operations and mitigating the conflict between short-term objectives, i.e. hydropower production and flood control. Moreover, we also successfully identify low-frequency climate information associated to El-Nino Southern Oscillation for improving the performance in terms of long-term objectives, i.e. water supply. Finally, we assess the value of better informing operational decisions for adapting the system operations to changing conditions by considering different climate change projections
- …
