Emerging and Innovative Technologies
397924 - Real-time control of storm water systems through Reinforcement Learning: objective formulations and controller convergence
Tuesday, June 5
8:30 AM - 10:00 AM
Location: Lakeshore C
Branko Kerkez, Ann Arbor – University of Michigan
In this work, we demonstrate the control of a simulated 5 sq. mile watershed using reinforcement learning, a control approach adopted from the artificial intelligence community. Integrating sensor-actuator networks with the city’s storm water infrastructure realizes the possibility to monitor the behavior of storm water network during a storm event and enhance its performance in real time through automated system-scale control. Our approach to autonomous real-time control leverages the advancements in deep learning to observe the state of the spatially distributed storm water assets throughout a watershed (e.g. detention pond, constructed wetlands) and coordinate their discharges to achieve the desired control objective. We discuss the formulation of reward functions (i.e. heuristic objective functions) that guide the behavior of the network towards achieving the intended hydrological or water quality response (e.g. improving denitrification). Through a simulation-based analysis we estimate the correlation between temporal horizon (i.e. number of previous time steps based on which a control action is chosen) and the ability of the controller to pick optimal control action. We also discuss its dependence on the physical attributes of the storm water network and the control policy with respect to which each control actions is decided. We describe how our results are also being used to identify the potential control sites for retrofitting in Ann Arbor, Michigan.