Emerging and Innovative Technologies

Oral

397924 - Real-time control of storm water systems through Reinforcement Learning: objective formulations and controller convergence

Tuesday, June 5
8:30 AM - 10:00 AM
Location: Lakeshore C
Co-Authors: Branko Kerkez, Ann Arbor – University of Michigan

In this work, we demonstrate the control of a simulated 5 sq. mile watershed using reinforcement learning, a control approach adopted from the artificial intelligence community. Integrating sensor-actuator networks with the city’s storm water infrastructure realizes the possibility to monitor the behavior of storm water network during a storm event and enhance its performance in real time through automated system-scale control. Our approach to autonomous real-time control leverages the advancements in deep learning to observe the state of the spatially distributed storm water assets throughout a watershed (e.g. detention pond, constructed wetlands) and coordinate their discharges to achieve the desired control objective. We discuss the formulation of reward functions (i.e. heuristic objective functions) that guide the behavior of the network towards achieving the intended hydrological or water quality response (e.g. improving denitrification). Through a simulation-based analysis we estimate the correlation between temporal horizon (i.e. number of previous time steps based on which a control action is chosen) and the ability of the controller to pick optimal control action. We also discuss its dependence on the physical attributes of the storm water network and the control policy with respect to which each control actions is decided. We describe how our results are also being used to identify the potential control sites for retrofitting in Ann Arbor, Michigan.

Abhiram Mullapudi

Graduate Student
University of Michigan

Graduate student in the department of civil and environmental engineering at the University of Michigan, Ann Arbor, working on storm water systems. Current research is focused on low cost distributed sensing, development of real time visualization and prediction tools for SCADA systems and development of artificial intelligence based control algorithms for water networks.

Presentation(s):

Send Email for Abhiram Mullapudi


Assets

397924 - Real-time control of storm water systems through Reinforcement Learning: objective formulations and controller convergence



Attendees who have favorited this

Please enter your access key

The asset you are trying to access is locked. Please enter your access key to unlock.

Send Email for Real-time control of storm water systems through Reinforcement Learning: objective formulations and controller convergence