Cities generate an immense, constantly shifting stream of human decisions. In New York City alone, millions of residents and visitors choose subway routes, hail taxis, request rides, navigate traffic, and adjust their journeys in response to delays. Every movement creates data, from GPS coordinates and vehicle speeds to public-transit transfers and travel times. Now, a Binghamton University researcher is developing an artificial intelligence system designed to learn from this urban complexity and help make transportation and other city services faster, safer, and more efficient.
Yingxue Zhang, an assistant professor in Binghamton University’s School of Computing, has received a five-year, $584,649 National Science Foundation CAREER Award to investigate how offline reinforcement learning can be applied to urban environments. The prestigious award supports early-career researchers whose work has the potential to shape their fields. Zhang’s project is among the first efforts to use this particular form of machine learning to model decision-making across the spatial and temporal dimensions of a living city.
Reinforcement learning is an AI technique in which a system learns how to act by observing an environment, taking actions, and receiving feedback. A robotic vacuum, for example, can gradually learn the layout of a room by exploring it, encountering obstacles, and adjusting its behavior. Over time, these experiences allow the system to develop a “policy,” or a strategy for choosing actions under different conditions. In a city, a similar policy might determine how a taxi should search for passengers, how a traveler should select a route, or how transportation resources should respond to changing demand.
Traditional reinforcement learning, however, depends on trial and error. That approach may work for a robot operating in a controlled setting, but it is risky when applied directly to urban systems. An AI model that experiments with traffic signals, transit recommendations, or vehicle routes could cause congestion, delays, or safety problems while it learns. Offline reinforcement learning avoids this danger by training models on historical data rather than allowing them to test unproven actions in the real world.
The challenge is that urban data is not clean or uniform. GPS signals can be inaccurate, vehicles and mobile devices may report information at different frequencies, and transportation records come from many independent sources. Each person may contribute only a small amount of information, while interactions among millions of people create complex patterns. A commuter’s decision affects congestion, congestion changes travel times, and those travel times influence the decisions of other commuters. Zhang’s research will examine how AI models can extract reliable decision-making patterns from this fragmented and highly diverse data.
Urban systems are also spatial-temporal systems, meaning that location and time are inseparable. A traffic pattern on one street can affect neighboring roads, while a delay during the morning rush hour may create consequences throughout the day. This combination makes city data substantially more complicated than information collected from many conventional robotics applications. Zhang’s models will need to account for spatial correlations, such as nearby vehicles moving together, as well as temporal correlations, such as recurring congestion or sudden disruptions caused by accidents and weather.
Another major obstacle is known as distribution shift. The data used to train an AI system may describe one environment, while the model is eventually deployed in another. A system trained using transportation patterns from one city may perform poorly in a different city with unfamiliar roads, travel habits, infrastructure, or demographics. Even within the same city, behavior can change over time. Zhang plans to develop methods that make offline reinforcement-learning policies more adaptable, reliable, and useful when real-world conditions differ from the training data.
The project will include safeguards before any resulting system is used in practical urban settings. Zhang and collaborators at the University of Maryland, College Park, the University of Pittsburgh, and institutions in Hong Kong will test their methods in simulated environments. Simulations allow researchers to expose models to unusual conditions and evaluate their decisions without placing people or transportation networks at risk. The research will also explore how natural-language information can improve machine-learning systems. Descriptions of an environment, a task, or an object could give an AI model additional context that raw sensor data alone might miss.
Zhang’s work is intended to influence more than transportation technology. The project is expected to support new courses and training in offline reinforcement learning, while outreach to schools and educators could introduce younger students to artificial intelligence and machine learning. Industry collaborations may eventually lead to workshops and broader workforce-development programs. At the end of the five-year award, Zhang plans to make the project’s results open source, allowing researchers, developers, and communities to examine, reproduce, and improve the models. By transforming everyday movement into a resource for safer and more responsive urban planning, the research could help cities learn from their own inhabitants without asking the real world to serve as an uncontrolled laboratory.
Subject of Research: Offline reinforcement learning for urban decision-making, transportation systems, spatial-temporal data, and smart-city applications.
Web References:
Binghamton University faculty profile: https://www.binghamton.edu/computer-science/people/profile.html?id=yzhang42
NSF CAREER Award: https://www.nsf.gov/awardsearch/show-award/?AWD_ID=2539629
Thomas J. Watson College of Engineering and Applied Science: https://www.binghamton.edu/watson
Binghamton University School of Computing: https://www.binghamton.edu/computer-science
Image Credits: Binghamton University, State University of New York
Keywords
Machine learning, artificial intelligence, offline reinforcement learning, computer modeling, urban planning, urban studies, traffic flow, cities, transportation engineering, smart cities
