PowerPoint Presentation

[Virtual Presenter] Reinforcement Learning based Traffic Signal Controller.

[Audio] Concept of reinforcement learning Reinforcement Learning is the science of decision making. It is about learning the optimal behavior in an environment to obtain maximum reward. The main elements of an RL system are: The agent or the learner The environment, the agent interacts with The policy, that the agent follows to take actions The reward signal, that the agent observes upon taking actions The RL system implementation techniques: If the agent task is episodic, then use Q-learning If the agent task is Continuous, then use network models.

[Audio] Representation of traffic signal controller using RL: The intersection and the traffic represents the environment The signal controller is an agent The traffic condition at intersection is the state Green length prediction is the action taken by an agent Traffic residual is the reward.

Simulation process. Environment Replay memory Experience Exploration + Exploitation Main network Reward Weight update Action Action prediction Minibatch I/P Layer O/P Layer Hidden Layer State.

[Audio] The simulation is carried out on SUMO (Simulation of Urban MObility), a opensource software tool..

[Audio] Four different models are designed as follows for both peak and off-peak hours based on traffic data and the number of agents used. (1). M1: Actuated traffic signal controller is the base model for all the controllers, and in this model, the signals are configured based on preliminary analysis and independent of dynamic traffic demands. (2). M2: Dynamic signal planning using a reinforcement learning agent to handle all the intersecting roads. In this case, the state is represented as a count of each type of vehicle. (3). M3: Dynamic signal planning using a reinforcement learning agent to handle all the intersecting roads. Traffic volume in PCU (Passenger Car Unit) is considered as the state. (4). M4: Dynamic signal planning using a dedicated reinforcement learning agent to handle each intersecting road. Hence, multiple agents are used to plan the signal configuration in coordination with each other by considering traffic composition as a state..

[Audio] Simulation results Models' learning rate throughout the simulation and the Influence of policies on Models' learning ability are presented in this slide.

[Audio] This slide shows the average delay encountered by the vehicles and Scatter plot of traffic residual. The results demonstrates that the proposed model outperformed the other three models, that are considered in this study for comparative analysis. Thus, the reinforcement learning based traffic signal controllers are suitable to manage the traffic at signalized intersection to avoid congestions and to distribute the waiting delay..