PptxGenJS Presentation

Faculty of Information Technology Network and Computer Communications Department University of Benghazi Faculty of Information Technology Network and Computer Communications Department Thesis was Submitted in Partial Fulfillment of the Requirements for The Degree of master’s in science Reducing SDN Switch-Controller Overhead Using Off-Policy Reinforcement Learning Submitted By Nagi A Mohamed Nagem Supervised By Dr. Farag Sallabi MAR 2024.

Introduction. 1. Growing Networking Needs. 2. Limitations of Traditional Approaches.

SDN Applications and Challenges. Network Flexibility.

x MACHINE LEARNING ¯ ' — Subset of AltchnIqwwhWt we s&UeNcal momods Mable maehlnøs knprow with.

Reinforcement Learning. Agent. The entity that learns and interacts with the environment is called an agent. The agent makes decisions and takes action to achieve its goals..

RL Concepts. Value Function (V or Q). The value function estimates the expected cumulative reward that an agent can obtain from a particular state (V) or state-action pair (Q)..

Model-Based vs. Model-Free Methods. 1. MDP. 2. Off-policy and on-policy.

Research Questions. 1. TCAM Efficiency. 2. Optimization Methods.

Research Objectives. Study SDN. Study SDN, its applications, and the challenges it presents in the networking field..

Research Methodology. 1. Initial Study. Begin with an initial study to formulate key research questions aimed at improving TCAM usage in SDN switches..

Research Scope. Off-Policy RL Model. Complete Research Cycle.

Research Significance. Flow Rule Management. Addressing the challenge of managing a huge number of traffic flows and associated rules in large-scale networks..

TCAM and SDN Switches. TCAM Role in SDN. Efficiency Challenges.

Communication Overhead in SDN. Communication overhead remains a major challenge in SDN, impacting the connection between control and data planes. Innovative solutions like DevoFlow and DIFANE have developed, each proposing unique mechanisms to enhance network efficiency and reduce the overhead on SDN controllers..

Efficient management of flow entries is important for SDN performance. Dynamic timeout and eviction strategies have been proposed to optimize flow table utilization and adapt to network traffic patterns..

Aggregating flow rules is a strategic approach to managing SDN's limited TCAM capacity. Techniques like IDFA and Agg-ExTable have been developed to reduce flow entries and enhance switch performance..

Splitting and distributing flow rules across network switches is a technique to address TCAM shortages. Approaches like SA, SSP, and OFFICER aim to optimize rule allocation and network efficiency..

Machine Learning (ML) offers a revolutionary approach to SDN, providing intelligent solutions for flow table management, traffic classification, and resource optimization..

Experimental Setup. 1. Controlled Environment. 2.

Setting up the Experimental Environment. Ubuntu Setup.

Network Topology. [image] A diagram of a computer system Description automatically generated.

Data Collection in a Mininet-POX Environment. Phase Description Execution Executing experiments with TCP traffic simulation using IPERF and recording flow entries. Storage Storing data in a structured CSV file for organized analysis. Flow-Removed Event Collecting detailed statistics about network flows, focusing on the duration of each flow and frequency..

POX Controller Modifications for Overhead Measurement.

Dynamic Flow Entry Insertion and Network Performance.

Utilizing RL and Interactive Tools for SDN Flow Management.

RL Agent State Space, Action Space and Reward Function.

Off-Policy RL with DQN. Off-Policy Learning. Off-policy RL algorithms, such as Q-learning and DQN, learn the value of the optimal policy independently of the agent's actions..

Deep Q-Network (DQN) Algorithm Overview. 1. 2. 3.

RL Framework Integration. 1. Environment Setup. Configuring the Mininet network and POX controller to emulate the SDN topology and manage network traffic..

Training Procedures for the RL Agent. 1. Perception of State.

1. Controller Overhead Minimization. The Agent showcased a remarkable ability to minimize controller overheads, which is an important element of efficient network management in SDN environments..

Deep RL (DQN) Agent's Efficiency. 1. Baseline Scenario.

2. 90 Seconds Hard Timeout. Adjusting the Hard TimeOut to 90 seconds, the agent decresed overhead by 45%, showcasing its adaptability and efficiency in rule management..

150 Seconds Hard Timeout. A graph with blue dots Description automatically generated.

With a Hard TimeOut of 200 seconds, the agent achieved overhead reductions of 65%, marking significant achivements in SDN management..

Timeout Value. Packet In Messages. 30. 4822. 90. 3676.

90 Seconds Timeout. 150 Seconds Timeout. An adjustment to 150 seconds resulted in a 40% reduction in the number of flow sessions, indicating improved efficiency..

Before DQN. Before the introduction of the DQN agent, the server and client had a consistent bandwidth of approximately 263 Kbits/sec for mice flows and 25.6M for elephant flows..

Conclusion. 2. Utilization of DQNRL. Our study utilized the off-policy Deep Q-Network Reinforcement Learning algorithm (DQNRL) to automatically determine the entries that should be kept on the switch flow table..

Optimizing SDN with DQN RL. 25%. Baseline Reduction.

Future Work. Implementing DQN RL Method. This involves implementing the Deep Q-Network (DQN) RL method across a variety of known SDN topologies, incorporating multiple switches, and utilizing different SDN controllers like OpenDaylight and Ryu..

Thank You.