Building the World’s First O-RAN-Compliant, AI-Powered, Closed-Loop Near-RT RIC

By Dr. Rajarajan Sivaraj, Brandon Larson, Rahul Soundrarajan, Wasi Asghar, Pankaj Kenjale

In the O-RAN India plugfest 2021 hosted by Airtel, Mavenir showed a live demonstration of the world’s first O-RAN standards-compliant Near Real-Time RAN Intelligent Controller (Near-RT RIC) with an AI-powered extensible application (xApp). This xApp controls the traffic steering functionality of a 5G Radio Access Network (RAN), a key feature that is responsible for managing the connectivity and mobility of users in the network. A distinguishing feature of Mavenir’s Near-RT RIC is that the platform offers “Network Intelligence-as-a-Service (NIaaS)” to the xApps with a built-in suite of sophisticated AI/ML models, analytics framework, and cloud-native tools. NIaaS enables xApps to adopt scalable and practical ML-driven intelligence towards building programmable RAN optimization solutions for 5G and beyond.

The demo featured a full closed-loop RIC control system for the RAN. It showcased how the Traffic Steering xApp leveraged live data reports from the RAN and harnessed NIaaS from Mavenir’s Near-RT RIC platform towards building an online reinforcement learning engine that optimized the mobility control decisions for the users in the RAN. This closed-loop control is exercised over the O-RAN-defined E2 interface using standards-compliant E2AP protocol, and E2SM-KPM and RC service models.

The results speak for themselves! – Mavenir’s traffic steering xApp drastically improved the RAN performance by reducing the network overhead substantially by more than 50%, while at the same time, increasing the cell-edge capacity and user throughput by over 20%.

This blog discusses the following questions:

• Why must intelligence be an integral component of the RAN in the modern era of 5G (and beyond) networking, and how the RIC is fundamental to making the RAN smarter?
• How is the RIC with interoperable apps key to building O-RAN use-cases, such as traffic steering, with different RAN management requirements from legacy services and SON engines?
• How is Mavenir’s Near-RT RIC, architected with O-RAN compliance, your quintessential RIC platform with an inherent framework of data-driven ML intelligence (NIaaS) and a built-in suite of cloud-native APIs?
• How has Mavenir’s traffic steering xApp, deployed in the Near-RT RIC, harnessed fine-grained user-centric AI towards making customized handover optimization decisions for individual users via low-latency control loops?
• How has Mavenir’s Near-RT RIC and traffic steering xApp showcased significant performance improvements in mobility and throughput KPIs on a large network scale in O-RAN India Airtel plugfest?

Section 1: Traffic Steering

Traffic steering functionality ^[1] is responsible for connectivity and mobility management of users in a 5G network. In particular, the functionality is associated with radio resource management (RRM) decisions dealing with the selection of primary cell and secondary cells for a mobile User Equipment (UE). In the process, the RAN performs handover of the UEs to target cells. To facilitate seamless connectivity of UEs in mobility scenarios and improve their Key Performance Indicators (KPI), the RAN should take optimized RRM decisions concerning handovers.

Images are subject to copyright. Copyright statement below.

WHY THE BIG DEAL?

Now, why is this really a big deal? Isn’t handover optimization an age-old problem in cellular RAN that has been heavily discussed and widely implemented?

Well, while it is indeed true that handover optimization is a classical problem, the use-case requirements and deployment scenarios keep changing with evolving radio access technologies. More specifically, 5G focuses on three categories of use-cases, namely (i) Enhanced Mobile Broadband (eMBB) – for bandwidth-intensive HD 4K to 8K video/VR streaming, immersive AR/VR, etc., (ii) Ultra-reliable low-latency communication (uRLLC) – for intelligent transportation services, factory automation, remote telesurgery, real-time drone surveillance etc., (iii) massive IoT (mIoT) – for wearables, smart grids, etc., requiring high coverage to support network densification. Beyond 5G networks shall focus on provisioning newer use-cases like metaverse, telepresence, etc., that can transcend boundaries of human imagination to deliver an unparalleled visual and perceptual experience to UEs.

Enterprises expect mobile network operators to deliver such applications to subscribers with certain assurances and guarantees in network performance and user experience. Towards this end, mobile network operators are slicing their network resources for serving these use-cases.

TRADITIONAL MANTRAS SIMPLY DO NOT WORK

Traditional techniques meant for processing handover procedures may not meet the requirements for serving these use-cases and slices over RAN. For example, 3GPP standards for 5G (Release 16) introduce a new feature, called Dual Active Protocol Stack (DAPS) for processing handover of uRLLC UEs, which reduces their mobility interruption time (critical to their latency requirements) by tens of milliseconds (close to 0).

Likewise, traditional RRM solutions, largely based on heuristics involving signaling and load thresholds, are not primed to handle UE-centric handover decisions for serving new use cases and slices. Solutions need to be more data-driven leveraging AI/ML tools that can learn intricate inter-dependencies and complex cross-layer interactions between parameters across the RAN protocol stack due to handover decisions and quantify their impact on individual UEs. In the process, AI/ML-driven approaches can optimize handover decisions and offer customized solutions to UEs in meeting their target requirements.

Section 2: Near-RT RIC Architecture and E2 Interface

THE BASIC BUILDING BLOCKS

The RAN Intelligent Controller (RIC) and the apps (xApps/rApps) are to the RAN what the central nervous system and the brain are for the human body. The RIC, together with the apps, makes the RAN smarter.

Architected according to O-RAN standards ^[2], the RIC is a new function that controls the RRM decisions for RAN functions. Based on software-defined networking principles, the RRM operations from traditional RAN functions in vendor-proprietary hardware are split with the RIC. There are two RIC components in O-RAN-standard architecture.

One is the Near-RT RIC, typically deployed at the edge of the RAN, which is responsible for fine-grained RRM of control-plane and user-plane procedures (mostly UE-specific) pertaining to functions across the RAN protocol stack over the E2 interface. The Near-RT RIC hosts extensible applications (xApps), apart from built-in platform services, to control RRM decisions for individual RAN functionalities at near-RT granularities (10 ms – 1 sec). The other end of the E2 interface is a RAN node, such as the gNB O-CU-CP, O-CU-UP, O-DU and O-eNB, that hosts RAN functions and layers of the RAN protocol stack. These logical nodes are collectively referred to as the E2 nodes.

The other component is the Non-RT RIC (out of scope of the current blog), responsible for setting high-level declarative policies, training offline ML models, and sending enrichment information to the Near-RT RIC.

WHY IS RIC WAY BEYOND SON?

The question that keeps coming to people’s minds is, “How is the RIC different from SON?”

There are 6 key areas where RIC clearly stands out:

Thus, customized UE-level decisions at fine granularities, backed by programmable and adaptive intelligence, is the centerpiece of the RIC, which distinguishes from SON.

THE NUTS & BOLTS OF NEAR-RT RIC

Like how the human brain in the central nervous system is responsible for processing information received as impulse signals and making decisions, the xApps in the Near-RT RIC receive crucial RAN information/requests from E2 nodes, process them, and use sophisticated AI/ML, analytics and optimization solutions to control RRM decisions for RAN functions over the bi-directional E2 interface ^[3].

xApps can be from third parties, and the platform functions enable inter-operability of the xApps with the RIC to control the RAN functions over E2.  The xApps communicate with the platform functions using APIs, being standardized in O-RAN specifications.

Just as attracting and assimilating a pool of global talents from across the world are instrumental in strengthening the economic and intellectual competence of a nation, an ecosystem of xApps from third parties is instrumental in fostering innovative, open, and competitive solutions for optimizing the RAN.

They interwork with each other and interoperate with the RIC towards:

• Enhancing a mobile network operator’s services
• Enriching a mobile user’s experience
• Guaranteeing service assurances for enterprise slices

O-RAN standards also define the E2 application protocol (or E2AP) ^[4] that is responsible for procedures and the associated messages between the Near-RT RIC and the E2 nodes. The service models for the E2 interface are standardized in E2 Service Model (E2SM) ^[5] specifications that define the RRM services and the associated RAN control parameters. E2AP and E2SM specifications jointly enable the RRM split between the Near-RT RIC and E2 nodes.

Fig: E2AP and E2SM – Images are subject to copyright. Copyright statement below.

Traffic steering use-case leverages the RIC for UE-centric connectivity and mobility decisions, given their heterogeneity in terms of RAN conditions, traffic subscriptions, slice profiles, QoS/QoE demands.

Section 3: Use-case realization

We now see how the traffic steering (TS) use-case is built in the Near-RT RIC via a TS xApp to control the RRM for optimizing handover decisions of UEs, using E2AP and E2SM.  The goal of the traffic steering xApp is to handover individual UEs to the optimal target primary cells that maximize their performance KPIs.

“RAN DATA” NEEDS

Let’s focus on two E2 Service models, standardized in O-RAN, towards building the xApp. One is the E2SM-Key Performance Measurement (E2SM-KPM) specification ^[6], and the other is the E2SM-RAN Control (E2SM-RC) ^[7]. E2SM-KPM feature allows UE-level, cell-level and node-level performance measurements computed at near-RT granularities to be reported from an E2 node at near-RT periodicities. Below table shows a representative list of KPMs supported by an E2 node using E2SM-KPM.

Table: Examples of E2SM-KPM data

On the other hand, E2SM-RC feature allows the RIC to send a control action to the E2 node for fine-grained RRM at lower round-trip latencies (~2 – 5 ms), along with enabling the E2 node to request for control from the RIC and/or report crucial RAN parameters and UE state/context information.

The TS xApp in the RIC receives periodic- and event-driven RAN data from E2 nodes using E2SM-KPM/RC and uses an AI algorithm based on reinforcement learning (RL) to pro-actively generate handover control actions for the UEs to optimal target cells using E2SM-RC. In subsequent sections, we discuss how we build this closed-loop control of the RAN TS functionality in the Near-RT RIC.

CLOSED-LOOP RIC SYSTEM DESIGN

Designed in accordance with O-RAN standards ^[8], the services-based architecture of the Mavenir Near-RT RIC platform, along with our TS xApp, is discussed as follows:

DOWNLOAD DIAGRAM

The TS xApp uses evolving O-RAN standard API procedures, built by Mavenir, to interact with the Near-RT RIC platform services, which further use E2AP procedures to interface with the E2 nodes. The APIs are based on cloud-native gRPC and RIC Message Routing (or RMR – a socket interface SI-95 based low-latency transport protocol) from O-RAN Software Community. These APIs facilitate seamless integration and inter-operability of operator, vendor, and 3rd party xApps with the platform. The following platform services and APIs in Mavenir’s Near-RT RIC are relevant to TS xApp:

NETWORK INTELLIGENCE-AS-A-SERVICE (NIaaS)

Mavenir’s NIaaS framework in the RIC offers a rich set of sophisticated data analytics and AI/ML services for the xApps. Our TS xApp uses online Reinforcement Learning (RL) ^{[9, 10]} to generate control actions to handover individual UEs.

• Environment: E2 nodes
• Training host: AI/ML training services in Near-RT RIC platform
• Inference host: TS xApp
• State: UE-specific E2SM-KPM/RC reports (aggregated across E2 nodes) fetched from database
• Action space: Candidate target cells for handover and serving cell
• Periodicity: 100 ms (and event-driven L3 RRC measurements) for E2 reports.
• E2 Control-loop latency: Less than 5 ms.
• Net Reward: Improvement in UE’s throughput KPI – Cost factor
• Algorithm: A neural network variant of the Q-learning algorithm.

To prevent handovers from happening too close to each other for a given UE, the reward also accounts for a cost factor for every handover, as shown above.

The RL agent learns the throughput KPI as a function of the input features from the state information and action spaces, and updates its state-action-reward mapping policy to accurately determine the Q-estimate ^{[9, 10]} that minimizes the Bellman error. It is initially trained offline in the Non-RT RIC (details are beyond the scope of current blog) and then downloaded in the Near-RT RIC, where it continues to learn online against live E2 data. The updated RL policy from the training host is periodically deployed in the TS xApp, which generates the optimal UE-specific handover control action based on the maximum Q-estimate. The actual reward from the state transition is then fed back to the RL agent, which iteratively updates the Q-learning policy in successive steps. This continues up until convergence. These steps are followed for all UEs whose handover decisions are controlled by the xApp.

Section 4: O-RAN Plugfest Demo results

We present the results for the closed-loop control of the RAN from the Near-RT RIC by the AI-powered Traffic Steering xApp using the implementations of the E2AP, E2SM-KPM and RC features, from the O-RAN India plugfest 2021 demo held in Airtel labs ^{[13, 15]}.

DEMO SETUP

DOWNLOAD DIAGRAM

We used the open-source network simulator ns-3 ^{[11, 12]} for the RAN to integrate with the Near-RT RIC and evaluate the performance of the TS xApp, in collaboration with Northeastern University, based on their code contributions to ns-3”. ns-3 realistically simulates functionalities of a large scale of network across all layers of the RAN protocol stack for both 4G LTE as well as 5G NR RAN (supporting both FR1 sub-6 GHz and FR2 mmwave bands). ns-3 also simulates UE and traffic models and varies their load in the network. We built the E2 interface along with E2AP and E2SM-KPM/RC features in ns-3 to support interfacing with the Near-RT RIC. The ns-3 RAN simulator, the Near-RT RIC platform services and xApps were deployed as micro-services in multiple pod instances of the same Kubernetes cluster ^[14]. The following table lists the ns-3 simulation parameters.

 
TEST PROCEDURES

We successfully tested the above E2AP procedures to demonstrate the closed-loop RIC control with standards-compliance in the plugfest.

PERFORMANCE EVALUATION

Mavenir’s RL-driven TS xApp was compared against SON-based RAN handover algorithms that adaptively modify L3 RRC and time-to-trigger threshold values to optimize handover decisions. Mavenir’s TS xApp drastically improved mobility overhead KPI by reducing the number of handovers by around 50%, while simultaneously increasing the throughput KPI by over 20% for cell-edge UEs ^{[14, 15]}. Over hours of simulation run, the improvements are tabulated as follows:

Mavenir’s RL algorithm enabled the neural network to learn the reward function during the offline training as a function of the input features with high accuracy, as characterized by low Bellman error values, shown below. This offline-trained model is downloaded in the AI/ML training services of the Near-RT RIC platform and acts on live E2 data with online RL exploration/exploitation to constantly learn/update the RL policy.

The RL model becomes more stable with online training, shown below by consistently low Bellman error values, and reaches convergence. The model updates are periodically deployed in the TS xApp (inference host) to generate optimal handover control actions for UEs. Mavenir’s RL agent addresses the trade-off between mobility (handover) and throughput KPIs to minimize the network overhead while significantly improving the UE performance, especially at the cell-edges.

Conclusion

We comprehensively discussed Mavenir’s Near-RT RIC platform, our NIaaS framework, and the AI-powered traffic steering xApp with closed-loop control of the RAN, to build the traffic steering use-case yielding significant benefits in network and user performance, through optimized user-centric mobility decisions. As O-RAN standards and deployments continue to evolve and gain market traction, the RIC shall be key to use-case realization with its unique abilities in AI and RRM.

Acknowledgments

We acknowledge the key contributions towards this effort from Prof. Tommaso Melodia and his research group in Northeastern University and the Wireless Institute of Things, both in terms of research as well as the required ns-3 simulations. In particular, we acknowledge Dr. Michele Polese, Principal research scientist, and Mr. Andrea Lacava, PhD student, from Prof. Melodia’s research group.

References

1. O-RAN-WG1.Use-Cases-Detailed-Specification: “O-RAN Working Group 1 Use cases detailed specification”.
2. O-RAN.WG1.O-RAN-Architecture-Description: “O-RAN Architecture Description”.
3. O-RAN-WG3.E2GAP: “O-RAN Working Group 3, Near-Real-time RAN Intelligent Controller, E2 General Aspects and Principles”.
4. O-RAN-WG3.E2AP: “O-RAN Working Group 3, Near-Real-time RAN Intelligent Controller, E2 Application Protocol (E2AP)”.
5. O-RAN-WG3.E2SM: “O-RAN Working Group 3, Near-Real-time RAN Intelligent Controller, E2 Service Model (E2SM)”.
6. O-RAN-WG3.E2SM-KPM: “O-RAN Working Group 3, Near-Real-time RAN Intelligent Controller, E2 Service Model (E2SM) KPM”.
7. O-RAN-WG3.E2SM-RC: “O-RAN Working Group 3, Near-Real-time RAN Intelligent Controller, E2 Service Model (E2SM) RAN
   Control”.
8. O-RAN-WG3-RICARCH: “O-RAN Working Group 3, Near-Real-time RAN Intelligent Controller, Near-RT RIC Architecture”.
9. S. Russell and P. Norvig, “Artificial Intelligence – A Modern Approach”, 3rd edition, Upper Saddle River (NJ), Prentice Hall Series in Artificial Intelligence, ISBN-13: 978-0-13-604259-4, 2010.
10. R.S. Sutton and A.G. Barto, “Reinforcement Learning – An Introduction”, the MIT Press, 2018.
11. M. Mezzavilla, M. Zhang, M. Polese, R. Ford, S. Dutta, S. Rangan, M. Zorzi, ”End-to-End Simulation of 5G mmWave Networks,” in IEEE Communications Surveys & Tutorials, vol. 20, no. 3, pp. 2237-2263, 3rd quarter, 2018.
12. ns-3 module for simulating mmWave-based cellular systems, https://github.com/nyuwireless-unipd/ns3-mmwave .
13. Bharti Airtel Ltd., https://www.airtel.in/press-release/12-2021/airtel-hosts-the-india-edition-of-o-ran-alliance-global-plugFest-2021-demonstrates-the-growing-maturity-of-o-ran-5g-ecosystem , Press Release, New Delhi, India, Dec 2021.
14. Mavenir Near-RT RIC, https://oranalliance.atlassian.net/wiki/spaces/OAH/pages/2228683300/PF3+Asia+Venue+3+-+Scenario+2, O-RAN Plugfest 3, Asia Venue 3 (India), Scenario 2, Dec 2021.
15. O-RAN Plugfest 3, Asia Venue 3, India session, “India_session_O-RAN_PlugFest_2021.mp4”, Dec 2021.

—

*Copyright © Mavenir 2022. All rights Reserved.

About the Authors

Dr. Rajarajan Sivaraj

Dr. Rajarajan Sivaraj is director for RIC architecture and standards at Mavenir, where he is responsible for the standardization and productization of the Near-RT RIC, Non-RT RIC, SMO, xApps, rApps and interfaces. He holds a PhD in Computer Science from UC Davis, and has a career spanning over 12 years in cellular telecommunications with prior positions at AT&T labs, Microsoft Research, Intel labs, NEC labs, Broadcom, Uhana (VMWare), etc. He has had numerous publications and granted patents, and his scholarly works have been highly cited. 

Brandon Larson

Brandon Larson is the Senior Vice President and General Manager of Mavenir’s AI and Analytics business. He previously ran the Sales Engineering organization for America’s region where he was instrumental in many key Mavenir projects including the world’s first IMS VoWiFi, VoLTE, and RCS 5.0 service launches. Brandon has diverse experience in sales, marketing, product line management, etc. in his previous companies, Tango Networks, Nokia, Ciena. He received a Bachelors in Chemical Engineering with High Honors from Rutgers University, and is a graduate of Navy Nuclear Power School. 

Rahul Soundrarajan

As Sr. Director for RAN Analytics at Mavenir, Rahul Soundrarajan is responsible for design, development and evaluation of Machine Learning Algorithms for Near-RT RIC and Non-RT RIC. His career spanning 21 years is a rich mix of Systems Architecture & Engineering of 2/3/4G RAN and applying ML algorithms for network optimization, for which he holds several patents. His prior experiences include positions at Lucent, Alcatel-Lucent, Nokia and HCL Technologies.

Wasi Asghar

Wasi Asghar is Group VP R&D in Mavenir, managing development of NGN and AI BU’s product portfolio including Near-RT RIC, Non-RT RIC, and SMO. He holds a master’s degree in computer science and engineering from IIT Guwahati India, and has a career spanning over 20 years in telecom domain. He has expertise in developing highly scalable core network solutions using cloud-native technologies serving millions of users across the globe. 

Pankaj Kenjale

As Group VP for AI, Analytics and Data R&D, Pankaj Kenjale leads the data and AI strategy and execution at Mavenir. He is responsible for the ML development of Network AI products such as Near-RT RIC, Non-RT RIC, NWDAF and data engineering of all products. He is also responsible for the development of EdgeAI products such as IVA, and IIOT. Over his career of 17 years with stints at Mavenir, Rancore (Reliance Jio) and Reliance Communications, he led the data transformation of telecommunication networks and built huge scale data platforms to serve billions of users.