Part 6. Validate the Setup (2024)

This section describes how to run end-to-end traffic from the UE to the edge Core Network.

Start CN5G Core Network

Use the following commands to start the CN5G Core Network.

 
sudo sysctl net.ipv4.conf.all.forwarding=1sudo iptables -P FORWARD ACCEPTcd ~/oai-cn5gdocker compose up -d 

Start theCN5G Edge Application

After the CN5G is started, use the oai-ext-dn container to run IPERF.

 
docker exec -it oai-ext-dn /bin/bash 

Start Aerial cuBB on the gNB

Execute the following command to set up the cuBB container.

 
# Run on host: start a docker terminaldocker exec -it $AERIAL_CUBB_CONTAINER /bin/bash 

Follow the Aerial cuBB documentationto build and run the cuphycontroller. The following instructions are for building and setting up the environmentfor running cuphycontroller. The following commands must be run from inside the cuBB container.

 
cd /opt/nvidia/cuBBexport cuBB_SDK=$(pwd)mkdir build && cd buildcmake ..make -j 

The Aerial cuPHY configuration files are located in the /opt/nvidia/cuBB/cuPHY-CP/cuphycontroller/config directory.For ARC-OTA 1.3, the setup has been validated with cuphycontroller_P5G_FXN.yaml for the the Gigabyte server and withcuphycontroller_P5G_FXN_R750.yaml for the Dell R750 server.

Before running the cuphycontroller, edit the cuphycontroller_<xyz>.yaml configuration file to point to thecorrect MAC address of the ORU and the correct PCIe address of the FH interface on the gNB.

 
sed -i "s/ nic:.*/ nic: 0000:b5:00.0/" ${cuBB_SDK}/cuPHY-CP/cuphycontroller/config/cuphycontroller_<xyz>.yamlsed -i "s/ dst_mac_addr:.*/ dst_mac_addr: 6c:ad:ad:00:02:02/" ${cuBB_SDK}/cuPHY-CP/cuphycontroller/config/cuphycontroller_<xyz>.yaml 

When the build is done and the configuration files are updated, exit the container. Run the followingto commit the changes to a new image. The name of the image will be used later in the Docker Composeconfiguration.

 
docker commit $AERIAL_CUBB_CONTAINER cubb-build:23-4 

Creating the NVIPC Source Code Package

In the latest release, NVIPC is no longer included in the OAI repository. You must copy the source package fromthe Aerial cuBB container and add it to the OAI build files. Execute the following to create the nvipc_src.<data>.tar.gzfile.

 
docker exec -it $AERIAL_CUBB_CONTAINER ./cuPHY-CP/gt_common_libs/pack_nvipc.shdocker cp $AERIAL_CUBB_CONTAINER:/opt/nvidia/cuBB/cuPHY-CP/gt_common_libs/nvipc_src.<date>.tar.gz ~/openairinterface5g 

This will create and copy the nvipc_src.<data>.tar.gz archive that is required to build OAI L2+ for Aerial.Instructions can also be found in the Aerial FAPI.

Build gNB Docker Image

In ARC-OTA 1.3, the OAI image is built in two steps. Instructions can be found in theAerial FAPI.

Pre-build Steps for OAI L2

1. When building OAI L2 for Aerial 23-4, remove line 50 and 51 in the docker/Dockerfile.gNB.aerial.ubuntu20 file before building.(ARC-OTA 1.3. includes the OAI 2024.w15 tag and Aerial CUDA-Accelerated RAN (Layer 1) 23-4). This will be mergedto OAI develop branch in the future.

2. In the same file, add moreutils on line 79.

 
moreutils \ 

3. Remove the pinning of p7_thread in the OAI FAPI integration (nfapi/oai_integration/aerial/fapi_vnf_p5.c). To do so,remove lines 58-64. Core 8 is occupied by L1 on the Gigabyte server and on the Dell the R750 server. This will be merged to the OAIdevelop branch in the future.

   Copy

   Copied!

    
   #CPU_SET(8, &cpuset);#s = pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), &cpuset);#if (s != 0)# printf("failed to set afinity\n"); 

4. Ensure the the FAPI polling thread is run on an isolated core by modifying line 585 of the nfapi/oai_integration/aerial/fapi_nvIPCfile.

   On the Gigabyte server, use core 10:

   Copy

   Copied!

    
   stick_this_thread_to_core(10) 

   And on the Dell R750, use core 21:

   Copy

   Copied!

    
   stick_this_thread_to_core(21) 

5. There is also a memory leak in the OAI 2024.w15 tag used with ARC 1.3. This is already fixed on the develop branch of OAI.The current ARC release has been verified by applying this MRon top of the 2024.w15 OAI 2024.w15 tag.

 
cd ~/openairinterface5g/wget https://gitlab.eurecom.fr/oai/openairinterface5g/-/merge_requests/2747.patchgit apply 2747.patch 

Use the below commands to build the OAI L2 code:

 
cd ~/openairinterface5g/docker build . -f docker/Dockerfile.base.ubuntu20 --tag ran-base:latestdocker build . -f docker/Dockerfile.gNB.aerial.ubuntu20 --tag oai-gnb-aerial:latest 

This will create two images:

• ran-base:latest includes the environment to build the OAI source code. This will be used then building the oai-gnb-aerial:latest image.

• oai-gnb-aerial:latest will be used later in the Docker Compose script to create the OAI L2 container.

Running gNB with Docker Compose

ARC-OTA 1.3 includes a Docker Compose configuration for running gNB software. Refer to theOAI instructionson how to run with Docker Compose. The Docker Compose configuration is located inthe following file path:

 
~/openairinterface5g/ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yaml 

Before starting the gNB, you will need to update the docker-compose.yaml file.

1. On line 27, switch the cuBB image to the image that was built, committed, and tagged in the previous steps.

 
image: cubb-build:23-4 

1. On line 30, add sudo to the command.

 
command: bash -c "sudo rm -rf /tmp/phy.log && sudo chmod +x /opt/nvidia/cuBB/aerial_l1_entrypoint.sh && /opt/nvidia/cuBB/aerial_l1_entrypoint.sh" 

1. On line 37, if you follow the guide, change the cpu_set for the OAI container to cores “13-20” for a Gigabyte serverand cores “23,25,27,29,31,33,35,37” for a Dell R750 server.

   1. Gigabyte server:

      Copy

      Copied!

       
      cpu_set: "13-20" 

   2. Dell R750 server:

      Copy

      Copied!

       
      cpu_set: "23,25,27,29,31,33,35,37" 

2. On line 60, add a volume for the oai.log file.

1. On line 61 (after the volumes), add a command for executing the nr-softmodem. This will replace the default command that is integratedin the docker image. This new command will set the priority and create an oai.log file with time stamp.

 
command: bash -c "chrt -f 99 /opt/oai-gnb/bin/nr-softmodem -O /opt/oai-gnb/etc/gnb.conf | ts | tee /var/log/aerial/oai.log" 

After following the instructions, you should have the following images:

 
$ docker image lsREPOSITORY TAG IMAGE ID CREATED SIZEcubb-build 23-4 be7a5a94f2d3 10 seconds ago 26GBgitlab-master.nvidia.com:5005/gputelecom/container/cubb Aerial-cuBB-container-ubuntu22.04-23.04.0-Rel-23-4.256-x86_64 3a46cace77de 3 weeks ago 24.6GBoai-gnb-aerial latest 0856b9969f42 3 weeks ago 4.88GBran-base latest c5d060d23529 3 weeks ago 2.42GB 

Docker Compose will start containers running cuBB and OAI L2+. The Docker Compose script includesan entry-point script for cuBB, which you need to modify before running. The script points atthe cuphycontroller_xxx.yaml configuration that you want to run. This script is located inthe following file path:

 
~/openairinterface5g/ci-scripts/yaml_files/sa_gnb_aerial/aerial_l1_entrypoint.sh 

Before running Docker Compose, update the aerial_l1_entrypoint.sh file.

1. The latest ARC-OTA release does not have to build gdrcopy.

 
# cd "$cuBB_Path"/cuPHY-CP/external/gdrcopy/ || exit 1# ./insmod.sh 

1. In the latest ARC-OTA release, you are not root by default. Add sudo to the following, including P5G_FXN_R750if you are using a Dell R750 or P5G_FXN if you are using .

 
sudo -E nvidia-cuda-mps-control-dsudo -E echo start_server -uid 0 | sudo -E nvidia-cuda-mps-controlsudo -E "$cuBB_Path"/build/cuPHY-CP/cuphycontroller/examples/cuphycontroller_scf P5G_FXN_R750 

Before running the Docker Compose script, you also need to add root for some commands that are run before starting Aerial cuPHY Layer 1.To do so, edit the ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yaml file.

 
command: bash -c " sudo rm -rf /tmp/phy.log && sudo chmod +x /opt/nvidia/cuBB/aerial_l1_entrypoint.sh && /opt/nvidia/cuBB/aerial_l1_entrypoint.sh" 

Also, OAI L2 must point to the correct configuration by editing the following row in the ci-scripts/yaml_files/sa_gnb_aerial/docker-compose.yamlfile. All L2 configurations can be found in targets/PROJECTS/GENERIC-NR-5GC/CONF.

 
- ../../conf_files/gnb-vnf.sa.band78.273prb.aerial.conf:/opt/oai-gnb/etc/gnb.conf 

You can now run ARC-OTA with the below command.

 
docker compose up -d// console of cuBBdocker logs -f nv-cubb// console of oaidocker logs -f oai-gnb-aerial // tail -f /var/log/aerial/oai.log 

CUE Connecting to 5G Network

Take the CUE out of airplane mode to start attaching the UE to thenetwork. Make sure that the CUE is in airplane mode before starting OAI L2 stack.

Observe 5G Connect Status

Refer to the Preamble log in the cuphycontroller console output.

Check the Core Network log or CUE log to see whether NAS authentication andPDU session succeeded.

Running E2E IPERF Traffic

Start ping, iperf, or other network app tests after the PDU session connects successfully.

You can install and run the “Magic IPerf” Android application on the CUE for this purpose.

Ping Test

Ping the UE from the CN:

 
docker exec -it oai-ext-dn ping 12.1.1.2 

Ping from the UE to the CN:

 
ping -I 12.1.1.2 192.168.70.135 

Iperf Downlink Test

Perform Iperf downlink test on the UE Side:

 
iperf3 -s 

Perform Iperf downlink test on the CN5G Side:

 
# Test UDP DLdocker exec -it oai-ext-dn iperf3 -u -P 20 -b 25M -t 60 -c 12.1.1.2#Test UDP bidirectionaldocker exec -it oai-ext-dn iperf3 -u --bidir -P 10 -b 50M -t 60 -c 12.1.1.2# Test TCP DLdocker exec -it oai-ext-dn iperf3 -P 4 -b 100M -t 60 -c 12.1.1.2#Test TCP bidirectionaldocker exec -it oai-ext-dn iperf3 --bidir -P 4 -b 100M -t 60 -c 12.1.1.2 

IPERF Uplink Test

Perform Iperf uplink test on the UE Side:

 
iperf3 -s 

Perform Iperf uplink test on the CN5G Side:

 
#UDPdocker exec -it oai-ext-dn iperf3 -u -R -b 120M -t 60 -c 12.1.1.2#TCPdocker exec -it oai-ext-dn iperf3 -R -b 120M -t 60 -c 12.1.1.2 

To stop the containers, use the following commands:

 
docker stop $OAI_GNB_CONTAINERdocker rm $OAI_GNB_CONTAINER