NSF Grant 2323189

IMR: MT: AirScope: A Flexible and Programmable Platform for End-to-End Mobile Network Measurements in Rural Environments Using UAVs.

List of personnel

  • Principal Investigator: Morteza Hashemi
  • Co-Principal Investigators: Shawn Keshmiri and Victor Frost

Overview and Goals

Motivation. The U.S. Federal Communications Commission (FCC) has introduced a procedure allowing cellular users to demonstrate that their carrier’s claims on service quality are inaccurate and that they do not have sufficient cellular service in a particular location. This demonstrates that significant differences exist between the FCC’s data, the claims made by cellular network providers, and users’ actual experiences.

In rural areas, the situation worsens since accurate and at-scale cellular measurements in rural areas are challenging due to various factors. (i) Rural areas have a more dispersed population, which makes it more difficult and expensive for mobile network providers to deploy and maintain network infrastructure, leading to variations and non-uniformity in network performance. (ii) Many mobile network performance tests rely on crowdsourced data to measure network speeds and quality. However, in rural areas, there are fewer people with access to these types of apps, and so the data collected are less representative of the actual network performance in those areas. (iii) The terrain and physical environment in rural areas (e.g., large agricultural fields, hills, etc.) along with large measurement areas make it more challenging for at-scale cellular measurements using the existing manual tools.

Despite the above challenges (e.g., data accuracy, sample size, large measurement areas), there is a pressing need to develop mechanisms for collecting reliable and comprehensive measurement data at scale in rural areas, particularly with the emergence of novel use cases such as precision Ag, remote work, tele-health, online education, etc. This is a technology policy issue that has implications for (i) policymakers and regulators to improve broadband access, quality, and fairness in rural areas, (ii) cellular service providers for network planning and deployment, (iii) researchers to develop connectivity solutions that are tailored for rural areas, and (iv) end-users, such as farmers and schoolchildren who rely on cellular networks for their operations and education.

Using unmanned aerial vehicles (UAVs) to perform programmable and automated data collection will open new possibilities to perform large-scale measurements more efficiently. The overarching goal of this project is to design, build, and publicly release a versatile UAV-based measurement suite (“AirScope”) for radio access network (RAN) and end-to-end network measurements in rural areas.

Goals. There are three major goals in this project: 

Goal 1: Developing a Versatile UAV Platform with Cellular Capabilities. In this project, we aim to design and build several vertical take-off and landing (VTOL) UAVs and integrate the LTE/5G modem with the flight controller and the UAV’s on-board processor, such that command and control (C2), telemetry, and payload data are communicated through commercial cellular networks. 

Goal 2: Measurement-Aware Algorithmic Development. As the second goal, we aim to design and implement a comprehensive database system that logs any pertinent internal and external measurement data (RAN, end-to-end metrics, GPS data, weather conditions, and UAV telemetry). To enhance the efficiency of the data collection tool, we develop a novel algorithm using Kolmogorov-Arnold representation theorem to estimate the spatio-temporal measurement map based on sparse observation points. We will develop the novel concept of measurement-aware trajectory design and refinement.

Goal 3: Extensive Evaluation Campaign and Large Cellular Dataset Curation. Finally, we test, validate, and refine our tool under a rich set of scenarios in rural areas. Our effort will result in an unprecedented dataset of cellular measurement data that can be used for ML/AI-based and data-driven research projects.

Key Accomplishments

An overall system model for the AirScope measurement platform is shown in the above figure, in which the UAV communicates with the commercial LTE cell tower, and is controlled with a remote automatic flight controller. Measurements are taken between the UAV and AirScope server. Through flight tests at different altitudes (from the ground level up to 400 ft), various RAN and end-to-end performance metrics are collected. Our in-house KU Automatic Flight System (AFS) 5.0, is designed for advanced aerospace applications and uses the powerful, energy-efficient NVIDIA Jetson Orin NX 16GB with a small form factor.

Our cellular measurement campaign has several phases. In the first step, we select the measurement environment and design the flight path. In the second step, the measurement will start and logging software will record time-stamped measurements on RAN metrics, end-to-end network metrics, weather conditions, and UAV flight information, along with GPS data. We will measure these performance metrics for different message types (C2, telemetry, sensory, and payload). In the third step, and upon the completion of the measurement, the pMLTE cellular module is accessed for remote data retrieval. At the end of each measurement, all on-board collected data can be easily transferred to a more permanent data storage using available services (e.g., FTP) provided by the pMLTE and Orin processor.

Major activities. We developed and deployed the proposed UAV-based measurement platform for end-to-end mobile network characterization in rural environments. We integrated a Microhard pMLTE modem, NVIDIA onboard computer, and Verizon LTE connectivity into vertical take-off and landing (VTOL) UAVs for cellular network collection. The platform was programmed to capture cellular radio access network (RAN) parameters, end-to-end latency and throughput to a remote edge server (located in the West Campus of the University of Kansas), and GPS data. We developed data logging software, post-processing tools, and visualization modules (2D/3D maps and statistical charts).

Significant Results. We have achieved the following results: (1) Successfully completed multiple UAV flight experiments capturing LTE RSRP, RSRQ, SINR, RSSI, and cell identifiers for serving and neighbor cells, (2) Collected datasets with 1-second sampling intervals that include latency (RTT) measurements to an edge server and bi-directional throughput, (3) Developed codebases for KPI extraction, statistical analysis, and visualization in geospatial formats, (4) Generated 3D maps of KPI distributions over the flight regions, which demonstrate spatial variations in coverage and network performance, (5) Released a dataset and source code package that includes flight logs, CSV data, and visualization tools. 
 


Measurement Tool 

AirScope tool consists of VTOL UAV platforms integrated with COTS cellular modems (pMLTE from Microhard), which are connected to an onboard computer (NVIDIA Jetson Orin) running data logging software. The system collects cellular network performance data during flight operations from ground level up to 130 meters altitude with GPS tracking. Data samples are recorded every 1 second and saved as CSV files. The data consists of:

  • RAN KPIs collected for serving cell (the cell that LTE service is provided from) RSRP, RSRQ, SINR, RSSI, CellID, PCI, LAC, MCC, MNC
  • RAN KPIs collected for neighbor cells (cells that the modem will connect to when serving cell service degrades) RSRP, RSRQ, RSSI, PCI
  • Latency Round-trip time (RTT) between the UAV and edge server
  • Uplink and download throughput 

Measurement datasets. Our collected dataset is available from this GitHub link. This dataset contains: 

  • Flight test log files in text file format (.txt) for:
  • LTE RAN (radio access network) parameters.
  • Latency (RTT in milli-second) between the on-board radio modem and dedicated server.
  • Bi-directional throughput of the LTE connection (Uplink and Downlink).
  • Data extracted from the log files mentioned above in CSV text file format.
  • The source code for:
    • Recording the RAN parameters.
    • Extracting the parameters values from the log files.
    • Visualizing the RAN parameters in maps (2D and 3D formats).
    • Visualize the RAN, Latency, and Throughput in Statistical charts.

Representative Results. The following are representative measurement results. More detailed results are reported in this article.