Archive July 2017

Big Data and Wireless Simulation

Big Data and Wireless Simulation

Introduction Big Data in RF Analysis


Big Data in RF Analysis

Big Data provides tools and a framework to analyze data, in fact, large amounts of data. Radio Frequency, RF, provides  amounts of information that depending n how it is modeled or created, its analysis fits many statistical models and is in general  predicted using passive filtering techniques.

The main tools for Big Data include statistical aggregation functions,  learning algorithms, and the use of tools. There are many that can be purchased but many that are free but may require certain level of software engineering.  I love Python and specially the main modules used in python are:

  • Pandas
  • SciPy
  • NumPy
  • SKLearn

and, there are many more used for the analysis and post-processing of RF captures.

Drive Test and Data Simulation

In general, many drive test tools are used to capture RF data form LTE/4G, and many other systems. As vendors, we can find Spirent, and many others, and we can capture RF from multiple base stations and map those to Lat/Long in a particular area covered by many base stations.  It’s obvious that drive test cannot cover the entire area, as  expected extrapolation and statistical models are required to complete the drive test.

In a simulator, just as in MobileCDS and other simulators, specially those in “Ray Tracing,” the simulator uses electromagnetic models to compute the RF received by an antenna.


Big Data Processing for a Massive Simulation

Unstructured data models are loaded with KML and other 3D simulation systems that include polygons and buildings that are situated on top of a google earth map or any other map vendor.  The intersection of the model with the 3D database produces the propagation model that needs massive data processing, Map-Reduce and Hadoop to handle the simulation.

HADOOP and MAP Reduce for RF Processing

The data is then stored in unstructured models with RF information, that include the Electromagnetic field, frequency, time, delay, error, and other parameters that are mapped to each Lat/Log or x,y, z coordinates in the plane being modeled.  The tools are usually written in Python and parallelization can be done in multiple hadoop nodes and processing of CSV/TXT files with all the electromagnetic data and the 3D map being rendered.


As you can see the Hadoop/GlusterFS is our choice, as we don’t see that much value for HDFS or the Hadoop Data File System are the ones that handle all the files and worker systems.  As you can tell, we are fans of GlusterFS and processing of all Hadoop cluster nodes is managed in a massive processing network of high-performance networks and 10Gb Fiber network.

Big Data models: OLTP and OLAP  Processing

The OLTP and OLAP data models definitions can be found online:

” – OLTP (On-line Transaction Processing) is characterized by a large number of short on-line transactions (INSERT, UPDATE, DELETE). The main emphasis for OLTP systems is put on very fast query processing, maintaining data integrity in multi-access environments and an effectiveness measured by number of transactions per second. In OLTP database there is detailed and current data, and schema used to store transactional databases is the entity model (usually 3NF).


– OLAP (On-line Analytical Processing) is characterized by relatively low volume of transactions. Queries are often very complex and involve aggregations. For OLAP systems a response time is an effectiveness measure. OLAP applications are widely used by Data Mining techniques. In OLAP database there is aggregated, historical data, stored in multi-dimensional schemas (usually star schema). “


We have different research areas:

  • Analysis of data for handover protocols,
  • Data mining for better antenna positioning,
  • Machine learning techniques for better PCRF polices and more



BigData Presentation – Radio Frequency / Mobile CDS

Dr. Hernnandez was invited to FAU to present at one of the MBA classes on “Big Data Analytics” and we went over the important concept and examples of MapReduce, Hadoop, Pandas, and sample on how Radio Frequency can be simulated and how Big Data is the key component to process, aggregate, and create dashboards of RF simulations over 3D KML maps loaded from Google Earth/Google Maps. This presentation also covered aspects on how the data is split and can be splitter in multiple GPUs using OpenCL as a framework.


LTE Filter Coefficients

LTE Filter Coefficients

LTE Filter Coefficients

LTE uses a linear filter to perform using a set of coefficients like in Equation 1 presented in this paper.

The LTE   quotes the following

“Once the UE is configured to do measurements, the UE starts measuring reference signals from the serving cell and any neighbors it detects. The next question is whether the UE should look at just the current measurement value, or if the recent history of measurements should be considered. LTE, like other wireless technologies, takes the approach of filtering the currently measured value with recent history. Since the UE is doing the measurement, the network conveys the filtering requirements to the UE in an RRC Connection reconfiguration message.”


  • Mn is the latest received measurement result from the physical layer;
  • Fn is the updated filtered measurement result, that is used for evaluation of reporting criteria or for measurement reporting;
  • Fn-1 is the old filtered measurement result, where F0 is set to M1 when the first measurement result from the physical layer is received; and
  • a = 1 / 2(k/4), where k is the filterCoefficent for the corresponding measurement quantity received by the quantityConfig.

The filterCoefficient is provided to the UE and the sample rate is assumed to be 200ms.

Where is this coming from?

In the following paper, different values of k, and speed are studied to determine which one is the best option to estimate the right value of handover based on previous UE Measurements.

The most interesting conclusion found in this paper is:

” Based on our analyses we can conclude that both of the L3 filters: dB and linear work but they may not have exactly the same performance when a terminal speed varies in a cell. The 3km/h case showed that there is nearly no difference between linear and dB filtering when the length of L3 filtering is such that samples used in L3 filtering are highly correlated. However, in case of higher UE speeds where log-normally distributed fading samples are no longer highly correlated over the whole L3 filtering period difference between linear and dB filtering increases. If we want L3 filtered results (e.g. CPICH RSCP or CPICH Ec/Io results) to follow the actual L1 behaviour better and we want to minimize required soft handover regions in deployments, where L3 filter is used and different terminals may be present in a cell, dB domain L3 filter should be selected. Logarithmic L3 filter also better allows to control the variation of reported absolute CPICH RSCP levels with different speed”


In summary, the use of the proper filtering technique and the right value of speed are keys to minimize required handover regions.  However, the simulation used a PathLoss equation:

PathLoss = 128.1 + 37.6 Log10(R) + LogF

In real life, PathLoss is a function of shadowing, captured by the LogF function.

Another important factor is shown when:

” Figure 3 illustrates well that when terminal speed is relatively small compared to the filter coefficient e.g. 3 km/h for k=7, linear and logarithmic L3 filters do not differ much from each other or from L1 filtered results (the green reference curve). This because the samples used in L3 filtering are highly correlated i.e. variation of different input values to the filters is not high.”

Therefore choosing the right filtering technique will properly anticipate handover with a linear- or log-based filtering.


LTE Handover Events

Based on the results of the filtering, several events are triggered in LTE.

Serving becomes better than threshold
Serving becomes worse than threshold
Neighbour becomes offset better than PCell
Neighbour becomes better than threshold
PCell becomes worse than threshold1 and neighbour becomes better than threshold2
Neighbour becomes offset better than SCell
CSI-RS resource becomes better than threshold
CSI-RS resource becomes offset better than reference CSI-RS resource
Inter RAT neighbour becomes better than threshold
PCell becomes worse than threshold1 and inter RAT neighbour becomes better than threshold2

Filtering and Prediction

Linear Prediction Coding is nothing but a linear filter or a low-pass filter.

Wikipedia says that:

Linear prediction is a mathematical operation where future values of a discrete-timesignal are estimated as a linear function of previous samples.In digital signal processing, linear prediction is often called linear predictive coding (LPC) and can thus be viewed as a subset of filter theory. In system analysis (a subfield of mathematics), linear prediction can be viewed as a part of mathematical modelling or optimization.

and Kalman Filter says that:

Kalman filtering, also known as linear quadratic estimation (LQE), is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by using Bayesian inference and estimating a joint probability distribution over the variables for each timeframe. The filter is named after Rudolf E. Kálmán, one of the primary developers of its theory.

In other words, Filtering is a predictive technique using linear equations that include a Kalman Filter with a linear quadratic estimation .


OpenAirInterface5G – Handover

OpenAirInterface5G – Handover

OpenAirInterface 5G: Handover Structures in the Source Code

The OpenAirinterface5G is an open source project that:

” The OpenAirInterface Software Alliance (OSA) is a non-profit consortium fostering a community of industrial as well as academic contributors for open source software and hardware development for the core network (EPC), access network and user equipment (EUTRAN) of 3GPP cellular networks. The Alliance sponsors the initial work of EURECOM to create OpenAirInterfaceTM towards development of 5G Cellular Stack on Commercial Off-The-Shelf (COTS) hardware. Read more…

The software can be found on GitHub at:

We analyzed the handover protocol found in this interface, and we can highlight the following messages:

For example: We can find a ReporConfigEUTR_Trigger, where the UE sends a measurement report, for example an “A3” Event as defined in the 3GPP specification.

This function, is used to measure and capture the values from all the UE_rrc_inst[…]  and reports the Measurement report data, prior to sending the report configuration message with all the signal strength values reported by the filter coefficients.

Here is key to identify the RSRP_db_filtered values, where RSRP represents

“RSRQ 3GPP DefinitionReference Signal Received Quality (RSRQ) is defined as the ratio N×RSRP/(E-UTRA carrier RSSI), where N is the number of RB’s of the E-UTRA carrier RSSI measurement bandwidth. The measurements in the numerator and denominator shall be made over the same set of resource blocks”

The Layer 3 filter of RSRP is shown here, as the filter coefficient is computed thru the formula:

All is represented in the ue_meas_filtering(….) function as shown in the code from the OpenAirInterface5G sources:


As shown here the filter uses the UE_rrc_inst[ctx_pP->module_id].filter_coeff_rsrp formula to compute the linear filter coefficient, that predicts the future state of the RSRP value and it is used for the handover decisions by the 3GPP system.

The MobilityInfo structure, includes all the paraders for example as TargetPhysCellID that is used to compute the Target ID for the target cell as identified by the X2 handover or the S1 handover protocol.

NYU Wireless Team awarded NIST grant to study mmW

NYU Wireless part of team awarded NIST grant to study mmWave for first responders

NYU wireless team (NYU)
Left to right: Coitt Kessler, Austin Fire Department Robotics Emergency Deployment Team Project Manager; Marco Mezzavilla, NYU Wireless program director for the NIST project; James Zottarelli, Austin Fire Department Robotics Emergency Deployment Team Grants Coordinator; and NYU Tandon Professor Sundeep Rangan, director of NYU Wireless.