As far back as June 2013 we were writing in Issue 2 of TMN Quarterly about a coming trend in crowd-sourced user experience testing, referring Root Metrics and Opensignal, and the ways in which they might disrupt more traditional methods such as drive testing and network data analytics.
Then in 2017 we profiled Tutela, before holding a round table discussion at MWC 2018 with Tutela, Opensignal, Terragence and P3. By 2018 we were profiling NetRadar, and its approach to embedded testing of quality of service. Some of these companies have been merged with each other (Opensignal and Tutela were both acquired by Comlinkdata and all three now trade as Opensignal), been bought by third parties (P3 changed name to Umlaut and is now within Accenture), or still stand as their own entities (NetRadar, Terragence).
The issue of benchmarking network and service quality by a variety of means is not a new one and has already been through its own consolidation period, so it’s perhaps surprising to see a company start to make a noise again about the need for more accurate, converged and thoughtful user-centric measurement of network and service performance.
That company is MedUX, a 10 year old company based in Spain, which over the last 24 months has been putting together a new approach to testing mobile, WiFI and fixed networks, in a converged way.
Rafael González, CMO of MedUX says that the company has been putting together an offering which combines network probe data with drive testing, dedicated test devices in a field setting, and crowd-sourcing via and in-app SDK.
There are several drivers for this. One is the ability of operators to carry out their own benchmarking and competitiveness analysis, to drive network investments and to give their quality and marketing teams better information. Another is the need of regulators to mandate or access more granular data than simple coverage maps derived by base station location. This is particularly important as countries or areas like the EU develop connected society, smart city and sustainability targets.
“We were a company with our own commercial robots [for example Android devices] dedicated to measurements. What we learned is that this is laser-focussed to determine particular use cases. If you want to scale it gets very tough.”
“So we needed something cheaper but with the same capabilities. What we did is merge both concepts for data collection, softwarised the robots and made them compatible to any OS, or device.”
Its SDK can be integrated with a variety of hardware platforms and OS, such as PCs, smartphones and devices. Those devices could be deployed in the field, such as on buses or car fleets (Gonzalez mentions a previous partnership with Uber). The software test agent can also be embedded in something like an operator’s customer care app, and provide quality of experience data and KPIs in a more crowd-sourced, but inevitably less controlled manner.
This, Gonzalez says, gives the company a toolkit that can fit the measurement requirements and budget of its customers, which range from regulators and governments to around 90 mobile operators in 25 countries.
“We are able to provide a multi-platform solution to measure fixed and mobile networks from everywhere, be it dedicated equipment, our own hardware, software and analytics platform or just software agents. That flexibility allows you as an operator to find the proper balance between accuracy and the data volumes that you want to collect, within a budget they can control.”
Operators can use the data to plan rollouts of specific 5G technology, or configure parameters such as bandwidth and latency, to meet specific use cases.
One differentiator MedUX claims is that it can combine measurements of Wi-Fi performance with macro mobile network and fibre.
MedUX’s involvement with a Proof-of-Concept developed with sponsorship from Telefonica, and presented as a multi-phase ™ Forum Catalyst Project, expanded the scope of the project from DPI and probe based-measurement of 5G networks, to include Wi-Fi monitoring. The project built an overall customer experience index, using measurements from the home-based Wi-Fi router.
“I would rather say that a very big percentage of ISPs are already convergent, or they are at least leveraging mobile and Wi Fi. So it is key for them to understand what the end user’s getting,” Gonzalez says.
“In the mobile or the cellular side they have much more insights. As soon as you jump into the Wi Fi area, or into the fixed network side, you lose visibility because deep packet inspection is not compatible or not scalable for fixed networks, and Wi-Fi is not so measurable as mobile.”
There’s also a grander ambition. The company opened up its MedUX Impact Programme in an effort to “give back” to the industry – opening and distributing reports and open data to contribute to the discussions in the industry in terms of connectivity.
“As part of our Impact Programme what we are doing is helping to support governments, regulators and ISPs to make sure levels of connectivity are as high as possible, and to understand the details of the technical that might impact the day-to-day experience of European end users.”
Gonzalez says that coverage targets are a “very basic way of measuring whether connectivity is suitable for transforming a society”.
From a practical perspective, just saying there is signal or not, or a base station close by that might deliver service, is not saying that we have the high quality connectivity we need. We need to be more ambitious – not only saying we need 5G or that there’s fibre 100m away from a house and calling it a home pass.
“When we look into the quality that is being delivered, the service level, it’s totally different when you access it in a particular area of the city, in a suburb, or more rural and remote areas. The types and methods of 5G deployment, for example, are totally different, obviously because those areas are not so profitable or not, and have no such return on investment for the operators”
As an example, Gonzalez asks if Non Standalone 5G in 20MHz of spectrum is really going to be suitable for many use cases that can create new business models. Or he notes that even where there is good mid-band coverage at 3.5 GHz, indoor quality might still be poor.
“We believe that this is a measurement gap that we need to face.”
So yes, finding a blended way to discover a user-centric view of customer experience is not a new quest. But what might be new is the criticality to understand that experience to Government plans for sustainability and transformation, not to mention telcos’ return on their own investments.
