In a recent blog post, NSN released results of tests its Smart Labs team conducted on the performance of different VoIP clients.
The tests found that devices with the VoLTE client showed, on average, 40% better battery performance, generated up to ten times less data, required 20-40% less data throughput and generated 100-200% less signalling load.
Overall, the quality scores (MOS) between VoLTE and the OTT apps were about the same, and in some cases the VoIP apps actually scored higher, although VoLTE had significantly lower “ear to mouth” time (about 94% lower) and the OTT apps required much more resources to achieve the same quality as VoLTE.
So why the disparity, and what does this mean in terms of how operators can view VoLTE both in terms of user experience and its impact on their network?
First off, were the results gamed towards VoLTE in some way? Not at all, said Gerald Reddig, Portfolio Marketing Manager for NSN Mobile Broadband, which includes the Smart Labs unit which carried out the tests. In fact, Reddig said, if anything the results were helpful to the OTT clients, as they were conducted on good radio conditions, in an uncontested cell environment. A congested cell would be expected to make things even harder for the OTT clients, Reddig added, given QoS support within VoLTE for adaptation to radio conditions. Further results of tests conducted in congested scenarios will be released later this year, Reddig said.
Methodology
Devices were tested in the lab on a simulated network in stand-by mode, with display off and the VoIP app running in background, and also in busy mode, with calls of differing durations and speech simulated by transmitting various audio samples. Both originating and received calls were tested, and one “unanswered” call was included. So for instance over a 20 minute period a test profile might be, 1 minute originating call, one minute break, two minutes receive a call, 2 minute break, 4 minute call, 6 minute break, one non-answered call, and so on.
Measured KPIs for the tests were Current Consumption (which impacts on battery lifetime), radio and packet core signal load (which impact on control plane dimensions), the number of data sessions, number of SIP sessions, data volume, and MOS and delay measurements (which effectively measure user experience).
(Note: NSN did not want to make public the performance metrics of specific clients, but among the clients it tested were major VoIP apps, as well as non-native and native VoLTE clients.)
Results
So what was causing the disparity in performance? One thing to keep in mind is that some VoIP clients delivered superior User Experience KPIs (better call quality) – it’s just that they required much higher battery and data resources to do so. (By the way, 3G Circuit Switch calling consumed by far the least power).
Another thing to keep in mind is that not all of the OTT clients presented similar results, some were in fact much more akin to the VoLTE client in terms of data and signalling requirements, although all of the VoIP clients required more battery/power resources than native VoLTE.
Reddig’s view is that one explanation for the differentiation in battery performance is the way that signalling is not optimised within certain OTT implementations. For instance clients that are not mobile optimised have a greater number of “keep alive” signals across the network and a higher number of SIP refresh sessions, generating both a higher signalling load and battery requirement. The results showed that during a busy period, some clients were generating over 1000 RAN signalling messages, compared to fewer than 300 for the native VoLTE client.
That lack of optimisation also translated to data performance. In one case, a specific client required over 100 IP data connections compared with fewer than 20 over an equivalent session for other clients. Additionally, non-native VoLTE clients and one client using codecs not optimised for mobile were generating 4-5x the data load of the native VoLTE client.
Another aspect Reddig highlighted was the mandatory requirement for Voice Activity Detection in VoLTE. Some clients don’t use VAD at all, and some use a non-standardised profile. But VoLTE voice activation gives a much faster response than non-standardised voice activation in OTT clients. One well known VoIP client generated over 4Mb of total data (UL & DL) over a call period, compared to just over 1.1Mb for the VoLTE client.
This combination of client and codec settings meant that a native VoLTE client on a two minute call had a current requirement of 231.8mA, compared to a requirement of 328mA from one of the OTT clients.
So what?
So what of it? Aside from battery life, which is an obvious issue, does it make a difference to the users what network resources a client uses, as long as quality is OK or better than native “telephony”?
Well, one thing of note for Reddig is that operators now are considering their response to increasing competition from the likes of WhatsApp’s intended voice service launch, Facetime and KakaoTalk, with internal teams making the case to management for investment in VoLTE as an answer. Marketing teams are also grappling with what their messaging can be around VoIP.
Understanding both the user experience and the network impacts of specific clients, and native VoLTE, can inform to those investment cases.
Reddig also pointed out that even if an operator is planning to go “open garden” and transition to a super-bit pipe model (and many have no choice but to do that), it would still have to consider the QoS profiles of different clients, as well as network loads that would impinge upon control and data plane planning.
