SamKnows is now part of Cisco | Learn more

Why are regulators encouraging ISPs to embed measurement agents in their customers' routers?

SamKnows has been embedding our extensive QoS and QoE test suite directly into ISP customers’ routers for many years – It’s a cost effective way to measure entire subscribed populations at scale. Over the last year or two, government regulators have been requesting ISPs to share the embedded data they collect to include in government reports. Why are government regulators encouraging ISPs to embed and what are they doing with the data?

More detailed and more accurate data for consumers

All government regulators want to ensure that consumers receive appropriate service information to allow them to make objective purchasing decisions, including a more realistic speed estimate and guaranteed speed at point of sale. Traditionally this meant publishing the relative performance of different access technologies, or a service tier comparison between different ISPs.

If ISPs have measurement agents in every CPE, they then can provide a huge variety of data to their customers. This includes point of sale information for customers when they are buying products. But also, as a side benefit, it means that this data set could be used by the regulator for national reporting as well.

Embedding measurements into ISPs routers means ISPs can give consumers even more granular information, such as an accurate estimate of potential broadband performance where they live at point of sale, not just a national average.

Embedded agents help a government regulators give consumers more realistic service information

Ofcom was the first regulator to start using embedded test data in their reports, many of the ISPs chose to partner with SamKnows to collect the data. It’s because of our proven track record, and because Ofcom requires the data to be collected following the principles of the SamKnows test methodology.

What does embedding require?

The key to ISPs successfully embedding directly into their CPE is collaboration between the ISP, regulator and third party supplier such as SamKnows. ISPs who work closely with our dedicated teams at SamKnows produce the best results, as we have years of experience working with the largest ISPs in the world to efficiently embed measurements (QoE and QoS) that use the same methodology as the Whitebox. Agents can be activated and deactivated as needed, and ISPs only pay a small licence fee for their active agents.

The embedded test data doesn’t make the Whitebox panel obsolete, the regulator's existing Whitebox data is kept as a ‘control’ - which is a good way to check that the ISPs embedded data is returning similar results, and it's also useful as it's the independent data source that belongs to the regulator, so they have the freedom to use the data as they wish.

Benefits of larger sample sizes

Comparing package performance

If you want to split by broadband package you need ~50 samples per package, from every ISP

If a regulator wants to split the data by broadband package, to have a statistically significant sample of data they will need ~45 samples per package, from every ISP. This allows the data to be used publicly in a report. For illustrative purposes we’ve used 3 ISPs and 3 broadband packages, but in the real world, there could be dozens of ISPs and dozens of packages. The samples required could be in their thousands..

Comparing regional performance

If you want to split by region you need ~50 samples per region, from every package, from every ISP

If a regulator wants a detailed view of regional performance to monitor the digital divide between urban and rural areas, for example, sample sizes would then become exponentially huge. In this example we’ve used 3 regions (A,B,C) but depending on the size of the country and the granularity of data required there could be hundreds of regions and require a sample count of hundreds of thousands, probably millions. In this situation, augmenting Whitebox data with embedded data from the router is the only practical solution.

If you are interested in finding out more about our embedded solution for ISPs or government regulators, get in touch below.

Request a demo