Platform Schematic

Our measurement platform is used by the world’s biggest ISPs and supports many millions of devices in a single logical database. The data is protected by world-class security provided by Google BigQuery. You can choose where your data is stored to meet compliance and regulatory requirements. You can stream measurement data almost instantly to be alerted as soon as there is any changes.

Test targets

To gauge the quality of a user's broadband connection, the SamKnows measurement agents need to measure network performance from the user's home to a measurement target on the internet, and vice versa. SamKnows maintains a variety of high-capacity test servers across the globe for precisely this purpose, and ISPs are able to deploy their own test servers within their network for use as additional test targets. When a Test is initiated the SamKnows backend will provide a list of candidate Test Servers to the Test Agent, and the Test Agent will determine the best server to use via a short latency check to each.

Test servers

Test servers can be deployed “off-net”, which means outside of an ISP’s network, or “on-net”, which means inside an ISP’s network. No measurement data is stored on the test servers; they simply act as endpoints to generate and receive traffic.

Real applications and content providers

The agents also perform measurements against real applications and content providers. These include services for video streaming, gaming, CDNs, websites, social media, voice over IP and video conferencing services.

We can provide a dedicated test server in any location, this maps shows where are existing servers are located.

Test agents

The agents perform the measurements and carry out data collection. Agents may be provided in both SDK form and as a standalone product (e.g. the Whitebox).

RealSpeed

A web app that works in conjunction with our Router Agent or Whitebox to show a side by side comparison of broadband performance inside and outside the home.

Router Agent

Can be downloaded via a firmware upgrade to your customers existing CPE. This is cost-effective for service providers to deploy test agents across their network at scale.

Mobile SDK

A customisable app for iOS and Android that provides a reliable measurement tool for volunteers and crowdsources huge amounts of performance data for regulators.

Whitebox

A purpose-built hardware measurement agent manufactured by SamKnows, capable of measuring fixed-line broadband connections of up to 1Gbps.

Our test agents can be integrated with any broadband router, smartphone app, or any device with a browser. Our standalone hardware device, the Whitebox, is also available for smaller testing studies.

Data pipeline

A key component of any platform that deals with large amounts of data is its data pipeline. We have invested significantly in ours over the years, ensuring that it’s robust and can scale to the ever increasing demands of our customers. Our latest iteration of the pipeline adds speed as a requirement. This latest pipeline allows us to deliver measurement results almost instantly to our customers and provides a firm grounding for future products that will act upon real-time data.

Flume

Flume acts as our gateway for incoming data. It receives data over HTTP (over TLS), validates and authenticates it, and then publishes it on a Kafka topic.

Kafka

Kafka is an event-streaming data store. We use it to stream realtime measurement data and perform some transformations on it (such as splicing in metadata).

BigQuery

BigQuery is the data store that SamKnows uses for long-term storage of measurement data and metadata. This provides very high scalability.

Google Datacenters

SamKnows uses GCP (Google Cloud Platform) to host its core data pipeline. Google’s cloud provides the resilience and failover automatically.

We’ve re-architected the entire pipeline to remove batch processing; we are now streaming data. But we still retain the ability to buffer data at any point in the pipeline if connectivity to the subsequent part is temporarily lost.

SamKnows services

These are some of our propriety tools and technology used to control and access the measurement platform and data.

MySQL

MySQL is a relational database we use to store user data, Whitebox data, CPE data, app data and metadata.

SamKnows One

Provides users with an interface to access the measurement results and manage their devices.

Test Agent Management

View and edit Agents and Metadata. This also allows you to import and export Metadata for Agents.

Triggered Testing Controller

Gateway between the Agent (Whitebox or Router Agent) and the Instant Test / RealSpeed functionality.

Scheduled Test

The ability for users to set a test schedule in SamKnows One for tests to run on an Enabled Router against a pre-defined schedule. 

The cornerstone of SamKnows services is SamKnows One, our cloud-based analytics system. Here, you can visualise your data and manage your measurement platform.

Programmatic interfaces 

We have a powerful set of APIs that let you interact with the SamKnows One backend programmatically.

Metadata API

The Metadata API allows you to attach supporting metadata to Whiteboxes or CPE. This includes items such as ISP, package, or service tier.

Data API

Raw and aggregated measurement results intended for clients who wish to integrate our data into their internal platforms or backend systems.

Instant Test API

The Instant Test API allows tests to be executed remotely in realtime.

Agent Activation API

The Agent Activation API allows you to activate or deactivate a CPE for testing.

It’s important to assigning custom metadata to your CPE or Whiteboxes, this allows you to split and filter your aggregated data for complex analysis. Our Metadata API makes this process quick and efficient.