Quality of Experience tests
Measuring the performance of real applications over real infrastructure
Gaming
Good internet performance is particularly important for gaming. Beyond downloading your 100GB game, gamers need fast latency and low packet loss to stay competitive when gaming online. Gaming servers are spread out all over the world, but the physical distance and performance to each of them is unique to every gamer.
We’ve built tests for the following popular games, to test the round-trip latency and packet loss performance to their real-world gaming servers. More games will be added over time.
League of Legends
DOTA 2
FIFA
Fortnite
Apex Legends
Heroes of the Storm
Diablo 3
Roblox
Overwatch
Among Us
Counter Strike: Global Offensive
PlayerUnknown's Battlegrounds
Hearthstone
World of Warcraft
Tom Clancy's Rainbow Six Siege
Valorant
Rocket League
StarCraft 2
Measurements
The IP address of the node that the user was directed to.
The average, minimum and maximum round-trip times of the ten packets.
The number of packets sent and received, allowing packet loss to be calculated.
Game Stores
Before you play a game, you need to download it. Game console markets make up the majority of new game purchases and can deal with high latency and congestion, especially during the rush of a new game release. The speed of game stores depend on the Content Distribution Networks that host their game files, and ISPs will have varying connection agreements with CDNs, meaning that your performance could vary significantly depending on the ISP you use and where the CDNs are located, meaning download speed performance varies by individual homes.
We’ve built tests for the following game stores, measuring download speed and latency performance from CDN servers to real users at home.
PlayStation Network
Xbox Live
Valve's Steam
Measurements
The IP address of the node that the user was directed to
Download speed from the CDN node
Latency to the CDN node
Video conferencing
Our video conferencing tests measure round-trip latency and packet loss to the most popular online video conferencing services.
Many different protocols and communications are used during video conferencing. Our measurement focuses on traffic to the video and audio relay servers, as problems on this link will differently manifest as connectivity and quality issues to end users.
Google Meet
GoTo Meeting
Microsoft Teams
Skype
Cisco Webex
Zoom
Measurements
Average round trip latency for 10 UDP packets (by default).
Minimum, maximum, meditation and standard deviations for the latency measurements.
The number of sent and received packets.
The IP address of the endpoint.
Video streaming
We have developed a variety of video streaming measurements that stream real content from major video streaming providers. This allows you to gain an accurate view of how video is delivered to customers.
Netflix
YouTube
iPlayer
Hulu
Multicast IPTV
Measurements
The bitrate reliably streamed
The startup delay (the time taken to download two seconds of video)
The TCP connection time
The number of stalls and their duration (this is only applicable if the test is not running in the ‘bitrate reliably streamed’ mode)
The downstream throughput achieved
CDN measurements
CDNs are widely used by large websites to distribute static content near to end users. This reduces the burden on the content providers' servers and improves web page load times. CDN operators will place caches inside ISP networks so that traffic does not even need to leave the ISP network. This is beneficial for both the ISPs (it reduces the load on their transit and peering links) and the CDN operators too (it spreads the content serving burden to a wider array of servers).
The SamKnows CDN test measures performance to multiple CDNs by fetching a small object over HTTP. Care is taken to ensure that the same object (or a very similar one) is fetched from each CDN, to provide comparability between the CDNs themselves. Currently an ~80KiB JavaScript library is fetched from each CDN.
Microsoft Azure
Cloudflare
Apple
Amazon AWS
Akamai
Measurements
TCP connection time
Download speed
Object fetch time (lower is better)
The IP address of the CDN node
Web browsing
Load real pages from real websites to assess browsing performance that users would actually see in their homes rather than to a centralised test server.
Website Performance Test
This test uses a full headless web browser to accurately capture the user experience of browsing a website from within a user’s network. The measurements are based on what is observable on the screen, the goal is to highlight how much waiting time is needed for a user to be able to start accessing content on the page.
Web browsing lite
Measures the time taken to fetch the HTML and referenced resources from a page of a popular website. This test does not test against centralised testing nodes; instead it tests against real websites, allowing for content distribution networks and other performance enhancing factors to be considered.
Social Media
This social media tests measure the round-trip latency and reachability of a selection of major social media services.
A single social media site can use a variety of endpoints for different content types (e.g. audio, video) and different activities (downloading and uploading). These tests capture the round-trip latency to all of the supported combinations.
Facebook
Facebook Messenger
Instagram
Instagram Messenger
WhatsApp
Snapchat
Twitter
Measurements
Average round trip latency for 10 packets (by default).
Minimum, maximum, median and standard deviations for the latency measurements.
The number of sent and received packets.
The number of hops to the endpoint (performed using a traceroute).
The resolver IP address of the endpoint.