Most people have used a speed test to check their internet connection’s online performance at some point. But what many people don’t know is that each test uses different processes, tools and methodologies to measure internet speeds and the overall quality of a connection. These different test processes are all valid, however the results tell the user different things even if that’s not totally apparent.
For example, CIRA’s Internet Performance Test is a user-initiated off-network, software-based, bulk transport test.
…..okay…. interesting….So, what does that mean? Read on!
On-network speed tests are conducted inside of a specific network, typically the one provided by an internet service provider (ISP). These tests measure the data transfer rates between the user’s device and the servers hosted by the ISP. On-network speed tests offer insights into the immediate performance of the user’s internet connection, providing valuable information about download and upload speeds, latency and overall network stability.
On the other hand, off-network speed tests involve assessing internet speed and performance beyond the local network. These tests are designed to evaluate how well the user’s internet connection performs when interacting with external servers, services and applications on the broader internet. Off-network speed tests consider the complexities of data transfer across various networks, including the user’s home network, ISP network, carrier network(s) and intermediary servers. They provide a more comprehensive perspective on internet speed and can help users understand how their connection performs when engaging in online activities that extend beyond the boundaries of their immediate ISP network. In other words, off-network tests reflect how most Canadians use the internet in their daily lives.
___ ___ ___
Hardware-based speed tests involve the use of a physical device designed to measure internet speed metrics. These devices, often called white boxes, connect directly to the home network router or modem, bypassing the user’s devices, such as computers, tablets and phones. Hardware-based tests can eliminate the impact on measured performance from users’ devices, software variables or traffic on the home network. These tests are commonly installed by network administrators to test at regular intervals and when the network is otherwise idle. As you’d expect, the equipment needed for hardware testing is expensive, which makes the technique unavailable to many.
Conversely, software-based tests use websites or applications that users can access and run on their computers, phones, or tablets anytime they wish. These tests measure the data transfer rates between the user’s device and remote servers that may be on their ISP network or off their ISP network (off-net v. on-net above). Software-based tests are convenient and typically easy to initiate, use and interpret the results. However, they may be influenced by the limitations of the user’s device the test is running on as well as other applications and processes running in the background.
___ ___ ___
Bulk transport tests (also called single-threaded or payload tests) involve sending a large amount of data in a single, continuous stream between the user’s device and a remote server (again off or on-net). This method simulates real-world scenarios where large files, such as high-definition videos or large software updates, are transferred. Bulk transport tests provide insights into the overall capacity and sustained throughput of an internet connection. Bulk transport tests are also particularly effective in measuring data packet loss, which can be an indicator of network configuration problems or security vulnerabilities.
Multi-threaded internet speed tests use parallel processing by utilizing many simultaneous connections to transfer data between the user’s device and the testing server (on or off the net). Multi-threaded tests are designed to assess how well a network can handle multiple simultaneous data streams, which is relevant for scenarios involving numerous small data transactions, such as web browsing or video conferencing. This approach can also provide the maximum potential speed of a connection, even if that speed is rarely achieved in real use.
Conclusion
There you have it! When running a test on your favorite internet testing platform, read the “about”, “info” or “FAQ” sections of its website—the testing process and methodology should be described there. All the different formats are valid, but will generate different numbers and results because they’re doing different things. Go ahead and run a few different tests next time you’re curious.
And while you’re at it, go ahead and test your internet on our favorite user-initiated off-network, software-based, bulk transport test: CIRA’s very own Internet Performance Test!
About Net Good by CIRA and the Internet Performance Test
Net Good by CIRA supports projects, communities and policies that make the internet better for all Canadians which includes the Internet Performance Test (IPT). The platform offers advanced and detailed diagnostic data enabling communities, researchers and decision-makers to better understand and improve internet access in Canada. Each year, CIRA proudly funds its Net Good program from the revenue generated through .CA domains and cybersecurity services.
Jeff is the Internet Performance Test (IPT) Program Manager. The IPT is the most advanced internet quality test in Canada that provides public access to the performance results. Jeff is an avid advocate for how IPT data, maps and reports can help stakeholders identify areas with limited access, improve funding decisions, evaluate the success of funded projects and do so at a high degree of geographic granularity.
