Broadband speed testing is a lot more than just showing people the highest speed seen during a test, even if that does give people a nice warm feeling inside and the ability to brag about their connection to their friends and facebook followers.

So we thought with the recent inclusion of our quality metric in our speed test results that we should share something of the overall picture for the various quality measures the tester records beyond the obvious download and upload speeds. Namely TCP latency, multiple download versus single download and the mysterious quality metric. The last one is only mysterious simply because if we tell the world it will be copied as have other features since we started developing the speed test many years ago.

All the charts in this blog are based on the speed test results from January 2017 and while the quality metric is something we are talking about properly for the first we have been tracking it for over a year to be sure it is reliable and consistent in what it tells us about a connection and thus the provider once you aggregate lots of them.

ADSL and ADSL2+ services and observed latency

Observed Latency from FTTC/VDSL2 Speed Tests

Latency from Speed Test Results for DOCSIS (blue is no cable broadband)

The latency for the three main technologies are interesting, and what is revealing is the difference in the ADSL/ADSL2+ and the FTTC charts. Now we know that people who had ADSL/ADSL2+ a decade ago will be saying they don’t remember broadband being that bad for gaming, but remember we are looking at TCP latency rather than ICMP ping, so it is generally higher as an absolute number but a good metric for comparison purposes.

The FTTC/VDSL2 graph is almost freaky in how good it demonstrates the difference in latency as you get further away from the speed test server cluster that we operate in London based data centers, and while cable broadband should be able to reproduce this it seems to fail to be so consistent and appears to have high latency even once just a short way from London.

While TCP latency is a proxy for ICMP ping, it is very important thing to measure in 2017 as the rise of mobile gaming and the small but frequent updates many apps require mean that the average user and not just the twitch obsessed gamers will notice a difference between a good consistent low latency connection and a bad one.

Quality metric for standard broadband connections

Quality Metric for Fibre to the Cabinet

Cable broadband quality metric

The quality metric has a scale where 1 is the ideal and if in the 1.4 to 1.6 region you are doing pretty well, so on the charts you are looking for green and yellow. Red is where we believe the broadband speed test is reflecting a poor connection, now this could be down to the Wi-Fi or poor telephone wiring (for ADSL/FTTC but not cable obviously). Unlike the pure latency plot you can see that geographic distance from London is less of a factor and even to the layman the amount of red in the cable broadband chart is clearly an issue and one only has to look at our own and other Virgin Media forums to see a lot of people complaining of poor broadband and people like gamers switching to VDSL2 finding while downloads are often slower on VDSL2 since it does not have the same maximum speed, gaming and web browsing is generally better.

The fact that the cable quality chart does have some good areas highlights that this metric is likely to be showing those postcode areas of the UK where congestion on either the local DOCSIS segment or in IP network are an issue.

ADSL/ADSL2+ congestion sensitivity

FTTC and single versus multiple thread testing

Single vs Multiple Thread Speed Tests on Cable Broadband

Not every version of our speed test runs the two download tests, i.e. one download for fixed time period and then six downloads at once for the same time period, there are actually some speed test versions that even skip the upload phase, but we still see more than enough tests to discuss the variation. A result of 1 (green) means the two download tests were identical and the red zone (a figure of less than or equal to 0.5) means the single download test was half or less than the multiple download result.

Why run two download tests? Simply put, a single download will be affected by the congestion either on the local Wi-Fi link or at some other point between the modem and our speed test server than downloading six files at once. Some testers go to the extreme and we see instances of 20 to 32 downloads at once from multiple locations to give as good a picture as possible. Our philosophy is that on a good connection and provider with decent connectivity there is no reason why a single download should not reach very high speeds, TCP RWIN may mean it takes a second to reach those speeds but it should get there. In practice the Internet and your provider is a shared medium and Ethernet/Leased line service while they cost more due to bandwidth guarantees are only good up to the point where your traffic leaves your chosen provider. The end result being that the single download will be very sensitive to problems with a connection, and the six downloads less so but still is more likely to highlight problems than many other testers, or put another way if we say you are getting good speeds it is real.

ADSL/ADSL2+ connections seem to fair pretty well, though there are some areas suggesting problems and one of those lines up with a poor area in the FTTC results i.e. LD postcode area in Wales.

Once again cable broadband comes off poorly with lots of variation across the UK on a metric that should not really vary across the UK, unless there are local issues. Fingers crossed continued investment by Virgin Media will mean their map will quickly gain more yellow areas and start to look at more green as the VDSL2 ones.

While many won’t make it this far, should just say a bit about why there is no satellite, fibre to the premises, mobile or fixed wireless charts. Basically while we have enough tests to talk about those services, once you split things down to the postcode level you don’t have enough samples to be confident, fingers crossed as more people buy fibre to the premises we will be able to talk about it later in 2017 at the postcode level.

Tags:

One Response


  1. Simon Farnsworth on 20 Feb 2017

    It would be nice to have a proper scale for the quality metric in layman’s terms. Something like:

    1 is perfect – nothing could improve.
    1.0 to 1.6 while the metric is seeing something, you as the end user will not be able to perceive the issue.
    1.6 to 2.0 will mostly show up when doing something like remote desktop or online gaming.
    2.0 to 3.0 download speeds may be affected when talking to distant servers – you will get high speeds from local servers. As the metric climbs, the distance where you’ll see issues goes down.
    Above 3.0 and only a carefully tuned host will be able to exploit the connection as sold to you.


Leave your comment