Our main broadband speed checker is fairly unique in running two sets of download tests and by showing people the resulting throughput graph for the tests you can potentially spot whether a connection is suffering from congestion. An example of a connection suffering from congestion is shown below:
With the vast number of tests that people are running we wondered if it was possible to get an idea for how a broadband provider behaves collectively. The spur for this has been the congestion problems hitting some parts of the UK on the BT Wholesale WBC network.
For those who do not know the background of our speed tester, the tbbx1 test is a simple single download using TCP port 8095 and the HTTPx6 test uses six simultaneous downloads which should saturate a connection. By running both tests you can see if settings like the TCP Receive Window (RWIN) size is too small (most operating systems now automatically tune the RWIN) or whether congestion is taking place, we generally cannot identify the source of the congestion, but by collectively analysing a big batch of tests you may be able to tell if the provider or wholesaler is having an effect. For OSX users there is an oddity where the flash tbbx1 test when using Safari will max out at around 12 Mbps – this is easy to spot as the tbbx1 line will be dead flat and investigation suggests it is a core flash issue that does not affect HTTP testing (hence why most other testers do not have the issue).
To assess the impact of congestion we are looking at the tests between 00:01 11th March 2014 and 09:00 24 March 2014 and are plotting the ratio between the HTTPx6 result and the tbbx1. Generally the closer the ratio is to one the better the result is. We have done this analysis for three providers, BT Retail, PlusNet (both who use the WBC network) and Sky who have their own full LLU network.
Beyond lots of pretty spikes, what you can see some days the ratio between the two tests is better than others. The rolling average used to plot the ratio does smooth out individual results, but this smoothing helps to reveal the peak periods on each day and identify that the evening of Friday 21st March looked to be a particularly bad evening.
While PlusNet shares the same wholesale provider there are differences in how their services operate, and this seems to be demonstrated by the shape of the plot, but the Thursday 20th onwards looks to have been different to the usual patterns. What is of interest is that the ratio between the two tests is closer with PlusNet, is their use of traffic management to help at peak times giving people a slightly better experience?
Sky has been included here to act as a control and it was surprising to see the amount of variation, particularly for an operator that has the sales tagline ‘we will never slow you down’, which really means they won’t slow you down, but what other people are doing online will have an impact on your speeds. It is this shared nature of consumer broadband that actually makes it affordable.
So what have we learnt from spending a chunk of time collating all the data? Given the congestion issues are not affecting everyone on BT, PlusNet and other providers that use BT Wholesale it is not surprising that the spikes are not totally conclusive. Our feeling is that the real use for something like this is helping to back up the comments from users who are complaining about congestion when a provider denies it totally.
What we need now is a big streaming event to take place and then see what effect it had on the UK broadband infrastructure.
UPDATE Thursday 27th March 2014
We wanted to see if there was any difference between ADSL2+ services on the BT Wholesale platform and the GEA-FTTC platform, in theory BT Wholesale congestion should affect ADSL2+ customers and GEA-FTTC users (where the provider uses BT Wholesale for backhaul), but if the congestion is widespread within the couple of miles for fibre optic cable between the cabinet and the exchange handover node it should only show up for the FTTC users.
We expected the ratios have different values, and as a general rule congestion impacts faster connections more, hence why when people post speed test results from South Korea we do not a massively high average speed.
So what do we make of the results, it is a varied bag, some days there are similar peaks and other days there are not. To pin down FTTC based congestion we need to concentrate on a smaller area e.g. one exchange or even down to one cabinet but the problem then is of course getting enough people to run the tests so you can get statistically correct results.
We will keep an eye on our speed tests and see what over clever ways we can play with the data to try and produce some information. For now what is clear is that many speed tests mask the effects of contention and can lead to people getting a pleasing answer even when normal downloads and browsing may be hit or miss.