The recent release of our postcode dataset detailing take-up, by technology and a selected set of ISP/operators, is now available to Point Topic thinkpoint subscribers.
This model offers a benchmark and does not include local level (postcode) real world input on ISP subscriber numbers. It is representative of the outcome if most of the variables are equal. For example it does not take account of particular successes or failures in particular areas due to hitting a marketing sweet spot or well established, foundational client bases that are for whatever reason resistant to churn.
If, as an ISP or operator, your own data shows you above or below our estimates it is a sign that you are over/under performing against expectations and attention should be paid to how to replicate or address those differences.
Overall outcomes
As you would expect at a national level the outputs (and inputs) are relatively straightforward. That is not to say it does not require some analysis and work. Very few ISPs and operators today publish their subscriber numbers regularly and in detail.
Vodafone reports across their European group, Sky follow a similar reporting format while others rarely break out their technology specific numbers (FTTC, FTTP, ADSL and so on). We have been reporting UK subscriber numbers by operator and technology for some time. The analysis to the end of Q3 2023 was a primary input for this model.
National level outputs echo the subscriber numbers and market shares from this report - https://subscribers-beta.point-topic.com/articles/657b380f72da5e371fec6396
Specific use cases
The data enables a number of views and use cases. We’ve selected three straightforward examples to demonstrate the insight the data can offer.
1.Significant market power (SMP) – generally set by Ofcom as 40% or more market share across a market.This is an interesting exercise rather than very significant given that no ISP reaches this level of dominance in the UK. However there are some who do so at a local level.
Perhaps surprisingly BT (BT - not EE or Plusnet) are not the ISP that achieve this mark the most frequently in Census Output Areas in our analysis. They achieve this level (in our benchmark analysis) in 37,671 COAs. This compares to 25,613 COAs for Sky, 20,479 for TalkTalk and 50,639 COAs for Virgin Media who take the crown. KCOM also does well in terms of percentage of their footprint that achieves SMP but others are working away at this base.
Virgin achieve their mark in part due to our methodology which splits their subscribers across their footprint based on the intersections they have with other operators. The upshot is that their subscriber base is spread across a smaller area than BT and so while they have fewer subscribers in total they do ‘win’ in this case as they have a smaller footprint.
The benchmark reports that Virgin have SMP in more than a third of their footprint.
TALKTALK | SKY | BT | VIRGIN | GIGACLEAR | HYPEROPTIC | KCOM | OTHER |
Any presence (COA count) |
|
|
|
|
| ||
215,214 | 211,265 | 225,805 | 145,033 | 3,530 | 9,101 | 1,854 | 191,494 |
40% or more market share (COA count) |
|
|
|
| |||
20,479 | 25,613 | 37,671 | 50,639 | 355 | 1,192 | 1,656 | 21,531 |
SMP across footprint (%) |
|
|
|
|
|
| |
9.5% | 12.1% | 16.7% | 34.9% | 10.1% | 13.1% | 89.3% | 11.2% |
2. Outline of an area – GU* (all postcodes beginning with GU)
Guildford and the surrounding area is a keenly contested set of postcodes. Demographically suitable for most ISPs it has coverage from a range of operators and technologies.
Our benchmark take-up, reporting by number of lines by technology and operator, can help to outline what opportunities exist where.
For example we estimate there are 12.3k ADSL subscriptions/lines in the GU area. Almost 5,300 of those are within an altnet footprint. In theory these should be primary targets for upgrade. The model offers a view of where that is possible.
Any ADSL line present in GU* (darker red, more ADSL lines) and overlaid with altnet presence.
There are plenty of views you can generate from the data. These are the specific total lines and market share fields from the output table that we provide:
Subscriptions by ISP and tech | Market shares by ISP |
BT_FTTC | BT_MARKET_SHARE |
BT_FTTP | SKY_MARKET_SHARE |
BT_ADSL | TALKTALK_MARKET_SHARE |
SKY_FTTC | KCOM_LIGHTSTREAM_MARKET_SHARE |
SKY_FTTP | VIRGIN_MARKET_SHARE |
SKY_ADSL | GIGACLEAR_MARKET_SHARE |
TALKTALK_OPR_FTTC | HYPEROPTIC_MARKET_SHARE |
TALKTALK_OPR_FTTP | OTHER_MARKET_SHARE |
TALKTALK_OPR_ADSL |
|
TALKTALK_CF_FTTP |
|
KCOM_LIGHTSTREAM_FTTP |
|
VIRGIN_CABLE |
|
GIGACLEAR_FTTP |
|
HYPEROPTIC_FTTP |
|
OTHER |
|
‘Other’ consists of all the ISPs and altnets not specifically broken out.
See the appendix at the end for a full list of fields in this output.
3. Technology and opportunity across a footprint. We can take a single ISP, Sky for example, and break down the total lines, split by technology and benchmark the opportunity to upsell from FTTC or ADSL to FTTP across their footprint.
This map shows the proportion of Sky FTTP subscribers to total subscribers in an area (Local Authority in this case).
Theoretically if they chose to try and upsell existing customers from DSL, Gfast or FTTC to FTTP they would have most return from the lighter red areas.
This brief review highlights some of the applications for our benchmarks. We will be publishing updates every quarter this year and expanding the list of ISP/operators that we break out individually as well as incorporating new inputs, time since deployment, speed tests and so on.
These are expected outcomes. If the ISP is over or under indexing against these numbers then perhaps there’s a reason worth investigating. It also helps to highlight the extant opportunities across the UK. There is plenty of market left to play for and we help to pin down and enumerate where and what the targets are.
Business market
We also model the SME market. The same methodology is used to distribute the business lines across the UK postcodes and expected outcomes can be generated.
This does not include the enterprise or SOHO market. In addition numbers for the ‘business’ market are harder to come by. There is a long tail of small operators that we have little to no information about.
Results should be viewed in that context and local variations can be considerable.
SME lines market shares by Government Office Region
GOVERNMENT REGION | TALKTALK | SKY | BT | VIRGIN | GIGACLEAR | HYPEROPTIC | OTHER |
South East | 20.9% | 3.7% | 51.8% | 9.7% | 0.2% | 0.0% | 13.7% |
North West | 19.2% | 3.8% | 48.7% | 15.4% | 0.0% | 0.2% | 12.6% |
Yorkshire and The Humber | 16.6% | 4.3% | 46.6% | 13.7% | 0.0% | 0.0% | 13.5% |
West Midlands | 16.5% | 4.0% | 51.4% | 14.5% | 0.0% | 0.0% | 13.6% |
East of England | 16.0% | 6.7% | 55.9% | 11.4% | 0.1% | 0.0% | 9.9% |
East Midlands | 15.6% | 6.5% | 52.2% | 14.7% | 0.1% | 0.0% | 10.9% |
North East | 15.2% | 8.0% | 55.5% | 8.0% | 0.0% | 0.0% | 13.1% |
Scotland | 14.7% | 7.6% | 57.4% | 9.9% | 0.0% | 0.2% | 10.3% |
South West | 14.4% | 7.9% | 55.7% | 9.9% | 0.2% | 0.0% | 11.9% |
London | 13.8% | 1.5% | 45.7% | 14.6% | 0.0% | 1.8% | 22.6% |
Wales | 13.8% | 10.4% | 59.6% | 10.0% | 0.0% | 0.0% | 6.2% |
Northern Ireland | 11.6% | 6.4% | 48.4% | 22.5% | 0.0% | 0.0% | 11.1% |
The benchmark allows for more granular analysis of course but particular care should be taken when reviewing the outcomes.
Infrastructure competition
An increasingly important part of the UK story. Openreach versus CityFibre has been around for a while but they will shortly be joined by an open network of from Virgin and potentially a fourth player at scale with an altnet wholesale platform on the horizon.
Our model only includes the Openreach and CityFibre wholesale footprints in this version, these are relatively early days for this view and our assumptions are fairly basic. The ISP will resell whichever wholesale network is most cost effective (cheapest) in an area.
This does allow some views however. Here we show the CityFibre v Openreach market share for TalkTalk FTTP residential retail lines. The darker green an area the higher CityFibre’s share.
TAKE UP VERSION 3 METHODOLOGY
1. High level overview
The main goal of the project is to predict how Internet Service Providers (ISPs) will distribute their lines across the UK at the postcode level. In simpler terms, it aims to estimate the market share of each ISP in every postcode. Moreover, the project goes beyond just giving the overall market share of ISPs in a postcode; It also attempts to provide insights on the proportion of technologies used by ISPs and the different types of lines within that postcode.
This model offers a benchmark and does not include local level (postcode) real world input on ISP subscriber numbers. It is representative of the outcome if most of the variables are equal. For example it does not take account of particular successes or failures in particular areas due to hitting a marketing sweet spot or well established, foundational client bases that are for whatever reason resistant to churn.
If, as an ISP or operator, your own data shows you above or below our estimates it is a sign that you are over/under performing against expectations and attention should be paid to how to replicate or address those differences.
The latest model outperforms its predecessor by employing improved methods like linear optimization and better utilization of input data to distribute ISP lines more effectively. It overcomes previous limitations, establishing a new standard for robust and efficient allocation of lines.
2. Project scope
2.1 ISP and technologies
As of now, this project includes 8 specific ISPs along with their respective technologies, with the rest combined into one ‘Other’ operator:
BT (FTTP/FTTC/ADSL)
Sky (FTTP/FTTC/ADSL)
TalkTalk – Openreach (FTTP/FTTC/ADSL)
TalkTalk – CityFibre (FTTP)
Gigaclear (FTTP)
Hyperoptic (FTTP)
Kcom Lightstream (FTTP)
Virgin (Cable)
Other
2.2 Type of line
There are 2 types of line included in this project, which are:
Residential
Business
2.3 Postcode
We will examine all 1.7 million postcodes, which are the total number of postcodes of the UK. In this project, we use the Point Topic UPC demographic and geographic dataset with annual updates to allow the correct balance between regularity and consistency.
2.4 Frequency
This project provides the line distribution result quarterly. The result for a specific quarter will be available within 1.5 to 2 months from the last day of that quarter, depending on the reporting cycles of the major operators.
3. Predict Methodology
Note: From this point, ISP also means the ISP with its technologies, like BT_FTTC or SKY_ADSL. For example: if BT exists in a postcode, that means at least one technology (amongst 3) of BT is available at that postcode.
This version of the take up project uses 2 main data sources as the input for the predicting algorithm, which are:
Technology footprint of ISPs at postcode level
Number of business/residential lines of ISPs across the UK
To decrease the amount of computation and inference time, instead of distributing directly at postcode level, we cluster postcodes that have the same presence of ISP combinations into groups called intersections. In other words, an intersection is a group of postcodes having the same ISP’s existence in it. After calculating an estimation of line distribution results at intersection level, we will distribute down the number of lines into postcode level.
Therefore, the main computational flow of this project is:
Estimate the market share of ISPs and their technologies in each intersection
Distribute down the market share from intersection level into postcode level
In these 2 above steps, the first step is the most important and also the hardest part. The second project is just simply distributing the result of the first step into a more granular geographical level by assuming that the market share of an ISP in a postcode is similar to the market share of that ISP in the intersection which the postcode belongs to.
3.1 Distribute at intersection level
In one intersection, what we have is the number of lines of the intersection and which combination of ISPs exist in that intersection. Our objective is to find the number of lines that each ISP accounts for in that intersection (or the market share).
For example, suppose we need to distribute lines for an intersection that has 2 ISPs A and B in it. What we need to find is the number of lines of each ISP in this intersection, and we call these 2 variables that need to be found are: line_A and line_B.
We define this as an linear optimization problem, where the objective function that we want to minimize is:
In above formula:
• total line A, B: Total number of lines of ISP A, B across UK
• fill factor: factor to describe how hard to fill for an ISP for the algorithm (or the solver)
• n: exponent used to transform the fill factor to into exponential, in order to emphasize the effect of fill factor
We will examine each factor right below.
A. Total line
As we defined above, this factor is the total number of lines of ISP in the UK. This factor helps to ensure the results that the solver provides to us are reasonable in terms of the scale of the ISP. Regarding the scale of ISP, we mean that if the size (sum of lines) of ISP A is much larger than that of ISP B at a national level, then it’s likely that ISP A owns more lines when existing in the same intersection with ISP B.
For example: considering (total line A > total line B) , then if these 2 ISPs exist in the same intersection, ISP A is likely to have more lines than ISP B. And by defining this factor in the objective function, the solver attempts to make line_A larger than line_B in order to minimize the objective function.
B. Fill factor
A drawback arises when ISP A has a significantly larger line capacity (the total lines it can be potentially provided) compared to ISP B. The line capacity of an ISP is the sum of lines across all intersections where the ISP operates. Essentially, it represents the maximum number of lines an ISP could have in an ideal scenario (where the ISP captures 99.9999% of lines in all its intersections). Consequently, the algorithm encounters challenges when distributing lines for ISPs with lower line capacity, as it lacks the flexibility seen with ISPs having higher capacity.
When distributing lines for two ISPs, the algorithm tends to favor providing more lines to the ISP with more difficulty in achieving its total number of lines. To convey this information to the algorithm, we introduce a factor known as the fill factor, defined as:
The closer the fill factor of an ISP is to 1, the more challenging it is for the algorithm to fill lines for that ISP.
If this factor is less than 1, it indicates that it is impossible to fill lines for that ISP, even in the hypothetical scenario where the ISP owns all the lines in all the intersections it operates in.
If this factor is still ambiguous, we can come to a closer to real life example:
Consider two companies, A and B, each with its revenue target. Company A operates in an area where the market size is 10 times larger than A's target revenue, while company B's market size is only 3 times its target revenue. Consequently, it is much more challenging for company B to achieve its revenue target compared to company A.
In the context of our problem, the market size corresponds to the ISP's line capacity (the numerator of the fill factor formula), and the revenue target represents the total lines of the ISP (the denominator of the fill factor formula).
C. Exponent n
When certain ISPs have fill factors that are very close to 1, meaning it's extremely challenging for the algorithm to fill lines for them, we aim to amplify the impact of the fill factor. Currently, we achieve this by exponentiating the fill factors. For instance, if the fill factor of A is 20 and that of B is 1.5, considering the squared fill factors, the algorithm's inclination to make the lines of B much larger than A is approximately 400/2.25 ~ 177, which is significantly higher than the non-exponentialized ratio of 20/1.5 ~ 13.
The formula we use to make the fill factors exponential is:
In this formula:
• To guarantee logical consistency, we subtract 1 from the fill factor before applying the exponential emphasis. This ensures that when a fill factor reaches 1, the emphasized fill factor becomes 0, which is the lowest possible positive value.
• In practical terms, this means that an ISP with a fill factor of 1 will effectively claim all available lines at any intersection where it's present. However, this adjustment primarily serves logical purposes and doesn't significantly alter the overall results compared to simply using
Therefore, it can be disregarded if desired.
A challenge arises in determining the appropriate exponent value. To address this, we examine the performance of various exponents across the three most recent quarters. The exponent that yields the best average performance across these quarters, as measured by the following criteria, is selected:
• ISP line error: This metric gauges the overall discrepancy in lines between the algorithm's filling results and the actual input data, specifically focusing on differences across individual ISPs.
• Intersection line error: Similarly, this metric assesses the extent of line discrepancies between the filling results and input data, but it concentrates on variations across different intersections.
• Count below 1: This metric tracks the frequency of filling results that fall below 1
• Count equal 1: This metric monitors the number of filling results that precisely equal 1
It's crucial to minimize the occurrence of filling results that are equal to or less than 1. This is because such values can signal a tendency of the algorithm to prioritize its own convenience over accuracy when allocating lines. By carefully monitoring these metrics, we can ensure that the algorithm remains unbiased and produces filling results that closely align with the actual input data.
D. Overall objective function
From the beginning, we aim to estimate values for line_A and line_B to minimize the objective function. By combining these 2 factors (total lines & fill factor), even if total line A is significantly larger than total line B, if the fill factor of A is greater than B, the algorithm adjusts the distribution so that line_A remains larger than line_B, but not excessively so as it would be without considering the fill factor.
By introducing these 2 above factors, the algorithm's robustness is enhanced significantly, as it now takes into account not only the total lines of ISPs but also considers the popularity or fill capacity of each ISP.
3.2 Postcode level
Once we've determined the number of lines each ISP has in each intersection, we can directly calculate their market share within those intersections. This market share then seamlessly translates to the corresponding postcodes. In other words, an ISP's market share within a postcode mirrors its market share in the intersection that the postcode falls under.
To illustrate this, consider the following example: If ISP A holds a 50% market share in intersection B, it logically follows that ISP A also claims 50% of the lines within each postcode situated within intersection B.
Because we already have the number of lines of each postcode, we can effortlessly ascertain the number of lines that each ISP possesses within each postcode.
Take-up benchmark model – fields
POSTCODE |
INTERSECTION_ID |
PREMISES |
HOUSEHOLDS |
BUS_SITES_TOTAL |
POST_SECTOR |
EASTINGS |
NORTHINGS |
COA_CODE |
LSOA |
MSOA_AND_IM |
LA_NAME |
LA_CODE |
GOVERNMENT_REGION |
COUNTRY |
MDFCODE |
EXCHANGE_NAME |
BROADBAND_TECH_AVAILABLE |
LINES |
BT_FTTC |
BT_FTTP |
BT_ADSL |
SKY_FTTC |
SKY_FTTP |
SKY_ADSL |
TALKTALK_OPR_FTTC |
TALKTALK_OPR_FTTP |
TALKTALK_OPR_ADSL |
TALKTALK_CF_FTTP |
KCOM_LIGHTSTREAM_FTTP |
VIRGIN_CABLE |
GIGACLEAR_FTTP |
HYPEROPTIC_FTTP |
OTHER |
BT_MARKET_SHARE |
SKY_MARKET_SHARE |
TALKTALK_MARKET_SHARE |
KCOM_LIGHTSTREAM_MARKET_SHARE |
VIRGIN_MARKET_SHARE |
GIGACLEAR_MARKET_SHARE |
HYPEROPTIC_MARKET_SHARE |
OTHER_MARKET_SHARE |
QUARTER |
REPORTED_AT |