No country knows the true number of people infected with COVID-19. All we know is the infection status of those who have been tested.
The total number of people that have tested positive – the number of confirmed cases – is not the total number of people who have been infected. The true number of people infected with COVID-19 is much higher.
Whilst there is no way to infer the true number of infections from testing data, it can help give us a strong indication of the quality of a country’s data on the pandemic and an idea of how informative the number of confirmed cases in a country may be.
Testing coverage
The chart here shows a measure of testing coverage – tests per thousand people.
Countries are reporting testing data in different ways: some report the number of tests, others report the number of people tested. This distinction is important – people may be tested many times, and the number of tests a person has is likely to vary across countries.1
Across different countries, we see an enormous range in testing coverage. In Iceland there have been more than 100 tests per thousand people – far more than in any other country. In Indonesia, testing coverage is very low – only 0.1 tests per thousand people.2
Generally, we would expect that more testing means more reliable data on confirmed cases, for two reasons.
Firstly, a greater degree of testing provides us with a larger ‘sample’ of people for which their infection status is known. If everybody was tested, we would know the true number of people who are infected.
Secondly, it may be the case that countries with a high capacity for testing do not need to ration tests as much. Where the capacity for testing is low, tests may be reserved (or ‘rationed’) for particularly high-risk groups. Such rationing is one of the reasons that tested people are not representative of the wider population.
As such, where testing coverage is higher, the ‘sample’ of tested people may provide a less biased idea of the true prevalence of the virus.3
Download the data: we make our full testing dataset, alongside detailed source descriptions, available on GitHub.
The number of tests per confirmed case
A further complication with using testing coverage as an indicator of reliability, is that the number of tests needed to have an accurate picture of the spread of the virus varies over the course of an outbreak.
At the beginning of an outbreak, where the number of people infected with the virus is low, a much smaller number of tests are needed to accurately assess the spread of the virus.
As the virus infects more people, testing coverage also needs to expand in order to provide a reliable picture of the true number of infected people.
For this reason it is helpful to look at the number of tests performed for each confirmed case. This gives us an indication of the scale of testing that accounts for the different stages each country may be in its outbreak.
The bar chart shows the number of tests, or people tested per confirmed case. The data can also be viewed over time in this chart.
The key insight from this metric is that there are very large differences between countries.
In some countries the number of tests are many times higher than the number of confirmed cases. As of 11 April, in Vietnam more than 400 tests had been conducted for each confirmed case. In Taiwan and Russia there had been around a hundred tests for each confirmed case.
But in other countries testing is very low relative to the number of confirmed cases. The US, the UK and Ecuador had performed around 5 tests or fewer for every confirmed case.
Download the data: we make our full testing dataset, alongside detailed source descriptions, available on GitHub.
What can we learn from these measures about the pandemic?
Both testing coverage and the number of tests per confirmed case help us understand what we can know about the true spread of the virus from data on confirmed cases.
But it is the number of tests per confirmed case that is arguably the most helpful in this regard, because this accounts for the fact that a smaller outbreak requires less testing.
Consider for instance the difference between three countries: the UK, Australia and Taiwan.
These countries are highlighted in the chart here, which shows the number of tests per million against the number of confirmed cases per million. The dotted comparison lines show the points on the chart where the number of tests are a fixed number of times larger than the number of confirmed cases – 2, 5, 10, 20, 50, 100, 200 and 500 times larger.
In terms of testing coverage the UK appears to be ahead of Taiwan, with at least twice the number of people tested per thousand, as of 11 April.4
But on the same date, there were 60 times more confirmed cases per million in the UK than in Taiwan – 1,035 per million and 16 cases per million respectively.
So whilst testing relative to population size is higher in the UK, testing relative to the size of the outbreak is much, much higher in Taiwan.
As of 11 April, in Taiwan one case was confirmed for every 120 tests. In the UK, a case was confirmed in fewer than every four tests.5
In Australia, testing coverage is much higher than in Taiwan. But in terms of the number of tests per confirmed case, the countries are much closer – one case was confirmed for every 55 tests in Australia as of 11 April. A number of issues with the data on testing – discussed here – mean that small differences between countries should not be overinterpreted.
But the very large differences – such as those seen between Taiwan, Australia and the UK – do tell us something important about the quality of the data.
A country that performs very few tests for each case it confirms is not testing widely enough for the number of confirmed cases to paint a reliable picture of the true spread of the virus. Whilst those people with the most severe symptoms may have been tested in such countries, there are likely to be many times more people with mild or no symptoms that were never tested.
Testing in the UK has not kept pace with the advancing outbreak. The number of people tested per confirmed case fell rapidly throughout March and early April – from more than 400, to less than 4. The current low level of testing, relative to the size of the outbreak, suggests that the true number of infections in the UK is likely to be far higher than the number of confirmed cases.
The large number of tests for each confirmed case in Taiwan and Australia suggests that the number of confirmed cases paint a much more reliable picture of the true number of infections in these countries.
Researchers from the London School of Hygiene and Tropical Medicine – Timothy Russel, Joel Hellewell, Sam Abbott and others – reach similar conclusions about the UK and Australia via a different method.6 They estimate the degree to which countries’ confirmed cases may underestimate total symptomatic cases by applying the case fatality rate (we explain this metric in detail here) observed in large studies in China and South Korea to data on the number of COVID-19 deaths in countries around the world.
They estimate that in Australia the number of confirmed cases reflect more than three quarters of the total number of symptomatic cases in the country. For the UK, they estimate that confirmed cases represent less than one in twenty symptomatic cases.7
As such, the gap between the UK and Australia in terms of the true number of infections is likely to be far higher than that indicated by their confirmed cases.
The intuition behind these researchers’ estimates is that where the number of confirmed cases looks low against the number of deaths, this is a clear indication that the true number of cases is likely to be much, much higher. But the fundamental reason for this is the limited extent of testing.
The rate of tests per case thus gives us another useful way of approaching the same question, by looking at the extent of testing relative to the number of cases directly.
