1. School funding 2020

The issue

A statement by the Conservative Party claimed that school funding had been increased by a minimum of £5,150 per pupil. A Labour Party MP, Wes Streeting, raised this with the UK Statistics Authority and said that the claim was false and misleading.

OSR’s review

We reviewed the available data on school funding and concluded quickly that this claim was a mistake. The level of funding had increased to £5,150, but the increase itself was only £150. This analysis aligned with that undertaken by others, including Full Fact. The Conservative Party also removed the claim from its published material and deleted the Tweet.

Consideration of misleadingness

This was not a case of misleading presentation or use of statistics. It was more straightforwardly wrong. It was not therefore necessary to consider misleadingness to guide our judgement.

2. Crime statistics 2019

The issue

In a speech at the launch of the Labour Party manifesto launch, the leader of the Labour Party, Jeremy Corbyn, stated that “violent crime had doubled under the Conservatives’ austerity programme.”

OSR’s review

There are two statistical sources on crime levels: crimes recorded by the police and crimes reported by the public in a survey. We have had long-standing concerns about the police recorded crime statistics and in 2014 we removed the National Statistics designation from the police recorded crime statistics because of quality concerns. Subsequent audits by HM Inspectorate of Constabulary and Fire Services demonstrated that quality of crime recording remained unreliable. Our long-standing public position has been that, for trends in most crimes, the ONS’s crime survey provides a more reliable source than the police recorded crime statistics. While the police recorded crime statistics showed a trend broadly consistent with the Labour leader’s statement, the ONS survey showed no significant change in levels of violent crime.

Consideration of misleadingness

– The nature of the issue
Although the statement could be defended on empirical grounds – there was certainly a time series that showed the trend being claimed – we considered that there was a risk that audiences could be misled by the statement, given its prominence in a prepared speech used to support a high profile political campaign. There was some media reporting of the claim too.

– Risk factors
The following risk factors are relevant:

• There is selectivity of a metric to support a claim which other related metrics do not support: Although this case is not really about cash or percentage presentations of data, the core point is that a claim was made based on the selection of a metric that other related metrics would not support: in other words, using the more reliable ONS survey would not support the same conclusion.

• The language used does not fully represent the available statistics, for example implying the statistics represent a much broader or narrower definition than appropriate: The claim did not make clear that it was drawing on a specific definition (crimes recorded by the police).

• Poor quality data is used, making it likely that the hearer will believe something which is untrue: The poorer quality of the police recorded crime data is well established.

– Materiality
We concluded that this use of statistics was material – it was designed for repeated use in political campaigning and was part of a prepared speech by a leading politician. We therefore advised the UK Statistics Authority Chair to write to the leader of the Labour Party, and the letter was published on the UK Statistics Authority website.

3.  Covid prevalence 2020

The issue

In a speech, the First Minister of Scotland, Nicola Sturgeon, claimed on July 3 2020 that the prevalence of Covid-19 infections was 5 times lower in Scotland than in England. The claim was repeated by other Scottish Ministers and in related news media.

OSR’s review

We could not easily find any publicly available source that would support this claim. When we approached the Scottish Government, they provided an explanation based on comparing estimates from two models. The model for Scotland was sourced from Scotland’s COVID-19: modelling the epidemic (issue no.6) 25 June and the England prevalence figure was sourced from modelling work done by the London School of Hygiene and Tropical Medicine, using a UK estimate as a proxy for England, but which was not readily accessible.

The Scottish Government then compared the upper prevalence rates published in Scotland’s COVID-19: modelling the epidemic (issue no.6) 25 June and the Office for National Statistics’ COVID-19 Infection Survey pilot: 25 June. This was done to corroborate the figures from the London School of Hygiene and Tropical Medicine.

Comparison of prevalence rates is not straightforward. If it is to be undertaken, the results and the uncertainties should be communicated transparently. We do not think that the sources above allow for a quantified and uncaveated comparison of the kind that was made. In future if such comparisons are made, we would expect to see sources made publicly available and a clear explanation of the limitations and associated uncertainty.

Consideration of misleadingness

– The nature of the issue
We considered this to be a material statement with the potential to influence both people’s understanding and also, potentially, their behaviour (eg willingness to travel).

– Risk factors

• There is selectivity of data points, for example from a time series, to support a claim which other data points do not support: We felt there was some element of this risk factor, in that the use of upper bound estimates were chosen, rather than a range or a central estimate. This appeared to us to be a selective approach.

• The language used does not fully represent the available statistics, for example implying the statistics represent a much broader or narrower definition than appropriate: We concluded that the language was far too confident in describing different prevalence rates between the two countries, and did not recognise sufficiently the inherent uncertainties in comparing COVID-19 prevalence rates. We would expect to see a clearer explanation of the limitations and associated uncertainty.

• No source or methodology is given, making it likely that a hearer could draw inaccurate conclusions about what the available statistics represent: This risk factor was strongly present. The absence of a clear source, and the lack of clarity on methodology, represented a significant risk factor.

• There is too much emphasis on data that are incomplete eg early results from a trial: This risk factor was also present, although perhaps less strongly than some of the other risk factors.

– Materiality
We concluded that this use of statistics was material – it was designed as part of a prepared speech by a leading politician, based on advice and analysis from the Scottish Government. We also judged that this was potentially misleading, in conveying a stronger conclusion than the available evidence would support. We therefore wrote to the Chief Statistician of the Scottish Government to outline these concerns.