• Overall, people welcomed the think-piece; it was valued as much as a trigger for discussion as for its content.

• One clear outcome was a recognition of the benefits of bringing together statistical, philosophical and regulatory approaches. Several people who provided feedback commented positively on this way of working.

• Having a clear statement of principles is helpful but we also need to recognise an irreducible complexity. Professor Kevin McConway of the Open University pointed out to us that it will always be difficult to produce a definitive document that describes every possible situation of misleadingness.

• A strong sentiment from the feedback was the need to distinguish production and use. The production of statistics by Government departments and ONS requires rigorous collection and presentation of data, in line with the Code of Practice for Statistics. Once statistics have been published (ie produced), they are available for use, including by politicians. ‘Production’ can be thought of as an upstream activity, and ‘use’ as downstream. In the think-piece, we are focussing on the downstream element of ‘use’.

• Although the paper focuses on downstream ‘use’, we should recognise that the way statistics are produced can raise risks of misinterpretation and hence be used in a misleading way. OSR frequently addresses issues with production, such as poor presentation or incomplete commentary, in our regular reviews of statistics. This work lies outside the scope of this paper.

• In thinking about ‘use’, we recognise that there is often a range of actors involved in presenting a claim about statistics: the Government body that produces and publishes the statistics; the communications team that presents information drawing on the statistics; media interpretation and summary of what is said; social media reuse of short segments of what is said; and many more actors. It is not OSR’s role to intervene at all points in this chain. Our role is to focus on how prominent politicians take the statistics and use them in their own communications – for example, speeches, press releases, social media statements. There are other organisations, including Ofcom and press regulators, who consider the work of various media actors.

• In terms of ‘use’, there will always be a risk that too much weight is put on a particular set of statistics. As Paul Allin, chair of the Statistics User Forum, told us: “Statistics rely on precise definitions of things being measured, but result almost invariably in some imprecision on the measurement. Statistics are not strict accounts and may have confidence or error limits.”

• Intention is not a helpful basis for guiding or supporting the OSR’s judgements about misleadingness. Both regulators and philosophers agreed that deciding someone has intended to mislead is difficult, subjective, and likely to lead to unnecessary controversy.

• It is far better to consider likely impact on audience, rather than intentions of the speaker. This approach is consistent with that taken when judging misleadingness in other contexts, for example by the Advertising Standards Authority and Ofcom.

• Judging intention may be important to some people – for example, journalists wishing to understand and explain the factors behind particular decisions or arguments. But OSR is forming a view on the appropriate use and interpretation of the statistics, not judging the motivations and drivers of the person using the statistics.

• Although judging the intention of the speaker may not be the right approach for OSR, when considering materiality we will look at whether the use of statistics is significant – and one element in this consideration can be whether a particular use is repeated over time or part of a prepared communication (speech, political ad, etc). These factors will inform how we take forward a case – for example, how strongly we express any concerns we may have.

• Some specific issues arose that need further consideration:

1. There were some risk factors that the original think-piece did not consider, for example the use of incomplete statistical evidence (eg placing too much weight on early results of a new policy) or recency (eg placing too much weight on the latest data, even if changes the new data appear to show are not meaningfully different from past data).
2. Many of the cases that OSR will deal with are relatively simple – for example, false statements that should be corrected, or use of unpublished data. In these cases, misleadingness would not be considered. In more complex or ambiguous cases where it is harder to reach a judgement, OSR would consider whether a statement has been misleading.

• The think-piece is relatively silent on the role of intermediaries. In one of our conversations we discussed a scenario in which a speech that is carefully constructed, well researched and uses statistics appropriately, is summarised in a single soundbite in media reporting. It was suggested that the speaker may actually intend this outcome, knowing that a careful speech will inevitably be packaged into a soundbite that could be misleading.

• As noted above, in the case of media intermediaries, the OSR approach would typically focus on the content of the original communication, not the media reporting of it. In the same way that we can’t assume the intentions of a speaker, it is similarly difficult to comment directly on the interpretation made by intermediaries. We can however give our view on the correct interpretation of the underlying statistics.

• One particular feature of political rhetoric, highlighted to us by Thomas King of the RSS Data Ethics section, is that different actors can draw widely different conclusions from the same underlying evidence. The point of democratic discussion is that different arguments are put forward; different narratives are presented; and different visions of good policy and the public interest are articulated. OSR’s role is not to judge these different perspectives, nor to limit the use of statistics to support them. Instead, our role is more humble: we simply try to ensure that the statistics are used in a way that does not give a misleading impression of the statistical picture.

• It is not OSR’s ambition to be an arbiter of political debate, nor would it be appropriate. Our role is to protect the role of statistics in public debate – that is, to ensure that their content and any caveats are respected in the way that they are used.