Standard seven of the Standards for Official Statistics in the Code of Practice for Statistics outlines what is necessary to help users and potential users decide whether the statistics are suitable and useful for their own use.
Being open about the quality of the statistics and considerate about the varying needs for different types of users can ensure that statistics are understood and used appropriately – or indeed not used if not relevant or suitable for a specific use.
This standard requires the producers to show their professional skill and judgement in understanding the quality characteristics of the statistics in relation to their use.
The Standard
7. Producers must prominently explain the quality of the statistics, including strengths and limitations, and communicate the uncertainty in estimates – so that the public can have confidence in using the statistics to make decisions and take actions
7.1 Prominently communicate the quality of the statistics and the strengths and limitations that impact their use, reflecting the needs of different types of users
7.2 Report on the key quality dimensions such as accuracy and timeliness, and, where possible, give estimates of error and confidence for the statistics. Summarise how uncertainty in the estimates may impact use by using qualifying words, numbers or graphics
7.3 Explain the nature of data sources and why they were selected, anticipating possible areas of misunderstanding or misuse. Prominently communicate limitations in the underlying data and explain their impact on the statistics
7.4 Be clear about the methods used. Explain quality issues related to the methods, systems and processes, including the extent to which the statistics are representative and comparable across the UK and internationally. Describe potential bias and steps taken to address it
7.5 Give advance notice to users of planned changes to methods and sources that will impact the statistics. Explain the nature and extent of the change, and provide a consistent back series where possible
7.6 Clearly flag where statistics are being developed and tested. Be transparent about developments, outlining the plans and expected outcomes, as well as the opportunities for users to be involved in the evaluation of the statistics
Questions to consider
1. Purpose
What is the original purpose for which the statistics have been produced? What are the main strengths of the statistics that make them particularly useful, and are there other purposes? How clearly do you explain the methods you use?
2. Limitations
What are the main limitations that might constrain how the statistics can be used? What would you not want a user to do with the statistics? Are there important sources of bias to factor in? How are you communicating about the strengths and limitations alongside the statistics?
3. Uncertainty
What is the likely size and direction of uncertainty in the statistics? What is the implication of the level of uncertainty for the interpretation and use of the estimates? What is the risk of users being misled by the statistics?
4. Misuse
If there is potential for misuse (for example, drawing a conclusion or making a decision that may not be borne out if you had perfect data), is there information you can give to help mitigate this risk?
5. Related statistics
Can you point towards an alternative set of statistics that might answer other aspects of the subject area that your statistics do not address?
Related guidance
Office for Statistics Regulation:
Government Statistical Service (GSS):
- Writing about data
- Communicating quality, uncertainty and change
- Uncertainty toolkit for analysts in government
- Guidelines for Measuring Statistical Quality
European Statistical System:
Office for National Statistics:
Survey Futures:
Good practice examples
- ONS guide – with a data quality framework rating for each source: Crime trends in England and Wales and how we measure them
Case studies:
- Office for National Statistics: Effectively Communicating Uncertainty in GDP
- National Records Scotland: Communicating uncertainty for Scotland’s Census 2022; a case study from National Records of Scotland
- DWP – Improving quality assurance and its communication to aid user interpretation
- ONS – Publishing information about data quality assurance processes
Blogs:
- Cabinet Office: Communicating the quality of ethnicity data
- OSR: Lessons in communicating uncertainty from the Infected Blood Inquiry: What to say when statistics don’t have the answers
- OSR: Why communicating uncertainty is a constant challenge for statisticians
- OSR: Revising GDP: The challenge of uncertainty
- OSR: How to communicate uncertainty in statistics