Conclusions and next steps

3.1 Conclusions

It is clear that showing uncertainty in estimates, for example through data visualisation, is essential in improving the interpretation of statistics and in bringing clarity to users about what the statistics can and cannot be used for. At the same time, however, we recognise that this is often not always a straightforward task.

We found that uncertainty presentation was best-developed in statistical bulletins. This often comes in use of words like “estimate”, rounded numbers and warnings of caution. In some cases, the warnings to users could be more helpful if they were more specific.

However, one of our discoveries was the relatively low level of reporting uncertainty in data tables. This is a clear gap as many users of statistics will only reference the data tables and extract the data to use for their own analysis. If the level of uncertainty is not evident then further misunderstanding could result. But we also recognise here that the task isn’t easy and we would encourage producers to adopt approaches such as the data shading illustrated earlier as a promising way to making improvements in this area.

Our regulatory work follows suit – our focus to date has been more on bulletins and less on data tables, and there is clearly more that we can do to challenge and support the statistical system in presenting uncertainty across the whole range of statistical and data outputs.

We also found that there is a good deal of guidance already existing to help statistics producers understand and present uncertainty. There is also a range of organisations – the Winton Centre, DQHub and Full Fact to name three – involved in enhancing understanding and developing presentation of uncertainty.

3.2 Next steps for statistics producers

With support from us and those at the centre of the GSS, we encourage Heads of Profession for Statistics to review whether uncertainty is being assessed appropriately in their data sources, and to review how this is presented in all statistical outputs.

As part of this, sharing good practice across the GSS on what has worked well in terms of communicating uncertainty will bring benefits right across the statistical system. One of the key routes to share this good practice will be through the data quality champions network and we encourage the network to support this endeavour. In terms of feeding into work on uncertainty across the GSS and beyond, finding good examples of where uncertainty has been presented well or described well by producers is important. This can serve both as a way of highlighting good work and also showing less experienced statistics producers ways of presenting uncertainty in their statistics that they may well not have thought of.

3.3 Next steps for us as regulators

We will continue to review the communication of uncertainty in our regulatory projects. We already have a good range of experience and effective guidance to help review uncertainty presented in statistical bulletins and methodology documents. We will continue to use this, and enhance as needed.

We will generate new guidance for ourselves to help us evaluate the presentation of uncertainty in charts, infographics, data tables and other “non-bulletin” presentation of statistics. We will use the examples identified so far to help us do this. We will also reinforce the benefits of using the QAAD framework to understand uncertainty associated with administrative data.

We will continue to collect examples that show both good communication of uncertainty and also that might require further work. Through this we can improve on the judgements that we make and the guidance that we produce and start to focus in on more specific areas where improvement is either needed or good work can be showcased. We will work with Heads of Profession for Statistics, and GSS networks (such as the Quality Champions) to help spread and reinforce good practice across the GSS.

We will also continue to work with other partners, particularly DQHub, to strengthen and enhance the current guidance to cover:

1. The presentation of uncertainty in data tables
2. Best practice on using data visualisation to communicate uncertainty
3. Uncertainty in administrative data

We will also work with the Analysis Function to develop guidance around how best to present uncertainty in a way that meets accessibility guidelines. Initially, this would require some scoping work to understand the technical barriers that exist with approved chart tools. This work would also benefit from engagement with the Winton Centre, to understand to what extend its work can be applied within the current accessibility guidelines.