Summary

1. Revisions made in autumn 2023 to estimates of the size and growth of the UK economy during the COVID-19 pandemic have generated some criticism of the Office for National Statistics (ONS). We therefore have conducted this review of the ONS’s approach to revisions to its estimates of Gross Domestic Product (GDP).

2. Estimating GDP and its growth poses several challenges because of the size and complexity of the economy. GDP can be estimated in three ways:

• The production approach, which is the sum of the production of goods and services in the UK
• The expenditure approach, which is the sum of household, non profit institutions and government spending, investment and exports less imports.
• The income approach, which is the sum of wages, operating profits, the income of the self-employed and taxes less subsidies on products and production

3. The information upon which these estimates are based accumulates gradually over time, with a relatively comprehensive picture only becoming available around two years after the reference period. As a result, revisions to early, near real-time estimates are normal and unavoidable and are an inherent part of the evolving measurement process, rather than the correction of avoidable errors. Furthermore, ONS’s approach to revisions mirrors standard practice worldwide. Users recognise that having access to early estimates is valuable but inevitably means revisions will need to be made once data coverage increases.

4. Regular users of the data recognise that there were particular challenges in measuring GDP during the pandemic, when changes in the size and composition of economic activity were relatively large, compared to conventional recessions and recoveries. In that regard, it should come as no surprise to see larger revisions than is usual at such turning points. The criticism was not in our view a reaction to the size of the revision itself, but the fact that it challenged prevailing narratives about the UK’s economy.

5. Our overall conclusion is that the more extreme criticisms that ONS received are misplaced. Nevertheless, this experience provides the basis for improvements in producing and communicating these key economic statistics. On this basis, we consider that ONS’s approach to revisions is appropriate and well managed. However, we have set a number of requirements for improvement:

Communicating GDP estimates, uncertainty, and revisions

• To promote public and media understanding, ONS should enhance its communication of the uncertainty that lies around GDP estimates as they evolve.
• When presenting revisions, ONS can improve its analyses by contextualising the magnitude of revisions relative to the size of GDP growth and by comparing them during economic turning points with earlier periods of structural change.

• When explaining revisions to early estimates of GDP, ONS should enhance the use of both qualitative and quantitative resources using diagrams that aim to visualise a range of possible values and indicate uncertainty.
• ONS could improve public understanding of uncertainty in early estimates of GDP without reducing trust in statistics, for example by using fan charts that show a range of possible values, or density strips which use colour intensity to show the proportion of revisions that fall between different values.

Understanding the use of the statistics

• ONS needs to enhance its understanding of the wide user base for early and more mature estimates of GDP and how these users interpret the statistics, given their uncertainty.
• To assist users in finding explanatory information on revisions to GDP estimates quickly and with ease, ONS should present such information in a central place so that users do not face the challenge of having to navigate through earlier publications and articles to piece the story together.

Quality assuring revisions to reflect the main economic story

• ONS should provide users with clearer information on how data sources that contribute to revisions reflect changes in the structure of the economy.

Improving access to data

• To maximise the chances of correctly capturing the economic signals from different parts of the economy in early GDP estimates, ONS should consider what data sources and estimation methods it can use to become less reliant on the production approach.

• As economic volatility continues to pose challenges in estimating economic growth, there is a need for ONS to have faster access to data sources for all three measures used in producing early estimates of GDP.
• To improve estimates of value-added by producers and real economic growth during volatile periods, ONS should consider enhancing its data sources to account for how producers adjust their intermediate expenditure when responding to economic uncertainty.

• ONS should consider sourcing more-timely data to strengthen the main expenditure and income components of early GDP estimates, and revisit the relative balance between these sources and the production approach.

• ONS needs to work with HM Treasury and government departments from whom it is seeking to access data, to enhance sharing and linking data.

Scope and Approach

6. This review covers:

• Processes and quality assurance involved in making revisions to GDP
• Potential improvements to early estimates of GDP enabled through enhanced access to data
• Communication of:
  • revisions to GDP
  • the story behind the most recent set of revisions
  • uncertainty in early estimates of GDP

7. Our review focused on revisions to early estimates of quarterly GDP, which are ONS’s lead measure of the size and growth of the UK’s economy, following the Blue Book annual Supply and Use balancing production process. We have spoken extensively with ONS, including those responsible for data collection, compilation and analysis of economic statistics and the economic communications team, as well as policy users from HM Treasury, the Office for Budget Responsibility (OBR) and the Bank of England (BoE), to discuss the three areas of the review in depth.