Findings

Communication of revisions to GDP

Presenting revisions

8. ONS provides a variety of information on revisions, including real-time databases, revisions triangles and articles explaining changes to GDP and its components. In October 2023, ONS brought forward the explanatory article on the Blue Book 2023 revisions to coincide with the publication date of the Blue Book. During previous years such an article was typically released up to six months after the Blue Book publication. The 2023 article analysed Mean Absolute Revisions (MAR), which adjust for revisions of opposite signs compensating each other, in ten-year intervals from the second quarter of 1961 to the fourth quarter of 2020. We consider that ONS has been clear in its communications in 2023 that revisions are normal and that in less-volatile times, revisions are small.

9. ONS also found some evidence that “revisions were marginally statistically significant” when comparing early estimates of GDP with those produced after the Supply and Use balancing process, three years after the initial estimate was produced (T+36 months).

10. In October 2023, ONS analysed relative revisions as a proportion of the initial estimate of the change in GDP. However, we consider that this analysis could be improved by two further features:

• Firstly, by using the Relative Mean Absolute Revisions (RMAR) discussed at the OECD’s Working Party on National Accounts ONS could contextualise the magnitude of revisions relative to the size of growth rates and devoid of directional qualities.
• Secondly, by comparing revisions during economic turning points with earlier periods of structural change, for example the Global Financial Crisis (GFC) in 2008-9, users can understand how volatile initial GDP estimates can become during more uncertain economic conditions.

Annex A provides a revisions analysis review, illustrating the Mean Revisions (MR) which show whether revisions are biased in a particular direction (positive or negative), as well as the MAR and RMAR during the COVID-period compared with both the pre-COVID and GFC periods. It shows MR and MAR to early estimates of GDP during the COVID period were on average substantially larger compared with the pre-COVID and GFC periods. However, relative to the larger growth rates, these early estimates during the COVID years were smaller compared with the earlier periods.

Communicating uncertainty

11. ONS’s communication of revisions and uncertainty in its National Accounts and economic statistics more widely is mixed. ONS has done some good work through its Economic Statistics Centre of Excellence (ESCOE) initiative to understand the user perspective on the communication of uncertainty. ONS also holds a journalist lock-in on every publication date when releasing early estimates of GDP to provide media briefing. To support the Quarterly National Accounts publication in September 2023, which was consistent with the Blue Book 2023 publication, ONS held an additional open invitation briefing for all media.

12. Users we spoke to as part of this review told us they would like to see ONS provide more analysis in articles explaining revisions in the annual Blue Book production cycle, and they are often frustrated by having to search through earlier articles to understand the full picture. They would prefer a more user-friendly way of presenting all the information that affects the revisions in one location, which they could access with ease.

13. There are some good features, for example:

• the use of words like “estimated” in statistical bulletins, and references to the fact that numbers have been revised. The latest quarterly national accounts publication, released 29 September 2023, demonstrates this, stating that “UK gross domestic product (GDP) is estimated to have increased by an unrevised 0.2% in Quarter 2 (Apr to June) 2023”.
• ONS’s publication of a range of articles at the start of the COVID-19 pandemic explaining the increased uncertainty in measuring economic activity, and effectively paving the way for user understanding about larger-than-usual revisions. For example the publication Coronavirus and the effects on UK GDP, the publication of revisions triangles, and articles covering revisions is also good. ONS has also explained how it communicates movements in GDP and the impact of revisions in helping to improve understanding during economic turning points, using times of little or no growth and contractions as examples.

14. Whilst these improvements enhanced the coverage of early estimates of quarterly GDP, ONS recognised that the trade-off between timely estimates and revisions still remained. These early GDP estimates remained as “estimates and they are subject to uncertainty and a variance around the central estimate.”

15. However, in general ONS could improve its provision of adequate information about uncertainty, including those areas where revisions are expected to occur, to help the appropriate reporting and interpretation of the statistics. The following points, while relating to GDP, reflect a set of issues that we see across the wider range of economic and other statistics published by ONS:

• ONS does not present any introductory narrative that places the latest figures within the context of recent or historic trends. We consider that the interpretation of the statement in para 13 “UK gross domestic product (GDP) is estimated to have increased by an unrevised 0.2% in Quarter 2 (Apr to June) 2023” could be taken to paint a somewhat different picture without the context of saying that the value of UK economic output for Quarter 2 (Apr to June) 2023 continues a broadly flat trend since 2022, the point at which economic activity broadly returned to its pre-pandemic levels, and that the latest estimate is preliminary and subject to revision.
• Although ONS has previously explained the challenges of measuring GDP during economic turning points and when there is little or no growth, there is little in the language that ONS uses that suggests that seemingly small changes – 0.1%, 0.2% etc – are likely nothing other than noise and should not be interpreted as a strong signal of a particular trend in the publications of early estimates of GDP. This is especially important as in some cases (particularly where economic growth is close to zero) subsequent revisions to data may change not only the magnitude of movements between periods, but also whether the best estimate is of growth or contraction. We consider that descriptions like “a largely unchanged picture”“a largely unchanged picture” as used by Darren Morgan in relation to ONS’s labour market statistics released on 14 November 2023, portray a more appropriate message about the latest trends given the uncertainty in the data.

16. When communicating uncertainty around early estimates of GDP growth, ONS should enhance both the qualitative and quantitative descriptions, making appropriate use of words such as “about” or “around” when referencing these early estimates. These terms can help inform users that the figures are not facts and therefore carry a level of uncertainty. We recommend avoiding phrases that appear to present the estimates as hard facts such as “GDP increased by 0.3%”, consistent with the guidance from OSR’s Approaches to presenting uncertainty in the statistical system. Building on the experience of how early estimates of GDP were presented in the media in September 2023, ONS could anticipate issues that might concern users through media monitoring and risk assessment, and prepare to pre-emptively warn users. By being open about the quality of the underlying evidence and what estimates of the UK economy can and cannot be used for, ONS can enhance the credibility of early estimates of GDP growth. In doing so, ONS could enhance users’ understanding by following the Government Communications Service guidance in the disinformation toolkit.

17. When presenting early estimates of GDP, ONS should also use the findings from the ESCOE project on communicating uncertainty in economic data. This research found that “communicating uncertainty information alongside the GDP estimates improved the public’s understanding of data uncertainty, without reducing its trust in the statistical office.” Supporting early GDP estimates with qualitative evidence was considered better than text descriptions and limited communication that didn’t go beyond referring to GDP as an “estimate”. Equally effective were the use of quantitative resources such as fan charts, density strips and bell curves, which enhanced users’ understanding of economic uncertainty when explaining revisions to early estimates of GDP.

18. While neither us nor ONS can stop journalists or influential social media commentators presenting an unbalanced perspective, ONS should follow OSR guidance on Communicating uncertainty in statistics and recognise, “different users may want different information about uncertainty depending on the nature of the decisions they’re faced with making and their level of expertise”. ONS has to make every effort to ensure that users have the opportunity to obtain the key messages from the statistics, and particularly the uncertainties involved so that any estimated changes are not overinterpreted and are reported in their proper context.

19. Users we spoke with told us that they would like to see ONS provide clearer explanatory information on how data sources are changing to support the main economic story. Users also said that it would help them if such information was in one central place, so that they do not face the challenge of having to collect information from earlier published articles to piece the story together.

20. Users told us they would like ONS to explain revisions affecting early estimates of GDP during economic turning points in the context of current affairs such as rising inflation, energy prices and interest rates, globalisation and geopolitical events. Whilst these changes make it challenging to measure the size and growth of the economy, they can also change the path of GDP growth and therefore add further complexity when assuring the quality of early estimates of GDP. We note that ONS’s word limits on some publications may have affected what has been able to be published, but would expect ONS to ensure that capacity is available to publish what is needed.

Processes and quality assurance

21. ONS reviews its National Accounts revisions policy annually and provides updated revisions triangles for GDP, which include the income and expenditure components and the GDP deflator. In 2018 ONS introduced a new publication model for GDP explaining which periods revisions would affect GDP growth, in monthly, quarterly and annual estimates and the three types of revisions; annual benchmarking, seasonal adjustment and the UK National Accounts: The Blue Book production cycle.

22. ONS publishes early estimates of quarterly GDP growth around six weeks after the reference period as a first quarterly estimate of GDP; and around 12 to 13 weeks after the reference period as part of the full quarterly National Accounts, where it provides a complete set of accounts for the quarter, taking into account revisions to the component items for earlier quarters. Annex B shows ONS’s revisions timetable for quarterly estimates of GDP up to September 2023.

23. After incorporating annual sources using the structural analysis of the Supply and Use framework, ONS does not revise the years covered by this process until the following year’s Blue Book cycle. This means during subsequent quarterly rounds, post-Blue Book publications, the years that have undergone the Supply and Use balancing process will be closed to revisions.

24. An important part of the quality assurance process occurs during so-called curiosity and balancing meetings, which aim to provide an opportunity for data from different sources to be compared. During these meetings, contributions to significant changes and revisions to the estimates are discussed in depth, along with survey response rates and associated bias and the extent of coherence with indicators from other sources. Statistics from outside ONS are also used to quality-assure the GDP estimates. These include international estimates of GDP, business confidence, car registrations, spending on debit and credit cards, traffic measures and job adverts.

25. Some users we spoke with as part of our review, however, explained that during the economic volatility notable in the latest years, they would like to see better alignment of these early estimates of quarterly GDP with the Public Sector Finance Statistics, though they recognised the constraints of the annual National Accounts revisions policy. Some users also wanted to see a greater focus on the levels of GDP, rather than the sole focus being the quarterly growth patterns, particularly as levels are used in several ratios that can determine fiscal and economic policies during economic shocks.

26. ONS compiles and confronts GDP estimates at a component level, bringing data together for the various measures via a central approach. Some users were concerned that ONS lacked the in-depth knowledge to challenge the economic story for particular component data, querying how ONS was unable to clearly explain the statistics with a plausible economic explanation, including in relation to policy. Another user told us they felt that ONS often was not able to explain the impact of GDP revisions on lower-level components such as the incoherent implementation of deflators for imports and exports in some products, the impact on business investment, inventories, trade margins, and industry breakdowns; all of which are important components in estimating GDP. However, at the Public Administration and Consultation Affairs Committee in September 2023, the National Statistician explained that retail and wholesale margins were one of the main causes of the GDP revisions, and recognised further work was needed to better understand this. This was a useful clarification of the drivers of change.

27. The need for ONS to become better at understanding the economic context and situation and therefore interrogating changes in the GDP path was also discussed in the Independent review of UK Economic Statistics, led by Sir Charles Bean in 2016. The review recognised the need to develop an “enhanced capability within ONS to interrogate the underlying microdata would better support the production of economic statistics and have the added benefit of making ONS staff better able to sense-check statistics before their release.” We recognise that while there are some very experienced senior statisticians and ONS aims to keep a balance of experience across teams, turnover of staff can be an issue for ONS when building and retaining this capability.

28. The National Statistics Quality Review of National Accounts and Balance of Payments, led by Dame Kate Barker and Art Ridgeway in 2014, also recognised the importance of robust early estimates of GDP for policymakers, “especially at turning points in the economy”. Using the larger revisions to the 2008-09 recession quarterly path for GDP as an example, the review recommended ONS consider how the various annual and quarterly processes and procedures contributed to these (larger revisions) and to consider whether any lessons could be learnt for future turning points in the economy.

Potential improvements

Data sources and coverage

29. In 2018, ONS introduced a new publication model, responding to recommendations from the National Statistics Quality Review of National Accounts and Balance of Payments and the Independent Review of UK Economic Statistics.ONS aimed to balance timely GDP estimates with accuracy and quality of available data, while reducing the likelihood and frequency of revisions to these early estimates. Even with these changes, ONS explained that the data content to estimate output remains higher, compared with the expenditure and income approaches. The production approach therefore continues to take the lead in deriving GDP growth. As more data become available from the expenditure and income approaches, ONS then balances all three approaches to derive an average growth.

30. One of the main catalysts of revisions to early estimates of GDP is the replacement of these indicators, predominantly from the production approach, with reconciled data sources from all three approaches used to measure GDP, via the Supply and Use framework. One of the main strengths of this compilation method is that it uses all available information to reach a single estimate of GDP. In doing so it maximises the chances of correctly interpreting the economic signals within the different data sources, particularly during economic turning points and/or times of economic shocks.

31. Whilst the granular industry and product details found in the annual data are not available when compiling early estimates of GDP, ONS could still benefit from a higher industry and product-level quarterly Supply and Use framework. By removing the reliance on the production approach, which could miss some economic signals during periods of uncertainty, and supplementing it with other data sources relating to expenditure and income, such as administrative sources and real time indicators which may capture the correct signals, early estimates of GDP could better measure economic reality.

32. This echoes the findings from theNational Statistics Quality Review of National Accounts and Balance of Payments, which recognised that “failure to fully use all three measures underutilizes the information content from the surveys feeding into the expenditure and income components and lowers the overall quality of the accounts and increases the risks of false signals to users”.

33. ONS explains that its short-term output estimtes have a higher data content compared with the expenditure and income approaches. To produce early estimates of GDP, ONS balances all three approaches to derive an average growth, except in the latest two quarters when the output data take the lead because of their larger data content.

34. Users we spoke with have challenged the suitability of ONS continuing to use output as an indicator or economic growth in early estimates of GDP growth, during times of economic shocks. This approach may not always reflect value-added by producers and real economic growth during volatile periods, because it does not take into account how producers adjust their intermediate expenditure when responding to this uncertainty.

35. Responding to theNational Statistics Quality Review of National Accounts and Balance of Payment recommendations, ONS reviewed the drivers of revisions to early estimates of GDP during the 2008 and 2009 downturn, introducing numerous improvements to quarterly output sources. These included:

• reviewing actual returns against forecasted elements of the Monthly Business Survey and building a bias adjustment in the construction output
• introducing VAT data, which has a larger sample size, to directly measure some of the output components
• reviewing current price measures in more detail while considering the impact of deflators and assessing coherence with chained-volume measures
• improving estimates in financial services and insurance industries
• paying more attention to the quarterly GDP growth path, particularly when annual estimates remain relatively unchanged and using quarterly information to apportion financial year data into the correct calendar years more optimally

36. The National Statistics Quality Review also considered whether using the production approach as the main indicator of growth enabled sufficient analysis of the different paths of the three measures and whether the practice of aligning expenditure and income to the production approach’s GDP growth led staff to be less focused on the quality of the other two measures.

37. Similarly, the Independent Review of Economic Statistics echoed these findings, encouraging ONS to identifying scope for “improving early estimates of GDP through the use of administrative data, including by making greater use of information from the expenditure and income measures.”

38. ONS recognised the challenges in measuring intermediate consumption highlighted during the COVID period and is reviewing how it can identify more timely information on intermediate consumption and has made efforts to enhance its data coverage, working closely with HM Revenue and Customs (HMRC). For example, to improve the data coverage on intermediate expenditure of market output producers, ONS has considered using VAT purchases data. However, these data are not timely enough for use in preliminary estimates. Furthermore, ONS found monthly VAT data to be heavily skewed in certain industry sectors that were more likely to receive VAT refunds. We encourage ONS to be open and consider more creative uses of these data and explore whether it can extract information even if it is not as timely as would be ideal. We also encourage ONS to consider making better use of published company accounts and existing business surveys that could potentially provide additional information on intermediate consumption, in early estimates of GDP.

39. Additionally, ONS recognised that “during the pandemic, the intermediate consumption ratios changed significantly from year-to-year”. This, combined with no quarterly data source for intermediate consumption, meant “it was not possible to anticipate the scale of change in these ratios”.

40. ONS has also welcomed the earlier access to intermediate consumption data for public sector health, on a quarterly basis, from the Department of Health and Social Care. Although miscoding issues in the incoming data were noted in the initial estimates and more-detailed briefings on the data movements would enhance the quality assurance of the data, ONS uses these data when compiling early estimates of GDP. As significant revisions came from the non-market output sector, to improve estimates of value added in the non-market output sectors of the economy, ONS should continue pursuing access to data that will improve the measurement of intermediate consumption in the public sector, such as education, social care and public administration and defence.

41. We recognise the challenges that ONS faces in finding suitable data sources to estimate the components of GDP. We also acknowledge that the data may not always be readily available in an accessible and user-friendly format. We therefore encourage ONS to seek all possible opportunities to supplement the use of information from a wide variety of sources: administrative sources; published company accounts; existing business surveys, adjusted as appropriate to meet methodological concepts; and new surveys where needed. Where suitable sources are not available, ONS should consider the use of modelling and clearly communicate to users the constrains and impact of these limitations, explaining what the statistics can and cannot measure.

42. ONS has recognised weaknesses in the data sources used to estimate the expenditure components of GDP, such as the significant decline in response rates to the Living Costs and Food Survey, which is an important source used to estimate the largest expenditure component of GDP – household final consumption expenditure. Changes in inventories is another expenditure component that is challenging to estimate and ONS would like to pursue better sources to improve these estimates.

43. ONS also encountered challenges in measuring Gross Operating Surplus of private non-financial corporations because the results from the Quarterly Operating Profits Survey were deemed to be of a low quality and value. This led ONS to suspend the survey, without being able to find an alternative suitable data source.

44. In light of the continued economic uncertainty, ONS should revisit the relative balance of producing early estimates using the production approach compared to using the other approaches (expenditure and income). We echo the recommendations from theNational Statistics Quality Review and the Independent Review of Economic Statistics, in the need to further enhance the data coverage of the expenditure and income components for early estimates of GDP.

Data sharing and linkage

45. Improved data sharing between ONS and other government departments could enhance early estimates of GDP and support policy-makers in making well-informed decisions earlier.

46. We recognise ONS’s perseverance in addressing barriers to data sharing and linkage. For example, working closely with HMRC resulted in ONS acquiring data such as on PAYE RTI, Self-Assessments, Corporation Tax and Tax Credits. Whilst these data have been shared in the form of one-off bulk supplies, ONS is in the process of transforming its Data Access Platform into a cloud based platform to meet HMRC’s encryption requirements, which are essential for securing automated regular deliveries of the PAYE RTI and SA data.

47. Progress on data sharing and linkage with government departments has been slow for ONS. ONS told us that when collaborating with other government departments it has experienced barriers such as low appetite for risk from departments and limited resources due to ministerial outputs being prioritised over statistical production. Furthermore, complex data security requirements could be better harmonised across departments.

48. ONS’s experience echoed the findings from OSR’s report: Data sharing and linkage for the public good, recognising the need to develop and establish suitable Privacy Enhancing Technologies and the importance of strong leadership. ONS should capitalise on the collaborative data sharing relationships established during COVID and work with Permanent Secretaries and HM Treasury to champion the public benefit of sharing and linking data with government departments it is seeking to access data from.

49. To support policy-makers in reaching well-informed decisions, ONS and HM Treasury should work with other government departments that ONS is seeking to access data from, to develop a timeline with commitments that would need to be met to enable ONS to facilitate data sharing and linkage with other government departments. We will continue to advocate for and support data sharing and linkage across government. Our recent review outlined 16 recommendations that, if realised, will better enable data sharing and linkage for the public good.