2. Findings

2.1 Prioritisation

2.1.1 As part of the Spending Review 2021, ONS developed a statistics prioritisation framework, which set out high-level priorities for ONS based on statutory requirements, spending review commitments and alignment with strategic goals. The framework prioritises the development of more-prominent, market-sensitive economic statistics such as consumer price inflation over statistics with a lower profile and smaller user base such as PPI. While it is right that ONS prioritises the more-important statistics, years of under-prioritisation have negatively affected the quality of the PPIs.

2.1.2 The business prices team has prioritised certain quality improvements over others. In particular, it prioritised the development and implementation of the annual chain-linking methodology. While this method has significantly improved the quality of the statistics and brought the methods in line with international best practice, the change came at the expense of updates to the PPI survey samples, which have been paused since 2019. This report highlights several other examples where a lack of investment has adversely affected quality, including the use of legacy systems.

2.1.3 Without reprioritisation and investment, there is a risk that the quality of the PPIs will further decline and that they become less robust, particularly in their primary use as a deflator. It is good that ONS is now formalising development plans to address some of the main quality issues with the statistics, including changes to the survey samples and moving away from the legacy Ingres-based system. Once finalised and fully resourced, ONS should publish the development plan to be transparent with users about the changes it is going to make and how these will improve the quality of the statistics for all users.

2.2 Survey samples

2.2.1 Most data required to produce PPIs are collected directly from manufacturers via the PPI, EPI and IPI surveys. The number of products in the samples changes due to products being removed from the index, either as they are no longer being manufactured, exported, imported or because products are no longer the most representative of the sales of the product’s category.

2.2.2 In 2005 ONS carried out a review to determine the optimal sample size for the PPI survey. This work determined that 9,000 price observations from around 5,000 manufacturers were required to calculate a robust PPI measure. In 2006, because of savings made as part of the UK Government efficiency programme, ONS subsequently reduced the PPI sample target by 25% to 6,750 price observations. The re-optimisation of the sample meant the sample in some sectors expanded and the sample in other sectors reduced. ONS focused on maintaining the accuracy of the high-level PPI, although inevitably the reduction in overall sample size meant a loss in the number of lower-level indices published because of lower quality.

2.2.3 A similar review of optimal sample size has not been conducted on EPI and IPI since the surveys were set up. Since 2006, the target sample for EPI has been 3,640 price observations and for IPI 3,550 price observations.

2.2.4 Currently, ONS is not achieving the target price observations for all three surveys. The number of products currently in the sample fluctuates from month to month, but for PPI in June 2023 the number was below the target, at 5,623 price observations. ONS is meeting the target sample for EPI (3,659 price observations in June 2023) but is well below the target for IPI (1,898 price observations in June 2023). Although the overall target sample for EPI was achieved, the business prices team told us that the distribution across industries of price observations is uneven.

2.2.5 Considering that the EPI survey covers the same industrial sectors as PPI but aims to measure prices within those sectors on both EU and non-EU bases, and the IPI survey covers a wider range of sectors than both PPI and EPI, the current sample sizes are not adequate for a robust measure, particularly for the lower-level indices. ONS considers EPI and IPI to be ‘low’ or ‘limited’ coverage surveys.

2.2.6 The business prices team told us that for some sectors and industries the EPI and IPI sample coverage is now too low to provide a reliable deflator and adequately measure the price changes of key volatile commodities. Users within ONS who use business price indices to deflate other economic series also raised concerns with us that the quality of certain indices is limited, to the point where they are no longer suitable for deflation. For example, the ONS deflators team told us that it uses around 20 PPIs as proxies for IPIs because the IPIs are not of suitable quality for use as deflators.

2.2.7 At various points the business prices team has attempted to boost the sample size of the surveys, but these efforts have not been successful. For example, in November 2016, the business prices team proposed to increase EPI and IPI sample sizes to 6,000 price quotations each. This was expected to have two benefits: it would eliminate any indices within the surveys that are constructed from single price observations; and it would improve the coverage of export and import indices. However, due to complications with the mapping of export and import data and the prioritisation of methodological improvements including annual chain-linking, updates to the samples were paused. Changes to the sample are now done on a needs-must basis by the data collection team and are generally focused on maintaining the PPI sample.

2.2.8 This lack of investment in updating the PPI, EPI and IPI samples poses risks to quality. For instance, the samples have become less-representative of trading manufacturers because they do not capture new manufacturers or new product lines. However, existing sampling processes do allow ONS to capture more-representative products from manufacturers already in the sample, as it asks manufacturers for the most representative product they produce.

2.2.9 Not updating the sample also increases the potential effect of sample attrition, which refers to products for which ONS no longer collects price observations because businesses in the sample are no longer active (referred to as “dead” products) but have not been replaced and yet still assigned a weight. Sample attrition has increased substantially since the sample updates were paused in 2019 and is particularly affecting the quality of the lower-level indices. Since 2019, the percentage of dead products in the PPI, EPI, and IPI survey samples has increased year on year and now stands at 17%, 20% and 32% respectively (Figure 1). Figure 1 also shows that between 2017 and 2019 there was a decline in sample attrition for some surveys, particularly the IPI survey.

Figure 1. Percentage of dead products in the PPI, EPI and IPI survey samples between 2017 and 2023

Survey response rates have fallen in recent years

2.2.10 As with other ONS business surveys, the COVID-19 pandemic significantly affected the number of respondents returning price data for the PPI, EPI and IPI surveys. ONS publishes response rates for all three surveys in the monthly statistical bulletin. These show that average response rates are currently around ten percentage points lower than they were before the pandemic. For PPI, the average weighted response rate between June 2022 and May 2023 was around 75% (range: 67.7% to 79.9%), compared with approximately 85% between May 2019 and March 2020 (range: 81.0% to 87.4%). The EPI and IPI surveys had similarly large falls in response rate over the same time period (Figure 2).

Figure 2. PPI, EPI, IPI weighted response rates between May 2019 and May 2023

2.2.11 Changes in working patterns during the COVID-19 pandemic have contributed to falling response rates, with more people were working from home. During the early part of the pandemic many survey response forms were sent to premises that were unoccupied. In addition, due to a restructure within ONS’s Business Data Operations Division, there is less resource to chase non-responders. The dedicated response chasing team was disbanded, with responsibility for response chasing transferred to the data collection team. As a result, fewer manufacturers are chased, and it has become more challenging to clear the backlog of non-responders. To minimise the impact of falling response rates the business prices team has asked the data collection team to take a more targeted approach to non-responders, by chasing responses from businesses in highly weighted industries in the PPI, which seems a sensible approach.

2.2.12 The falling response rates for all three surveys mean that ONS has less data to compile the indices from. As a result, more imputation is applied to the data compared to pre-pandemic periods. This can lead to less-representative trends in price changes, which affects the quality of the PPI estimates (see 2.5.10). Not only does low response rates adversely impact the data, but due to legacy systems, prolonged periods of non-response can result in notable revisions to previously published data (see 2.3.2).

2.2.13 It is essential that ONS addresses the issues with the PPI, EPI and IPI survey samples. Therefore, reviewing and updating the samples for all three surveys, but particularly EPI and IPI, should be a priority for improving the quality of the statistics for all users. We recognise the efforts of the business prices team to make remedial changes to the samples and acknowledge the team’s recognition that a wider review is needed. We also recognise that such a review may take some time to carry out. Given these factors, we encourage the business prices team to reconsider what else it might do with the resources available to quickly boost or reduce the sample in the areas it knows are suboptimal.

Moving survey data collection online will support higher quality data

2.2.14 Most manufacturers respond to the PPI surveys via telephone data entry (TDE): the data collection team told us that 61% of respondents in the PPI survey provided data via TDE in June 2023. ONS is currently developing an electronic questionnaire (EQ) for the PPI, EPI and IPI surveys, and expects to roll out this long-awaited improvement towards the end of 2024 or early 2025. This is part of an ONS-wide drive to move all statutory survey collection online. The larger economic surveys, including the Annual Business Survey, were prioritised first. The PPI surveys continue to be deprioritised because of the challenges associated with moving these surveys online. Sending a respondent’s product specification (i.e. a pre-populated questionnaire) back to them has been the main blocker of moving the PPI surveys online, but there are also challenges associated with protecting the confidentiality of data received from and sent to manufacturers.

2.2.15 We welcome the move to online data collection. The use of EQs, and other electronic data collection methods, will support higher quality data by making data easier to return for manufacturers and enabling point of collection validation. As part of ONS’s plans for rolling out EQs, the business prices team is looking to implement several additional collection and processing improvements such as developing Ingres’s capability to use data from EQs and automation of some aspects of the production pipeline. These additional improvements will further improve data quality, by reducing the burden on both manufacturers and the data collection team, and by giving the data collection team greater control over the data.

2.3 Legacy system

2.3.1 The business prices team told us that the Ingres-based system for producing PPIs is vulnerable to processing errors and poses several risks to the quality of the statistics. A lot of PPI data are input manually by the data collection team, which is resource-intensive, and increases the risk of human error. For example, new respondents’ price data, exchange rate data from published internet sources and IPI source data (100+ products) are all input manually. Certain aspects of the process have become more manual since the COVID-19 pandemic. Due to the change in working patterns during the pandemic, many manufacturers now complete a PDF form rather than enter data via telephone. The PDF forms need to be processed manually, which takes up additional resource.

2.3.2 ONS also told us that the Ingres system is inflexible and difficult in several ways:

• It has limited functionality to apply Reproducible Analytical Pipeline (RAP) principles, for example to automate existing processes.
• It is inflexible when it comes to handling missing data, which compounds the issues of falling response rates. The system does not accept gaps in price returns. For example, if a manufacturer has previously provided prices up until November 2022 and has not made any responses until ONS directly contacts them, for the missing data, the ONS will impute estimated prices for the missing months. If the manufacturer provides a price in March 2023, in most cases, the data collection team must enter the November 2022 price for all periods up to January 2023 before the February price can be entered. This replaces the previous imputed value which may cause a revision in previously published data.
• It is not good at dealing with new, non-traditional data sources and has struggled to keep pace with structural changes in the economy, for example, advances in sectors such as telecoms and computer hardware.

2.3.3 Furthermore, any significant development of the ‘front end’ of PPI (sampling, data collection and validation) is hindered by reliance on Ingres. For these reasons, ONS is planning to migrate all existing business surveys run on Ingres onto a new, more-flexible platform: the Statistical Processing Platform (SPP). In the meantime, ONS is implementing an Ingres-reduction strategy that aims to: ensure the effective integration of EQ collection tools; give the data collection team greater control over response and validation; and integrate non-survey data sources within data, for example, the telecoms deflator and existing administrative data sources.

2.3.4 Replacing the Ingres-based system is essential for futureproofing the production and development of PPIs. We recognise that there are plans in place to do this and that they will take some time to implement. Currently, there is no information in the public domain about the plans to transform the systems used to produce PPIs.

2.4 Quality assurance

Validation tests act as an early warning of large price movements

2.4.1 The business prices team has a positive relationship with the data collection team. As part of conducting the PPI surveys, the data collection team assures the quality of the price data collected from manufacturers. The two teams meet monthly to review data. At these meetings, the business prices team provides an overview to the data collection team of what has been published and the impacts of the data collection team’s work. The data collection team told us that it finds the business prices team supportive; for example, if it fails to get a manufacturer’s response after three attempts, it sends the query to the business prices team to make the final decision.

2.4.2 The data collection team carries out two main automated validation tests, which are used to check the monthly movement of prices and act as an early warning:

• Dubious prices – monthly price movements of plus or minus 12% or more are labelled as ‘dubious’ and are checked with respondents for confirmation of the increase or decrease.
• Incredible prices – monthly price movements of plus or minus 30% or more are labelled as ‘incredible’ and are checked with respondents for confirmation through a valid explanation for the price change.

2.4.3 The data collection team then contacts respondents to clarify the price data and updates Ingres accordingly. The values in para 2.4.2 have been used since early 2022, when they were adjusted to reflect increases in the volatility of prices; the threshold for dubious and incredible prices used to be plus or minus 7.5% or more and plus or minus 20% or more, respectively. When inflation was low, these lower thresholds were appropriate for checking for outliers in price changes. However, with annual input producer price inflation rates reaching 24% in June 2022, the number of legitimate price changes that were failing the validation tests increased sharply. To ensure the tests were proportionate, and to manage the workload of the data collection team, the business prices and data collection teams jointly reviewed the validation tests and adjusted the thresholds[5]. Since then, the thresholds for the validation test have been reviewed quarterly. It is good that the business prices team continues to review them in collaboration with the data collection team.

2.4.4 Apart from the 2022 adjustments, the validation gate thresholds have not been formally reviewed for a long time. It is important that ONS reviews and updates the validation checks so that they provide an appropriate level of quality and are adaptable to changes in the economy. The move to a new processing platform (see 2.3.3) provides a good opportunity to do this.

2.4.5 This validation process is not the only method used to identify incorrect price data. Subsequent analysis of results by the business prices team will highlight the more significant prices changes that are impacting results, which in some cases might be smaller price changes for high-weighted products. The data collection team then follows up with respondents to gather the correct data.

Curiosity meetings are an effective component of quality assurance

2.4.6 The second layer of quality assurance (QA) is the business prices team’s ‘curiosity’ meetings. In these meetings the team aims to understand which unexpected price movements are genuine and then makes judgements about how suitable it is to include the relevant data. The team considers whether price movements for a product with a small weight are having an amplified effect as a result of imputation, as these are considered to be detrimental to index quality and their effect must be mitigated. This is done on a case-by-case basis.

2.4.7 There are two curiosity sessions within the monthly production cycle: one that looks specifically at PPI, EPI and IPI, and another that brings together all ONS’s price indices (PPI, CPI, House Price Index (HPI)). These meetings are used to interrogate and query data after the indices have been produced and are held prior to the writing of the statistical bulletin. The meetings will flag if the prices of any products are having an unexpected impact on the indices, including those which do not fail the validation tests.

2.4.8 One growing area of focus for the curiosity meetings is the impact of imputation on the suitability of data, which is discussed below. The business prices team is also increasing the number of congruence checks it carries out as part of the curiosity meetings; it is looking to expand the data sources it uses to validate and corroborate EPI and IPI data. Because these indices have low sample coverage, at times they reflect specific products rather than general trends in those sectors, so there is a greater need for validation with alternative data sources. It is good that the team is exploring the use of alternative data sources to corroborate indices. We encourage the team to be transparent about which data sources it uses for this.

A formal quality review is carried out every year

2.4.9 In addition to the QA carried out for each publication cycle, the business prices team carries out a more formal annual quality review, using the Statistical Quality Maturity Model (SQMM) self-assessment tool developed by ONS. The SQMM review is done in consultation with the quality champions in the division, and the business prices team keeps a log of the discussions. The review identifies any areas for improvement and highlights areas that should be picked up in the curiosity sessions. It is good that the team carries out more-formal reviews of quality, and we encourage it to share the main findings of these reviews with users and explain how they are improving the quality of the statistics.

The quality assurance process has been improved since a series of errors

2.4.10 Between November and December 2022, the business prices team identified a series of errors in the statistics. These errors were mostly driven by the incorrect allocation or linking of weights (see Annex B for further details). The errors demonstrated that there were gaps in the business prices team’s QA process. The business prices team cancelled the publication of statistics in December 2022 to allow it to undertake further investigations.

2.4.11 The business prices team asked ONS’s Methodology and Quality Division (MQD) to review the QA process for the PPIs, including the code used to produce them. This review provided an additional layer of assurance about the robustness of the QA processes and whether these were proportionate to the complexity and impact of the output.

2.4.12 The MQD review made some recommendations for improving the QA process, including several essential improvements that had to be made before any further publication of the statistics. These included increasing the number of QA checks on database queries and improving senior oversight of QA. The review informed the decision to cancel the publication of the statistics in December 2022. The business prices team made all essential improvements before it started republishing the monthly statistics in January 2023. These improvements have reduced the risk of future errors and we welcome the team’s openness to peer review.

2.4.13 The review identified a range of other improvements that would further strengthen the QA process, such as designing and implementing a formal error management process and expanding the range of QA checks to include a broader definition of values of concerns. Implementing these recommendations would further enhance the quality of the statistics.

ONS should regularly analyse revisions

2.4.14 The statistics are revised according to the PPI revisions policy, which is in line with ONS’s National Accounts Revisions Policy. Figures for the latest two months are provisional and the latest 12 months could be revised. Several users told us that the revision window is too long and that revisions can be very volatile. They said they would like the revision window and the size of the revisions to be minimised, as the current arrangements can cause issues with pricing contracts.

2.4.15 ONS told us that when Ingres was still being used to process data, the revision window was only five months and most indices’ values were stable by the end of that window. The window was extended to 12 months with the movement to DAP but with the expectation that most data would be stable by five months; similar to the old Ingres process. However, due to the impact of the pandemic on respondent behaviour ONS now receives more late returns, which means that significant revisions span a longer period. ONS told us that the move from paper-based to online data collection (with the introduction of the EQ for the surveys, discussed earlier), is expected to give it greater control over revisions, by fixing imputed data after it is satisfied with the response level. This is expected to prevent subsequent revisions by retaining what it considers to be a sufficient estimate of the data.

2.4.16 While the business prices team records revisions to the data and publishes revisions triangles for the latest five-year period, it does not carry out regular analysis of revisions. This is another gap in the QA process. We carried out our own indicative analysis of PPI revisions to understand the volatility of the indices and the robustness of the preliminary figures (see Annex C). We found that, in general, preliminary figures are revised upwards, but the size of revisions is small. This suggests that improvements could be made to the survey or results systems to enable revisions to be centred more closely on zero.

The suitability and quality of all data sources should be reviewed

2.4.17 As explained earlier, most price data are collected via the three monthly surveys, but some price data come from other survey and administrative sources provided by third parties (see Annex A for full list). Many of these sources have been used for a long time, but their suitability and quality have not been fully assessed in recent years.

2.4.18 The business prices team engages directly with the suppliers of some of these other data sources. For example, DESNZ supplies ONS with a time series of energy prices data, which includes a monthly index number and a summary brief. The brief contains charts and a quality report with commentary about how estimates have changed and the scale of the revisions. This helps ONS contextualise major price changes which aids interpretation for users.

2.4.19 It is important that ONS understands the quality of all data sources used to compile the PPIs and communicates their quality to users. Our Quality Assurance of Administrative Data (QAAD) Toolkit would be helpful for this. We also think more-proactive engagement and closer dialogue with other data suppliers would enable ONS to better manage the quality of the statistics, particularly as initial QA is carried by the data supplier rather than the business prices team. This was also highlighted as an area for improvement in the MQD review (see 2.4.11).

2.5 Methods

2.5.1 International best practice for producing the PPIs is outlined in the IMF PPI Manual. The manual was produced in collaboration with four other international economic organisations, including Eurostat, and was last updated in 2010. In 2012, Eurostat published a handbook on PPI which brought together best practice across EU member states and reflected recommendations outlined in the IMF PPI Manual.

2.5.2 The price statistics experts from the international statistical community that we spoke to praised ONS’s methods and agreed that, in general, ONS works in line with international best practice. They also said it might be helpful if ONS highlighted the areas where it is less harmonised with international best practice.

2.5.3 The business prices team told us that it intends to remain consistent with international best practice as much as possible for producing PPIs but will examine where it may be appropriate to depart from standard international practice to best meet UK users’ needs. For example, the business prices team suggested expanding the scope of the statistics to include areas of the economy with particular interest to users such as imports of natural gas.

2.5.4 In 2020, ONS made three changes to the way it produces and publishes PPIs. These have improved the quality and international comparability of the PPIs and brought the methods in line with international best practice and the needs of users. The most significant change is the implementation of annual rebasing and chain-linking. ONS also moved from leading with a ‘net of inter-sector transactions’ measure to a gross measure in its industry weightings and removed duty from the headline indices.

Annual rebasing and chain-linking

2.5.5 Annual chain-linking is a well-established method for price statistics. Chain-linking is used as part of annual rebasing where the indices’ weights are updated every year. Previously, the weights were updated only every five years. Annual chain-linking is considered international best practice and is recommended by Eurostat[6]. The main advantage of annual rebasing is that it maintains accurate and representative weightings, which is beneficial for measuring a dynamic economy such as the UK’s. For example, advances in the computer hardware industry led to new products frequently replacing obsolete products. By implementing annual chain-linking, PPIs are better equipped to adapt to structural changes in the economy, reducing bias, and enabling comparison of price changes over time.

Net to gross measures of industry weightings

2.5.6 ONS moved from leading with a ‘net of inter-sector transactions’ measure to a gross measure in its industry weightings. A gross measure includes the sales within two manufacturers in the same sector while a net measure excludes these transactions. There is a trade-off between these measures. The IMF PPI manual recommends using net measures as it avoids the problem of double counting transactions in the production process (for example, when one industry’s output is another’s input). However, the manual recognises that the gross measure is recommended over a net measure for deflating the sales revenue of industries as, by definition, the sales revenue of industries is a gross measure. As part of the user consultations for the methods changes, ONS found that users preferred a gross measure of inflation and that users of the net series were not fully aware of the calculation methods or what the net series was articulating. Publishing a single headline input index on a gross measure is a good example of where ONS has balanced international best practice against the needs of UK users.

Removal of duty

2.5.7 ONS removed duty from its headline PPIs. According to the IMF PPI Manual, it is international best practice to capture basic prices, as the valuation basis used is important in capturing the revenue received or costs faced by manufacturers. Basic prices are the amount received by the manufacturer from the purchaser for a unit of a good or service produced minus any tax, plus any subsidy[7]. Basic prices are preferred over other valuation methods as they capture the per-unit revenue received by the manufacturer and filter out distortive effects such as fiscal policy. Removal of duty from the headline index ensures that the valuation basis for PPIs is consistent with international best practice and therefore improves international comparability.

Index construction methods

2.5.8 ONS uses the Laspeyres-Lowe index formula to compile the PPIs (see Annex D for further details about index formulae). The international statistical community considers the Laspeyres-Lowe formula suitable for compiling PPI; it is used by many Organisation for Economic Co-operation and Development (OECD) member countries. The Laspeyres-Lowe index could be subject to extensive bias if the indices weights are not updated frequently enough. However, as ONS is rebasing its indices annually due to chain-linking, the impact of this bias is largely mitigated.

2.5.9 ONS produces PPI in two ways: by economic activity and by type of goods. A third internationally recognised way of producing PPIs is by stage of processing (raw materials, intermediate goods, and finished goods)[8]. Only some non-EU OECD countries such as Australia publish PPIs that have been produced using this method, so the UK’s PPIs are not directly comparable to the PPIs of those countries. This is a limitation of PPIs; because of such differences in methods, they are less internationally comparable than other prices indices (consumer and housing price indices).

Imputation methods should be reviewed

2.5.10 Price data are imputed when price data have not been received from manufacturers. The falling response rates for all surveys (discussed earlier) mean that the business prices team has less data to compile the indices from. As a result, the team is carrying out far more imputation than it used to before the pandemic, which it told us can lead to less-representative trends in price changes. Price data are also imputed for products for which price data are no longer collected because the product is no longer produced, but which has not yet been removed from the sample (so still has a weight).

2.5.11 The business prices team told us it sees the impact of imputation as one the main threats to the quality of PPIs. The falling response rates reduce the robustness of imputed values as they are based on a smaller selection of data. The potential negative impact of inaccurate imputation is magnified by the increase in sample attrition (also discussed earlier), as products which must be imputed represent a greater proportion of their indices. With increased reliance on imputation on increasingly smaller subsets of price data, it is important to regularly review the imputation methods.

Adjusting for the changing quality of products

2.5.12 The aim of PPIs is to measure the price change of products of fixed quality over time, but the quality of products may change over time. For example, as the speed of computer processors increases, buyers of processors will receive a higher quality product[9]. When constructing PPIs, statistics producers must adjust their indices for quality changes in products to ensure that they measure only pure price changes. Not properly adjusting for quality changes may lead to biased indices which distort price changes.

2.5.13 The IMF PPI manual outlines two types of methods for quality adjusting:

• Explicit methods: these methods use external information to quantify the quality difference between the old and replacement product. The methods estimate the extent of the quality and pure price changes with less reliance on assumptions.
• Implicit methods: a measurement technique is used to compare the old product with its replacement, so that the extent of the quality and pure price change is determined by the assumptions of the method chosen.

2.5.14 Explicit methods are generally viewed as more reliable but are more resource intensive to implement. The 2016 Independent Review of UK Economic Statistics identified that between 3-9% of the UK’s PPIs were explicitly quality adjusted to some extent. ONS has since stopped explicitly quality adjusting the PPIs as it was too resource intensive and required too much manual intervention. The business prices team told us it now only uses an implicit method of quality adjustment known as chaining, which assumes that the price difference between the previous and replacement product in the same period, are solely due to quality differences[10]. ONS could be more transparent about the methods it is using to account for changes in the quality of products and the effect on the estimates of applying the method.

2.5.15 Some explicit quality adjustment is carried out by ONS’s deflator team (the team responsible for researching, developing and improving the deflators used), to prepare the indices for use as a deflator in National Accounts. The computer hardware and peripheral equipment PPI is explicitly quality adjusted as it is inadequate for deflation in its non-adjusted state. While it is good that ONS is explicitly quality adjusting this index, it may be more efficient if it was quality adjusted by the business prices It would also improve coherence by ensuring that all users of the PPI data have access to the same quality data.

2.6 User engagement

2.6.1 The business prices team’s main forum for engaging with users about quality and methods is ONS’s cross-government business prices group, which includes key stakeholders from HM Treasury, the Bank of England, the Department for Business and Trade, and other bodies. There is also a cross-government inflation analyst group, which includes a broader range of government users and meets quarterly to discuss price statistics.

2.6.2 The business prices team has a close working relationship with primary users within ONS. For instance, it has fortnightly meetings with the ONS deflator team to discuss any issues arising from the use of PPIs across ONS economic statistics teams, including the trade statistics team. In addition, the business prices team has joint steering group meetings with National Accounts teams several times a year to discuss their PPI requirements.

2.6.3 ONS consulted users extensively about the recent methods changes. The business prices team ran two user consultations (in 2017 and 2019) to gather users’ views on the proposed changes and how they would affect their use of the statistics.

2.6.4 We asked a range of users covering the three main uses of PPI (see 1.3.1) for their views on the quality of the PPIs. Users outside ONS who use PPIs as the basis for indexing prices in contracts told us that the quality meets their needs and that they are satisfied with the level of granularity in the indices, for example, data at lower levels of industry and product level. Users who use the PPIs as an early economic indicator for forecasting inflation also said they are satisfied with the quality, in particular with the timeliness and relevance of the data.

2.6.5 The business prices team responded positively to user feedback during this assessment. Several users that we spoke to requested that ONS publish the weights used to compile the indices, as these are useful for forecasting the movement of indices. ONS had not published the weights used to compile the indices since 2020. We mentioned this to the business prices team and from March 2023, it started to publish the weights of the product groupings as part of the statistical bulletin.

2.6.6 The business prices team handled the series of errors very well. It was transparent with users about the nature of the series of errors and the scale and impact of the resulting corrections. For example, it issued a correction notice about the diesel fuel weight error, and a further notice about the cancellation of the statistics that highlighted the table error and the mapper error (see Annex B). Users told us they welcomed ONS’s openness and advance notice about the errors, despite the inconvenience caused by the cancellation of the December 2022 publication.

2.6.7 Currently, ONS publishes a large number of indices (1,540 PPI, EPI and IPI indices), most of which are lower-level indices. Some of these lower-level indices are likely to be poor quality due to the small sample sizes (discussed earlier), and some are likely to be unused. The business prices team should rationalise the number of indices it produces, focusing on producing high quality indices that meet user needs. This needs to be done in the context of a longer-term, wider review of the use of alternative data sources to produce PPIs.

2.7 Transparency about quality

2.7.1 ONS publishes a Quality and Methodology Information (QMI) report for the PPIs. The report gives a brief overview of data sources and methods used to compile the statistics, their strengths and limitations, and the QA process. It discusses all the key quality dimensions (relevance, coherence, etc) and highlights some limitations, such as the limited coverage of the EPI and IPI surveys.

2.7.2 However, ONS does not publish information about several important areas of quality, which means that users do not have a complete view of the quality of the statistics. In particular, it does not explain:

• Statutory surveys. The QMI report contains no information about the sample design of the PPI, EPI and IPI surveys, even though these are the main source of price data used to compile the statistics. It does not describe the issues with sample attrition and response rate highlighted in this report (see 2.2) and does not discuss the impact of the economic shocks on data collection, such as the COVID-19 pandemic and the war in Ukraine.
• Administrative data sources. The QMI report contains no information about the quality of the various administrative data sources used, how ONS reviews the suitability of those sources for producing the statistics, and how ONS (and the data suppliers) assures the quality of the data (see 2.4.17).
• Systems. The QMI report contains no information about the Ingres production system, including the challenges and limitations of using the system for collating and processing data, and how this might impact the quality of the PPIs (see 2.3).

2.7.3 In addition, ONS has not updated information about quality regularly – the QMI report was last updated in November 2020 – which means that some information is no longer relevant and does not reflect the current quality of the statistics. In some cases, it gives users a misleading picture of quality.

• Quality adjustment. The language used in the QMI report implies that ONS explicitly adjusts for the quality change of products. It only explicitly quality adjusts one index, and this is done by ONS’s deflator team rather than the business prices team (see 2.5.15), so these adjustments are currently not included in the published PPIs.
• Quality labelling. Until November 2020, PPIs were published with quality classifiers that identified indices with low product coverage and encouraged users to treat these with caution. ONS stopped publishing the classifiers because they were not in the scope of the transition to the DAP platform. The QMI report and metadata state that ONS still publishes classifiers.
• Standard errors. The QMI report states that standard errors for PPIs are published annually (as a table alongside the statistical bulletin), but they were last published in March 2019.
• Quality assurance. ONS changed the ‘dubious’ and ‘incredible’ validation test thresholds to manage the workload of the data collection team (see 2.4.3). The QMI report does not explain these changes.

2.7.4 Up-to-date information about the data sources, methodology and QA process is important to enhance trustworthiness, give users a better understanding of the quality of the statistics, and support appropriate use and interpretation of the statistics.

2.7.5 We welcome the level of transparency about the recent method change (see 2.5). ONS published a series of articles about the changes and their impacts on the statistics. One article outlined the broad changes to methods and sources. While this article provided details and assurance about the implemented methods, it could have been clearer about why the changes were made, especially in terms of meeting international best practice, and how they improved the quality of the statistics. For example, users would benefit from a clearer explanation of the benefits of moving from net to gross measures of industry weightings.

2.7.6 ONS also produced a technical article which outlined the formulae and processes behind chain-linking. It contains helpful visualisations and intuitive explanations that are accessible to a less technical audience. Another article explained the changes to the weighting of the headline indices due to structural variations in the economy and changes to the methodology and sources used to produce the weights. This article was helpful in differentiating the sources of variation in weights and reduced the uncertainty around the impact of the methodology.

2.7.7 While the level of detail in these articles is helpful and proportionate to the complexity of the methods, the accessibility of this information could be improved by bringing it together in place, and by signposting it more clearly in the QMI report.

2.7.8 It is good practice to highlight uncertainty around the statistics to support interpretation and appropriate use of the statistics. The quality classifiers mentioned earlier used to provide helpful guidance for users about the quality of individual indices. By removing the classifiers, ONS is not being as transparent about quality as it could be; it may give users the misleading impression that the index has no quality issues even if the index is unreliable. ONS still has the information to produce the classifiers but does not publish them; users can still request the classifiers by contacting ONS.

2.7.9 Standard errors are another useful indicator of uncertainty around the indices, but the business prices team also stopped published this information, to focus resource elsewhere. While most of the users we spoke to did not use the previously available information on standard errors, some users thought it might be useful if ONS produced a one-off article that explains the uncertainty around the PPI estimates. We encourage ONS to be clearer about the extent and nature of uncertainty around the indices.

[5] The new thresholds were calculated by modelling the number of price movements that failed the validation tests and resetting them so that, on average, the same volume of price movements failed the tests as with the previous thresholds.

[6] The UK was still a member of the EU when ONS consulted on the changes to methodology.

[7] Eurostat, 2017, Glossary: Basic price, Available from: https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Glossary:Basic_price

[8] See section 7 of the OECD’s Frequently Asked Questions about Producer Price Indices.

[9] If the manufacturer maintains the price despite a quality change, it can be viewed as a fall in price as consumers are now receiving a better quality good for the same price.

[10] The Eurostat (2012) handbook on constructing PPIs, states that “[this] method… should only be used if there is a considerable quality difference between the old and new representative products, and no additional information is available for the application of another quality correction procedure. Applying the chaining quality adjustment procedure will bias the index to show no price change over time”.