Quality

Introduction

Quality means that the statistics and numerical information represent the best available estimate of what they aim to measure at a particular point in time and are not materially misleading.

Quality is analytical in nature and is a product of the professional judgements made in the specification, collection, aggregation, processing, analysis, and dissemination of data.

The Statistics

ONS delivers one of the richest sets of productivity data in terms of range and granularity in the world and an international comparison study bears this out. This includes labour productivity at both the whole-economy level, as well as disaggregated by industry and by region. Output per hour is its preferred measure of labour productivity and estimates are presented primarily in the form of indices and growth rates. Output per hour is a very useful measure at the whole economy level but becomes more unreliable below regional level in the UK where output per worker may be a more reliable indicator for sub regional areas such as Local Authorities.

Going beyond labour productivity, ONS now publishes annual and quarterly estimates of multi-factor productivity (MFP) for both the whole economy and by industry. For any given change in output, MFP measures the amount that cannot be accounted for by changes in inputs of quality-adjusted labour and capital. This means it is a theoretically superior measure to labour productivity, as it takes account of the effects of labour and capital inputs. MFP is often described as the measure of our ignorance, since as a residual measure, if one fails to take account of all inputs in the production process, MFP can reflect missing inputs. A decline in MFP growth has been found to be an important feature of the productivity puzzle.

Data Sources and Methods

Labour Productivity statistics are derived by dividing measures of output by some measure of labour input. The output measures used in the productivity statistics are taken from the Quarterly National Accounts – Gross Value Added (Output) estimates. Output measures are of real (inflation-adjusted) gross value added (GVA), for the whole economy, and sub-sectors of the economy for which productivity statistics are produced. Labour inputs for productivity statistics are primarily measured as hours worked, but also as numbers of workers or numbers of jobs. All these inputs use data from the Labour Force Survey (LFS) to some degree, which is a survey of households that collects information about employment on a headcount basis. Hours worked are available by sub-sector and by region derived from estimates of average hours.

Unit labour costs and unit wage costs are also published in the Labour Productivity Statistical Bulletin. These series measure the labour costs incurred in producing one unit of output. Although not a direct measure of productivity, an inverse relationship between these measures and productivity tends to be observed: the higher the productivity of a worker, the lower the cost of labour per unit of output, and vice versa.

ONS publishes the Statistical First Release of UK Labour Productivity estimates around about 45 days after the end of the quarter with more detailed and revised statistics published around 95 days after the end of the quarter. The latter are designated as National Statistics.

MFP estimates are experimental statistics constructed using inputs from the Volume Index of Capital Services (VICS), quality-adjusted labour input (QALI) and output data from quarterly national accounts. MFP covers the UK market sector, which means that general government and non-profit institutions serving households are excluded from these estimates. Like labour productivity, MFP is presented in index form.

To estimate MFP growth over time, changes to both the quantity and quality of labour input in the economy need to be accounted for. The former is measured by the number of hours people work in the period, which is the same measure used for labour productivity estimates. The latter is calculated by accounting for changes to the quality and composition of that labour over time. Both are combined to estimate quality-adjusted labour input (QALI) figures. Capital inputs are estimated by calculating the volume of capital services that are employed by the economy in a period, from the existing capital stock. These estimates form the Volume Index of Capital Services (VICS) figures. Quarterly MFP growth estimates cover 10 industries of the UK market sector with annual estimates further disaggregated to 19 industries.

Findings

Data Sources

Hours worked are derived from ONS’s Labour Force Survey (LFS), in conjunction with other data sources including the Business Register and Employment Survey (BRES). For many years, and well before the COVID-19 pandemic, there have been known issues about LFS response rates that had been steadily declining. LFS response rates fell from around 55% in 2009 to under 40% ten years later. OSR has previously commented on the declining response rate of the LFS for example in our 2020 assessment of employment and jobs statistics. ONS acknowledges the declining response rate as an ongoing issue. We make no specific requirement here as we required ONS in our 2020 assessment report on employment and jobs statistics to address the issue and share any relevant information with users.

The COVID-19 pandemic has had a dramatic impact on LFS data collection and ONS has instigated mitigating measures in response to this. An impact of COVID-19 has been changes to the labour market. The imputation method used for the LFS was not designed to deal with the changes experienced in the labour market in recent months. Consequently, ONS has been working on a new experimental imputation methodology. ONS has not been possible to fully integrate this methodology into its labour market estimates yet, but early indications suggest that there is little impact from the use of existing methodology on the headline measures of employment, unemployment and economic inactivity (less than 0.3 percentage points)there is little impact from the use of existing methodology on total hours, with measures relating to total hours understating the actual number of hours worked by approximately 0.2%.

This early work suggests that estimates of total hours worked from the LFS remain robust. ONS says it intends to provide more information in later releases of labour market estimates as this work develops.

There are known issues about the measurement of GDP during the pandemic. Economic impacts of COVID-19 have led to record declines in Gross Domestic Product (GDP) in advanced economies in 2020. International comparisons suggest that the UK has experienced amongst the largest contractions in volume GDP amongst the G7 advanced countries. However, international comparisons are complicated by how National Statistical Institutes record public service output. ONS recently published an article explaining how international comparisons of GDP have been affected by the COVID-19 pandemic. We are pleased that ONS is working alongside the Organisation for Economic Co-operation and Development (OECD) and with other National Statistical Institutes (NSIs) to explore this in more detail, and strongly encourage this type of international collaboration.

Sound Methods

Following a fundamental review of the source data as well as investment in new systems delivering greater flexibility and efficient production processes, ONS publishes quarterly estimates of capital services for the whole market sector of the UK economy as well as breakdowns by 19 sectors. These estimates are produced on the same timetable as the Quarterly National accounts and Labour productivity statistics. We welcome ONS taking steps to meet users’ needs for greater detail on capital services and producing more-timely estimates of productivity.

In its MFP estimates, ONS accounts for changes in the composition (or “quality”) of the employed workforce as well as for changes in hours worked through its Quality Adjusted Labour Input (QALI) statistics. QALI weights hours worked by different types of workers by their relative income share (reflecting their contribution to economic production). ONS recently moved publication of its QALI statistics from annual to quarterly. The official estimates of QALI go back to 1970 and are for the whole economy and the market sector. They are broken down into 19 industry groups, by sex (two groups), age (three groups), and educational level (six groups). This represents almost twice as much industry detail in the MFP statistics as previously. We commend ONS on providing greater detail around the adjustment of labour inputs to MFP estimates.

In MFP, capital utilisation rates are assumed to be constant which is a reasonable assumption in long-period productivity analysis. However, during the pandemic this assumption results in a large implausible increase in capital deepening (the amount of capital that a worker can use in an hour). The increase is driven by the drop in the number of hours worked and changing patterns of capital usage across the economy during the pandemic as many offices and factories have remained empty or have been used at a reduced rate compared with previous years. Both factors have caused capital utilisation to fall.

To tackle the issue ONS recently introduced a factor in its model to allow the capital utilisation level to be adjusted. This adjustment is designed to reflect the impact of the pandemic on capital utilisation. ONS produced experimental statistics to complement its core MFP estimates but has said that the new utilisation factor has certain weaknesses. The methodology assumes that, within each industry, all workers use all types of capital in proportion to their hours worked during the lockdown and related restrictions. This has the effect that the decline in hours worked of any worker reduces utilisation of all types of capital in that industry. ONS intends to continue to work on improving its methods of adjusting for capital utilisation and has committed to publishing a research paper later in 2021 setting out a more robust approach to estimating capital utilisation in the UK. ONS’s initiative in introducing this adjustment and its ongoing commitment to improve the methodology in the coming months is commended

In 2018, ONS suspended its International Comparisons of Productivity (ICP) statistics, due to problems with data sources and classifications and methods that were not harmonised with many other countries. ONS recognised that there is an upwards bias in counting the actual hours worked in the UK. The lack of comparable estimates has been a significant weakness in understanding the UK’s progress in addressing the productivity puzzles. ONS intends to reinstate estimates of ICP in 2021, with detailed explanation of the limitations of the comparisons as they are.

ONS is also currently developing a new UK-tailored method for establishing International Comparisons of Productivity (ICP), making use of the full range of sources to find the best estimate for each component adjustment, and to align these as closely as possible with national account concepts. ONS intends to replicate its provisional findings for multiple time periods, with the view that its estimates may ultimately be included in quarterly productivity publications. ONS also aims to identify LFS data sources that require adjustments for the purpose of making international comparisons, for example annual leave, parental leave, and sick leave. These adjustments have been identified as playing a major role in previously inconsistent international comparisons. ONS demonstrate a creative approach here to improving its statistics and data.

In the short term, ONS is also exploring the possibility of producing international comparisons of productivity within ranges of uncertainty. These ranges will represent the likely range of difference between levels of productivity in other countries relative to the UK. They will allow a reader to tell if the gap was likely to be large or small, but it would not be possible to estimate a significance level as in statistical methods. ONS told us that it intends to collaborate with others such as the Organisation for Economic Co-operation and Development (OECD). We strongly encourage ONS to work with the OECD and with other NSIs to understand their measures and data in more detail and to inform the construction of uncertainty bands for ICP estimates. Given the importance of understanding the gap in productivity between the UK and its major trading competitors, we wholeheartedly encourage ONS in taking the various steps planned to re-establish these statistics.

We are aware of the difficulties in presenting these uncertainty bands to users. Given the novelty and complexity of the approach, ONS should talk to users about its plans to communicate international comparisons of UK labour productivity within ranges of uncertainty and test understanding of the estimates as they develop them.

Quality Assurance

There is a raised risk to the quality of the statistics due to the reliance on informal collaborative relationships with ONS colleagues who are data suppliers. The productivity statistics team has not documented with the compiler teams:

• data requirements for statistical purposes
• data transfer process
• arrangements for data protection
• sign-off arrangements by data suppliers
• content specification

The risk is further elevated as there is regular inflow of new staff coming into the productivity statistics team and then moving on quickly due to the ONS staff rotation policy. The specific risk is of a loss of continuity and corporate memory. We believe that the risk is currently well-managed as there is regular engagement between data compliers and the productivity statistics team and compiler team members attend the productivity team’s curiosity sessions.

The Office for Statistics Regulation has produced a Regulatory Standard on the Quality Assurance of Administrative Data (QAAD) for statistical producers. Using the associated QAAD toolkit helps statistical producers decide on the level of assurance required, and put in place appropriate quality assurance measures. While the OSR Regulatory Standard was developed for quality assuring administrative data the accompanying toolkit is helpful in quality assuring any source data, regardless of where the data come from. Using the QAAD toolkit, ONS should consider the level of assurance required, and ensure that its processes to assure the quality of data are appropriate.

Findings and Requirements