UK Statistical System UK Statistics Authority Code of Practice
Labour Market

Assessment Report: Pay in the Northern Ireland Civil Service Statistics

Published:
10 December 2021
Last updated:
9 December 2021

Executive Summary

Judgement

These statistics provide relevant and trusted information on pay in the Northern Ireland Civil Service (NICS). They support pay negotiations and equality monitoring for the NICS.

In requesting this assessment, the statistics team at the Northern Ireland Statistics and Research Agency (NISRA) is demonstrating its commitment to produce Pay in the NICS statistics that meet the standards required of National Statistics and the Code of Practice for Statistics.

We judge that these statistics meet the highest standards of the Code and we have not identified any requirements for the statistics to achieve National Statistics status. OSR therefore recommends that the UK Statistics Authority designate the statistics as National Statistics.

 

Back to top

What we found

Statistics on Pay in the Northern Ireland Civil Service (NICS) have been produced annually since 2011/12. The statistics are used widely to understand the distribution and equality of pay in the NICS. Users are invited to feed into the development of the Pay in the NICS statistics through the annual HRCS Customer Satisfaction Survey, which looks to collect feedback on both the statistics and the service provided by the statistics team. The statistics team made several improvements to the 2021 Pay in the NICS bulletin in response to user feedback, some of which we heard in user feedback for this assessment, to improve the clarity and accessibility of the statistics.

Where users have perceived there to be data gaps, the statistics team has been open and receptive to exploring the feasibility of producing these breakdowns. We found a common area of interest for users is the desire to make comparisons of pay for civil servants across the UK. The statistics team showed enthusiasm and willingness to work with its counterparts across the UK to explore the feasibility of filling this data gap to improve the coherence of pay statistics for the UK. We would encourage the statistics team to take forward this idea to understand whether the user need for UK wide comparable statistics on Civil Service pay exists outside of Northern Ireland.

We found that users had no concerns about the quality and methods used to produce the Pay in the NICS statistics. The data suppliers and statistics team have strong relationships which make it easier to identify and respond to errors. The statistics team has demonstrated its understanding about the strength and limitations of the data by publishing a Quality Assurance of Administrative Data (QAAD) report, which details the quality elements of the statistics such as data collection, quality assurance and potential bias.

In 2021, the statistics team has made efficiency improvements to the production process to improve the timeliness of their output, in response to user feedback about the timeliness of the statistics. The team is exploring additional changes to streamline the production process. This includes submitting a proposal to the NISRA Tech Lab, to assess the possibility of automating the production of the bulletin.

The statistics team adopts and exhibits best practice on data governance. We heard from one of the regulator bodies in Northern Ireland that it was so impressed by HRCS’ approach to data governance and confidentiality, that it has taken this best practice and adopted it in its own organisation. This is a great example of sharing best practice and demonstrating trustworthiness in statistics production.

We heard from users that it is valuable to be able to refer to independently produced pay statistics, when talking to senior leaders or trade unions about pay as they are free from political pressure. The statistics team is seen as professional, capable and helpful. The roles are clearly defined and the internal workplan sets out the timeline and responsibilities for publishing the statistics each year, which even includes user engagement activity.

 

 

Back to top
Next »
Download PDF version (158.11 KB)

Assessment Report: The Living Costs and Food Survey

Published:
7 July 2021
Last updated:
7 July 2021

Summary

Why we carried out this review

A household expenditure survey has been conducted each year in the UK since 1957. Since 2008, this has been in the form of the Living Costs and Food Survey (LCF), which collects information on spending patterns and the cost of living that reflect household budgets. It is the most significant survey on household spending in the UK and provides essential information for key social and economic measures including price indices. LCF data are published annually by ONS in Family Spending in the UK, and are also used in the production of other statistical series such as the Retail Prices Index (RPI).

In recent years, there have been several errors in LCF data which have led to errors in the RPI in both 2019 and 2020 through the use of incorrect expenditure weights. In 2020 for example, if the corrected LCF dataset had been used, it would have led to an upward revision of 0.1 percentage points to the published RPI annual growth rate for six months of 2020. In line with its policy not to make revisions, ONS did not revise the time series for the RPI. The errors in LCF data also affected the Office for National Statistics’ (ONS) Family Spending statistics for years 2017-18 and 2018-19, as they are the main output of LCF data.

In line with the Office for Statistics Regulation (OSR) and ONS strategies, we have carried out an assessment of the LCF against the Code of Practice for Statistics. Our review aims to identify opportunities to improve the quality and public value of LCF data. To inform our review, we spoke to a range of statistics producers and users who make use of LCF data, to understand the impact of LCF on other statistics and data.

 

What we found

The LCF data processing system is not fit for purpose. The system is unstable, often producing inconsistent results between processing runs of data. The statistics team has endured challenges with the system and resource it is working with to run the LCF, and it has done remarkably well to keep the LCF afloat in spite of these challenges.

We found that the LCF is highly valued by users and is seen as unique in bringing together data on spending habits with information on the households who are doing the spending. Users we spoke to do not see an alternative approach to collecting this information in the short term that does not involve a survey in some form. The production of the LCF is supported by a steering group made up of statistical teams in ONS and other government departments who use LCF data, as well as external think tanks. The LCF team would like to enhance its engagement with internal and external users, across academia, the private sector and other government departments, to ensure developments best meet user needs.

The COVID-19 pandemic has seen the landscape of user need change with increased demand for timely and granular data. The LCF has been underperforming in terms of sample size and response rate in recent years and this was highlighted as the main limitation of the LCF by users we spoke to. Without an increase to the sample size, the devolved administrations are unable to make use of many of the categories available in the LCF data.

The quality assurance arrangements that the LCF team has in place are generally appropriate for the data and go some way to reduce the risk of errors associated with the processing systems. However, the sample size of LCF can make it difficult to determine whether changes between periods are genuine or not. We heard from users of LCF data in ONS that widening access to the LCF data for the purposes of quality assurance could support the LCF statistics team in identifying errors and would allow it to focus on the pre-processing stages.

In 2016, a National Statistics Quality Review (NSQR) was carried out for the LCF. The NSQR recommended several alternatives that could be explored to improve the response rate and thus the achieved sample size of the LCF. The team has taken forward most of the recommendations with no additional headcount being allocated to progress them. The remaining recommendations remain relevant to improving LCF.

The lack of progress in the use of alternative and administrative data sources in the UK has impacted on the quality, accuracy, and international comparability of the LCF data, a perspective which is seen as important to users to gauge the impact of Brexit and the pandemic on UK households.

 

Requirements and next steps

We have identified several ways the LCF needs to be improved to meet users’ needs and to comply with the highest standards of the Code. Urgently fulfilling the requirements of this assessment is necessary to ensure that the LCF and its outputs continue to be fit for purpose. In order to retain the National Statistics status for Family Spending in the UK, we require ONS to:

a) Demonstrate a positive direction of travel by making some short-term gains by the end of 2021 as follows:

  1. ONS needs to take remedial action to improve the stability of the existing LCF processing system and to develop a new system which meets the needs of users and the staff running the systems.
  2. ONS should enhance its understanding of the value of the statistics by improving its engagement with users, within and outside ONS, to capture a wide range of views and use these to drive its priorities for development. ONS should reflect on the Government Statistical Service’s User Engagement Strategy for Statistics to help determine the best methods for engaging with users.
  3. ONS should provide a mechanism and relevant access for other teams in ONS who make use of LCF data to be able to contribute to the quality assurance of the data.

b) Publish a plan which includes specific actions, deliverables and a timetable by the end of March 2022, that explains how it will address the following strategic improvements:

  1. ONS needs to develop a solution to address user need for more-granular breakdowns of data, so that the devolved administrations and other key users can use the statistics in the ways that they need to for the public good.
  2. ONS needs to invest time and resource to pursuing initiatives to improve the quality and robustness of LCF data. ONS should be open to creative solutions to improve the response rate, such as continuing exploring the use of different short and long form questionnaires/diary, alternative sampling strategies and linking with other data sources, rather than focusing only on increasing the existing sample.

We have also highlighted several considerations for ONS to reflect on as part of its LCF and RPI improvement project.

a) ONS should consider extending the scope of its project work to include input from some of its key external users, such as those in the devolved administrations, where additional intelligence could be gathered on the use and issues faced by the government in its use of LCF data.

b) ONS should consider the management of risks throughout the end-to-end production process as part of the LCF projects medium term work and ambitions.

c) ONS should determine a longer-term solution for the LCF which draws on a broader base of data, international best practice and wider transformation initiatives.

We expect ONS to report back to us every quarter, starting from the end of September 2021, demonstrating its progress against these requirements. We will review the National Statistics designation of Family Spending in the UK at each of these points.

Next »
Download PDF version (262.99 KB)

Assessment of Personal Independence Payment Statistics for Northern Ireland (produced by the Department for Communities, Northern Ireland)

Published:
13 May 2021
Last updated:
14 May 2021

Executive Summary

Judgement on National Statistics Status

These statistics provide relevant and trusted information on Personal Independence Payment (PIP) in Northern Ireland (NI). They enable users to better understand PIP as a relatively new benefit and its role in social security reform.

In requesting this assessment, the statistics team at the Department for Communities (DfC) is demonstrating its commitment to produce PIP statistics that meet the standards required of National Statistics and the Code of Practice for Statistics. We have identified four actions for DfC to address in order to enhance the public value and quality of the NI PIP statistics and to achieve National Statistics status. These are described in chapters one to three of this report.

Once the statistics team demonstrates that these steps have been undertaken, OSR will recommend that the UK Statistics Authority designate the statistics as National Statistics.

Key Findings

Public Value

The PIP statistics are used by a range of users across the advice and charity sectors, as well as within DfC, which is interested in how PIP as a relatively new benefit is performing and its role in social security reform. The key user engagement activity is coordinated by the PIP operations team through a quarterly consultative forum, attended by advice and disability groups. However, we found that users would welcome direct engagement with the statistics team to ensure developments to the statistics are prioritised in line with user need.

The majority of users we spoke to highlighted that the Department for Work and Pensions’ (DWP) PIP statistics contain more breakdowns for Great Britain (GB) than are available for NI, even though the data are sourced from the same system. The statistics team needs to engage more actively with users to communicate its ambition to better align the statistics in NI with what is available for GB. Where data gaps can’t be addressed, this should be communicated clearly to users and reasonable alternatives should be explored.

We found that user interest has begun to shift away from reassessments of individuals who were previously receiving the Disability Living Allowance (DLA) onto PIP, towards understanding award reviews and how these may impact individuals’ entitlements. The statistics team should ensure the statistical bulletin and data tables draw out the relevant insights to help users understand where NI is in its journey of rolling out PIP.

Users we spoke to were positive about the length and presentation of the bulletin. However, we found that accessibility could be improved by better signposting between the bulletin and the supplementary tables as some of the detail can be missed if the underlying data tables are not viewed.

Quality

The statistics team maintains a good working relationship with the PIP operations team and engages with it regularly. The users we spoke to have no concerns with the quality or methodology of the statistics, as the data for the PIP statistics are sourced from the PIP Computer System which represents a 100% population of PIP claimants with a postcode in NI on the reference date.

The statistics team has carried out a self-assessment of its understanding of administrative data against our Quality Assurance of Administrative Data (QAAD) framework. The statistics team should bring out the relevant information from its QAAD self-assessment to expand the methodology and quality notes in the bulletin and tables, to support the appropriate use of the statistics.

The Professional Services Unit (PSU), in which the responsible statisticians for PIP sit, is currently exploring the use of Reproducible Analytical Pipelines (RAP) in the production of its statistics, with a view to improving the quality of the statistics by reducing the risk of human errors associated with its current software packages. Once RAP has been fully implemented, PSU anticipates that this will free up resources, which can be reallocated to address developments requested by users.

The statistics team told us that although it has access to the same data as DWP, there is not currently a process in place for sharing code needed to produce statistics which are comparable with DWP. The statistics team needs to build stronger links with DWP to ensure a common understanding of the quality and priorities of PIP statistics and of the PIP Computer System.

Trustworthiness

The statisticians working on PIP are well established and users we spoke to said the team is always helpful and knowledgeable when responding to their queries. DfC has a dedicated webpage about its statistics protocols and compliance with the Code of Practice for Statistics and users in DfC told us that pre-release access is controlled well and that the statistics team are not influenced by senior authority within the organisation.

The statistics team could enhance its trustworthiness by being open about its statistical development plans and its progress towards achieving these developments, even if the timescales are uncertain. Whilst PSU does not routinely publish a development plan, some users we spoke to felt that the lack of transparency around development priorities can lead to data gaps being perceived as DfC withholding information.

Next Steps

We expect the DfC statistics team to report back to us by September 2021 outlining the steps that it has taken to address the requirements. The UK Statistics Authority will take advice from OSR based on the evidence received and decide whether to award the National Statistics designation.

 

 

Back to top
Next »
Download PDF version (191.02 KB)

Assessment Report: Benefit statistics

Published:
18 November 2020
Last updated:
17 November 2020

Assessment Report: UK employment and jobs statistics

Published:
26 March 2020
Last updated:
26 March 2020

Assessment Report: UK employment and jobs statistics

Published:
26 March 2020
Last updated:
20 July 2020

Statistics on Sickness Absence in the Northern Ireland Civil Service

Published:
30 April 2019
Last updated:
7 August 2019
Compliance Review: The quality of Labour Force Survey estimates produced by the Office for National Statistics

Compliance Review: The quality of Labour Force Survey estimates produced by the Office for National Statistics

Published:
23 June 2017
Last updated:
17 August 2017

‘Young people not in education, employment or training, UK’, ‘Focus on property Crime, England and Wales’, ‘Index of private housing rental prices in Great Britain’

Published:
6 June 2017
Last updated:
6 June 2017

Statistics on Outcomes of Work Capability Assessments

Published:
2 March 2017
Last updated:
2 March 2017