UK Statistical System UK Statistics Authority Code of Practice
Assessment Report

Assessment Report: Benefit statistics

This assessment report was produced during the COVID-19 pandemic which has seen dramatic shocks to the labour market and placed increased pressure on the welfare system. The Department for Work and Pensions (DWP) has continued to produce its statistics despite significant resource pressures, and we highlight the impact of COVID-19 on these statistics where relevant in this report.

Key Findings

Our findings point to a lot of untapped potential in DWP’s benefit statistics. We found a strong need for DWP to engage more proactively with users to understand their needs and ensure the statistics on legacy benefits keep pace with statistics on Universal Credit.

COVID-19 has brought to light users’ interest in information on the characteristics of individuals or households claiming benefits. Solving these data gaps would aid understanding of which groups have been most impacted by the pandemic so that services and policies can be targeted effectively.

Information about the quality of the statistics does not support users’ understanding of the strengths and limitations of the statistics and risks the data being misinterpreted. Signposting between Stat-Xplore (DWP’s interactive tabulation tool) and the DWP landing pages for the statistics needs to be improved to ensure users are equipped to interpret and use the statistics appropriately.

Recommendations

We identified several actions for DWP to further enhance the public value, quality and trustworthiness of the DWP benefit statistics, as described in chapters one to three of this report. Fulfilling the requirements of this assessment will ensure that these statistics can continue to be designated as National Statistics.

Assessment Report: Statistics on Prescription Cost Analysis: England

In 2020, we fully assessed the NHS Business Service Authority’s (NHSBSA) Prescription Cost Analysis (PCA): England statistics against the Code of Practice for Statistics. NHSBSA are a new official statistics producer and this assessment was prompted due to a change in publication ownership from NHS Digital to NHSBSA.

Key Findings

The PCA is a valued, long-standing and trusted source of information used by government departments, the pharmaceutical industry, academics and professional bodies. The statistics enable users to understand the cost and demand for prescription items (drugs, appliances or dressings) in England.

The main focus of our requirements is around improving the public value of the statistics to meet the needs of users through a number of ways. Encouragingly, the team at NHSBSA has a number of user engagement activities planned to enhance and develop its understanding of the uses and users of the PCA. As part of this, we expect to see collaboration with pharmaceutical experts and clinicians to improve the level of insight, liaison with statisticians in the devolved administrations to improve the comparability and coherence of PCA across the UK and improve explanation and signposting between existing PCA data sources at NHSBSA.

The statistics are robust, with an enhanced level of quality assurance. NHSBSA have plans to revise and consult on the methods used to produce the PCA. We recommend that they collaborate with other experts in the UK to enhance the comparability and coherence of the statistics.

Finally, as a new official statistics producer, the NHSBSA has to improve some elements of trustworthiness to demonstrate full compliance with the Code. More regular contact with the Head of Profession at the Department of Health and Social Care (DHSC) will ensure that the Lead Official at NHSBSA is appropriately supported in their role. There were also wider concerns raised by users around the clarity of the process to request data held by NHSBSA. The NHSBSA should ensure that the process is clearly defined and understood by users. FoI requests should also be reviewed periodically with a view of publishing more routine data.

Recommendations

The PCA statistics can maintain National Statistics designation once we have confirmed that the improvements outlined in the six requirements set out in the report have been made. NHSBSA have committed to publish an action plan in November 2020, setting out how they plan to meet each of our requirements. They will report back to us by the end of February 2021.

Related Links:

Ed Humpherson to Michael Cole: Assessment of Prescription Cost Analysis: England statistics

Nina Monckton to Ed Humperson: Requesting assessment of the prescription cost analysis

Ed Humperson to Nina Monckton: Assessment of Prescription Cost Analysis

Assessment Report: Estimates of Station Usage (produced by the Office of Rail and Road)

Judgement on National Statistics Status

  • Estimates of station usage provide a valuable, trusted source of information about the use of the rail network across Great Britain (GB). They are used by a broad range of people, who appreciate the granularity of the data and the ability to use these statistics for further analysis.
  • We identified several ways in which the Office of Rail and Road (ORR) should develop the statistics in order to meet the highest standards of the Code of Practice for Statistics and achieve National Statistics status. These are described in chapters one to three of this report. As we identified the improvements required during the assessment, ORR has made great progress towards addressing them. ORR has committed to further actions which will be carried out before the next release of the statistics in December 2020.
  • Once ORR demonstrates that these steps have been taken, we will recommend that the UK Statistics Authority designate estimates of station usage as National Statistics.

 

Key Findings

Quality

  • One of the main strengths of estimates of station usage is that they provide a data series going back to 1997/98. This means that users can explore trends over time and combine the data with their own local knowledge to understand the impact of infrastructure projects or changes to the usage of the rail network.
  • Estimates of station usage are based on appropriate data and methods. The primary source for these statistics is ticket sales data. This is the best available source without fully gated stations or manual passenger counts at every station. ORR works well with its contractor Steer to improve the methodology used to calculate the statistics each year, while balancing the need for consistency to allow users to make comparisons over time.
  • The use of ticket sales data as the primary source for these statistics results in several limitations. Although ORR publishes information about data quality and methods, it should be more detailed, and be available and communicated in a more accessible way to enable people to understand the caveats and use the statistics appropriately. We also consider that existing quality assurance practices carried out during the production process should be developed and put into a structured framework in order to further mitigate the risk of errors.

Public Value

  • Estimates of station usage provide unique information about each of the approximately 2,500 mainline rail stations in GB. This granularity is one of the main reasons that the statistics are of interest to a very broad range to users: they are relevant to anyone no matter where they live. The team at ORR has a good understanding that the statistics are used by a variety of different users. A better understanding of what people do and how they interact with the statistics will allow ORR to prioritise improvements and provide greater insight for users. We encourage ORR to continue its work to explore the underlying data and to work more closely with stakeholders to make use of their local knowledge. Providing contextual information, for example about infrastructure projects or wider issues such as the impact of COVID-19, will support users to understand the statistics.

Trustworthiness

  • ORR is a well trusted and respected producer of official statistics. Consequently, the ORR website is often the first place that users go to find statistics and data about the rail industry. The analytical team at ORR is well managed, responsive to information requests and knowledgeable on issues which impact the GB rail network. ORR currently publishes several National Statistics which were reviewed by OSR via a compliance check in 2019. We found that there are good practices in place to ensure independent decision making and orderly release of ORR’s official statistics.
  • To enhance transparency, in addition to the information on user engagement for its broader set of official statistics, ORR has published an improvement plan for estimates of station usage that covers both short and mid-to-long-term developments. Maintaining this plan will keep users informed of changes, give them the opportunity to shape developments, and allow them to monitor progress against the plan. We would like to see ORR increase direct communication with users outside the rail industry. This will ensure that all users are informed about changes to the publication schedule or revisions to the statistics.

Next Steps

  • ORR is aiming to meet the requirements described in Tables 1, 2 and 3 by its next annual release of the statistics in December 2020. Once ORR reports back to OSR on how it has met the requirements within this report, the UK Statistics Authority will take advice from OSR and decide whether to award the National Statistics designation. We expect to have a decision before the next publication.

Related Links

Lyndsey Melbourne to Ed Humpherson: Requesting assessment of estimates of station usage

Ed Humpherson to Lyndsey Melbourne: Assessment of estimates of station usage

Ed Humpherson to Lyndsey Melbourne, Office of Rail and Road: Assessment of estimates of station usage as National Statistics

Ed Humpherson to Lyndsey Melbourne: Confirmation of National Statistics designation for estimates of station usage

Assessment Report: UK Business Demography Statistics

In 2020, we fully assessed ONS Business Demography statistics, reviewing their compliance against each of the pillars in the Code of Practice for Statistics.

Key Findings

Our view is that ONS should aim for its business demography statistics to be considered key economic indicators. But they are not regarded as such at the moment, because they are not as good or as useful as they should be.

The ONS’s business register – the Inter-Departmental Business Register (IDBR) – holds a wealth of data on the UK’s business population. Some of these are used to produce business demography statistics. The remainder of which, however, remains a largely untapped resource.

In response to the COVID pandemic, ONS in conjunction with Companies House, introduced a weekly indicator of business births and deaths. ONS has also published the first of a quarterly series of experimental business demography statistics, which draw on data from Companies House and the Insolvency Service. These significant innovations present a platform for further development of business demography statistics.

Some required improvements to the statistics rely on significant investment. It is clear that the redirection of funding away from the Statistical Business Register project has hindered ONS’s ambitions to enhance the contribution that the business register makes to economic statistics. Work to develop ONS’s business register should urgently be re-introduced to ensure that users’ needs for business population statistics are met.

Recommendations

In the short term (by the time the next annual statistics are published in November 2020) in order to retain the National Statistics status for these statistics, ONS:

  • must have demonstrated progress in understanding the access difficulties users are experiencing when using and linking IDBR data with data
  • should publish its plans for publishing more timely business demography statistics, and its plans for developing the recently introduced quarterly experimental statistics
  • should publish a narrative covering what ONS already knows about the range of key data quality issues, building on the supporting quality information provided with the new quarterly experimental statistics
  • should publish its plans to restart and resource work to develop its business register

In the longer term, ONS should publish a plan which includes specific actions, deliverables and a timetable by the end of January 2021, that explains how it will address the improvements identified in the report, including plans for reviewing the funding of the Statistical Business Register.

Assessment Report: Northern Ireland Planning Statistics

These statistics provide relevant and trusted information on planning activity in Northern Ireland (NI), with a good overview of the main trends. They enable users to better understand the planning activity performance of NI councils against statutory targets.

In requesting this assessment, the statistics team at the Department for Infrastructure (DfI) is demonstrating its commitment to produce planning statistics that meet the standards required of National Statistics and the Code of Practice for Statistics. We have identified four actions for DfI to address in order to enhance the public value, quality and trustworthiness of the NI planning statistics and to achieve National Statistics status. These are described in chapters one to three of this report.

Once the statistics team demonstrates that these steps have been undertaken OSR will recommend that the Statistics Authority designate the statistics as National Statistics.

Related links:

Assessment Report: National Rail Passenger Survey

This assessment focuses on statistics from the National Rail Passenger Survey (NRPS) which are produced by Transport Focus. Statistics from the NRPS are not currently produced to the standards required of National Statistics and several areas of non-compliance with the Code of Practice for Statistics need to be addressed.

The UK Statistics Authority considers that in their current form statistics from the NRPS have the potential to be misrepresented or used misleadingly. Transport Focus has agreed to publish an action plan setting out its proposals for addressing the assessment’s Requirements by January 2020.

 

Related Links:

Letter: Ed Humpherson to Louise Coward (January 2019)

Statement: ‘The Office for Statistics Regulation calls for action to improve official rail passenger satisfaction measure’

Assessment Report – PSNI Motoring Offence Statistics

Assessment of Motoring Offence Statistics, produced by Police Service of Northern Ireland.

 

Related Links:

Letter – Ed Humpherson to the senior statistician (December 2019)

Response – Request for Assessment: Northern Ireland Motoring Offence statistics

Request for Assessment: Northern Ireland Motoring Offence statistics

Letter – Ed Humpherson to the senior statistician: Designation for Northern Ireland motoring offence statistics (March 2020)

Assessment Report – Purchases Survey Statistics

Understanding what goods and services businesses purchase, and from whom, is essential to building an accurate picture of the economy. ONS uses its Annual Purchases Survey as the basis for estimating the value of goods and services that businesses use as part of their production processes – in the National Accounts called ‘intermediate consumption’.

ONS previously ran a Purchases Inquiry, with the most-recent usable results relating to 2004. The survey was suspended in 2007, but this suspension became permanent. There followed a ten-year period where no data were collected on intermediate consumption patterns, following which ONS introduced the Annual Purchases Survey to collect data from 2015 onwards.

The hiatus meant that ONS was relying on outdated information about purchases at a time when substantial changes are taking place to purchasing patterns and production processes, not least in the light of the global financial crisis in 2008. ONS mitigated the effects of this to some extent by adjusting its modelled estimates of intermediate consumption based on additional data from the Annual Business Survey, HM Treasury and the Ministry for Housing, Communities and Local Government. In addition, ONS boosted the resources dedicated to manual balancing using staff members’ specific knowledge and broad economic experience. Estimates of intermediate consumption also fell short of EU best practice of updating the supply-use information at least every five years.

We recognise the significant challenges and costs that ONS faces to collect purchases data, and the significant work that it has done to date. We also recognise ONS’s ambitions to achieve National Statistics status, which means that the statistics meet the highest standards of public value, are high quality and are produced in a way worthy of trust and comply with all aspects of the Code of Practice for Statistics.

This report documents the findings of our assessment, which has focused on key elements of quality and value.

 

Related Links:

Letter – Ed Humpherson to Jonathan Athow (December 2019)