Ed Humpherson to Sean Whellams: Coronavirus Job Retention Scheme and Self-Employment Income Support Scheme statistics

Dear Sean

Coronavirus Job Retention Scheme (CJRS) and Self-Employment Income Support Scheme (SEISS) statistics

I am writing to endorse the approach you have taken to produce statistics about the Coronavirus Job Retention Scheme (CJRS) and the Self-Employment Income Support Scheme (SEISS). These statistics estimate the number and value of claims made to the CJRS and to the SEISS administered by HM Revenue and Customs (HMRC). I would like to congratulate everyone involved for their work to produce these important statistics.

My team has conducted a rapid regulatory review of the published information. We have reviewed the extent to which they have been produced in accordance with the Code of Practice’s Trustworthiness, Quality and Value pillars, while taking account of the pressures you and your teams have faced to deliver timely statistics about an important topic. A summary of our findings is set out below and we have already provided more detailed feedback to you.

Value

  • These statistics and data help us understand the scale to which employers, employees and the self-employed have relied upon these schemes. There has been considerable press interest in the UK and in the devolved countries in the estimates particularly highlighting the take-up and value of the schemes at UK, regional and local authority levels
  • The releases include information such as glossaries, background information about the schemes, the strengths and limitations of the statistics and links to other related statistics which help users to use and interpret the statistics. There are good breakdowns for each set of statistics which draw from the datasets available. The presentation of charts engages readers’ interests and where data are ranked it’s easy to see where or what is highest and lowest
  • Your approach to engaging with devolved countries’ statisticians about improvements that they would like to see in the statistics is praiseworthy. You have set out in the CJRS bulletin that there’s scope to include additional analysis in further releases. For example, you have mentioned including breakdowns of the CJRS claims by age and sex of employees. This engagement helps the statistics meet users’ needs as well as they can, and provide a strong foundation for future developments
  • These statistics have been developed from management information that HMRC was tweeting on a daily basis following the launch of the CJRS on 20 April. Releasing the management information so promptly about the CJRS including numbers of unique applicants, the number of furloughed jobs and the value of claims was a good example of transparency. It is particularly good that your team went on from releasing the management information to match data from its Real Time Information (RTI) system with Coronavirus Job Retention Scheme data to produce breakdowns of claims providing even more insight in the official statistics now being produced
  • The estimates of take-up of the SEISS relative to the estimated number of people eligible is helpful in making comparisons, we encourage you to do something similar for the CJRS, using a suitable denominator to enable easier geographic comparisons of the impact of furlough
    • Your decision to publish sub-regional statistics will helpfully provide further insights from the wealth of data in tables accompanying the main statistics. We suspect that as part of managing the recovery from the pandemic at a local level, users would benefit from further sub-UK insights. For example, in relation to SEISS there might be benefit to users in providing some background on the plausible reasons for significantly lower take-up rates of SEISS in some of the London local authorities and significantly higher than average take-up in some south Wales local authorities

Quality

  • The rationale for the assumptions made are well explained and reasonable. For example, that PAYE schemes provide a reasonable proxy for employers for the purposes of the release
  • To allocate scheme applicants into regions and countries you have used the best available address data, for example in CJRS statistics, people’s residential addresses. There are many reasons why someone’s residential location would be in a different region or country to where their job is. CJRS bulletin already provides an explanation about the use of residential addresses in section 4 of the CJRS bulletin but it may be worth repeating the explanation in section 5 for people who go directly to that section. Similarly, in the SEISS statistics the claimant’s self-assessment address is used which may be different to their residential address, for example the self-assessment address could be the claimant’s accountant’s or solicitor’s address. Using a self-assessment address might result in some distortions in the figures. You might urge greater caution on placing too much weight on statistics on eligibility and take-up of SEISS at Local Authority and Parliamentary Constituency levels due to factors such as this
  • There is interest in the numbers of people eligible for SEISS, for example the House of Commons Treasury Select Committee and the Institute for Fiscal Studies. IFS for example suggested that the numbers of people eligible for SEISS are around 3.12m whereas the estimated numbers in SEISS are given as 3.4m. We consider that acknowledging and explaining differences, where possible, would offer further reassurance to your users about the quality of the HMRC estimates of totals numbers eligible for SEISS support

Trustworthiness

  • You have pre-announced through the HMRC forward release calendar the release date for these statistics and has given a specific release date for the next release at least four weeks in advance

We look forward to seeing these statistics develop as circumstances change. As set out in our guidance on changes to statistical outputs you can include a statement to accompany information about these statistics such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”

I am copying this letter to Jane Whittaker, Director of HMRC’s Knowledge, Analysis and Intelligence directorate.

Yours sincerely

 

Ed Humpherson

Director General for Regulation

Ed Humpherson to Jim Harra: Strengthening the quality of HMRC’s official statistics

Dear Jim

 Strengthening the quality of HMRC’s official statistics

Today we publish the findings of our review of the quality of HMRC’s official statistics. HMRC invited us to carry out this review after identifying a significant error in its Corporation Tax receipt statistics. The aim of the review was to provide an independent assessment of HMRC’s quality management approach for official statistics and to identify improvements.

HMRC produces many official statistics and most of the time it does this without issues or errors. We found much to admire in how the Knowledge, Analysis and Intelligence directorate (KAI) manages the quality of HMRC’s official statistics, but several areas where it could strengthen its approach. We make nine recommendations to address these issues. The most immediate recommendations relate to maintaining effective relationships between those producing official statistics and other parts of HMRC they receive data from; understanding data journeys; and quality assurance of data, analysis and publications. Further recommendations focus on strengthening leadership of quality management; building more effective knowledge sharing about quality; and improving information published about the quality of official statistics.

Our report notes that, like many other large government departments, most of HMRC’s IT systems have been designed for operational delivery, rather than for statistical analysis, and have been in place for many years, so may not have the functionality or flexibility of newer systems. This means it can be difficult for analysts in KAI to ensure that they have access to the right data, in the correct format, for their analytical work. Long-term improvements to quality management will rely in part on IT system and data owners in HMRC understanding and prioritising the Department’s statistical requirements alongside operational considerations. Investment in underlying IT systems will also be important to enable HMRC to continue to confidently exploit the value of the data it holds.

By inviting this review, HMRC has taken a proactive approach to strengthening the quality of its official statistics, and I’d like to thank HMRC for its open and positive engagement with my team throughout. I look forward to future engagement with HMRC as it takes steps to strengthen the quality management and assurance of its official statistics. I am copying this letter to Ruth Stanier, Director General for Customer Strategy and Tax Design, Jane Whittaker, Director of HMRC’s Knowledge, Analysis and Intelligence directorate, and Sean Whellams, HMRC Head of Profession for Statistics.

Yours sincerely

Ed Humpherson

Director General for Regulation

 

Related links:

Ed Humpherson to Ruth Stanier: Strengthening the quality of HMRC’s official statistics

Ed Humpherson to Sean Whellams: Review of HMRC statistical quality management

Review – Strengthening the quality of HMRC’s official statistics

Ed Humpherson to Ruth Stanier: Strengthening the quality of HMRC’s official statistics

Dear Ruth

Strengthening the quality of HMRC’s official statistics

Today we publish the findings of our review of the principles and processes underpinning the quality of HMRC’s official statistics.

We welcome HMRC inviting OSR to carry out this review after HMRC identified a significant error in its Corporation Tax receipt statistics, which affected the period from April 2011 to July 2019. The aim of the review was to provide an independent assessment of HMRC’s quality management approach for official statistics and to identify improvements that lower the risk of errors, especially errors that have a large impact on the statistics and their interpretation by users.

HMRC produces many official statistics and most of the time it does this without issues or errors. We found much to admire in the way that HMRC manages the quality of its statistics, but several areas where it could strengthen its approach: we make nine recommendations to address these issues. Our findings highlight the issues that big operational departments working with administrative data can face with respect to quality and will ring true for other Government departments.

The most immediate of our recommendations relate to establishing and maintaining effective relationships between those producing official statistics in HMRC and other parts of HMRC they receive data from; understanding data journeys; and quality assurance of data, analysis and publications. Further recommendations focus on strengthening the leadership and oversight of quality management; building more effective routes for knowledge sharing about quality; and improving the information HMRC publishes about the quality of its official statistics. Implementing these recommendations will strengthen the quality culture in HMRC and make sure that the structure and tools are in place to manage statistical quality effectively, minimising risk.

Our report also notes that, like many other large government departments, most of HMRC’s IT systems, from which the administrative data that underpins almost all of HMRC’s official statistics, and other analysis, are drawn, have been designed for operational delivery, rather than for statistical analysis, and have been in place for many years, so may not have the same functionality or flexibility as newer IT systems. Along with the recommendations about statistical quality management, investment in underlying IT systems to ensure they keep pace with these modern expectations (while also being of wider benefit to HMRC in fulfilling its duty of collecting taxes), will be important to enable HMRC to continue to confidently exploit the value of the data it holds. I have written separately to Jim Harra, First Permanent Secretary and Chief Executive for HMRC, about the need for this wider investment.

I’d like to thank HMRC for its open and positive engagement with my team throughout this review, and I look forward to future engagement with HMRC as it takes steps to strengthen the quality management and assurance of its official statistics. I am copying this letter to Jane Whittaker, Director of HMRC’s Knowledge, Analysis and Intelligence directorate, and Sean Whellams, HMRC Head of Profession for Statistics.

Yours sincerely

Ed Humpherson

Director General for Regulation

 

Related links:

Ed Humpherson to Jim Harra: Strengthening the quality of HMRC’s official statistics

Ed Humpherson to Sean Whellams: Review of HMRC statistical quality management

Review – Strengthening the quality of HMRC’s official statistics

Review of HMRC statistical quality management

Dear Sean

Review of HMRC statistical quality management

I am writing to thank you for inviting the Office for Statistics Regulation to carry out a review of the quality assurance principles and processes underpinning HMRC’s published statistics. The aim of our review is to provide an independent assessment of HMRC’s quality management approach and to identify improvements in the way it produces official statistics.

This review was initiated after HMRC identified a significant error in published Corporation Tax receipt statistics, which affected the period from April 2011 to July 2019: estimates for this period were corrected in your Corporation Tax statistics publication on 24 September. We note that this was a statistical reporting error, which does not affect the amount of tax paid by companies and received by HMRC, or the figures reported in HMRC’s accounts; and that the overall trend of increasing onshore Corporation Tax receipts remains the same following this revision.

We know that HMRC very much regrets this error, and we are pleased to help you take a proactive approach to improving quality management processes across your statistics. We intend that our final recommendations will be forward looking and focus on improvements HMRC can make to help avoid issues in the future. The review will consider:

  • existing quality assurance (QA) processes and current improvement plans within the Knowledge, Analysis and Intelligence (KAI) directorate of HMRC;
  • how QA of official statistics works in practice within KAI;
  • the processes in place for handling errors and managing risk during statistical production;
  • the levels of risk of different sources of error in KAI’s statistical production processes.

We welcome your approach to us to undertake this review and look forward to engaging with you and your teams over the coming months. We aim to publish our initial findings and recommendations at the start of next year.

Yours sincerely

Ed Humpherson
Director General for Regulation

Scotland’s Block Grant Adjustment

Dear Tom

As you are aware, the Office for Statistics Regulation was contacted about the use of HMRC statistics on Scottish Income tax receipts in 2017-18 in an article by Liz Truss MP, in the Scottish Daily Mail on 19 July. This article followed the publication of income tax reconciliations for Scotland, which were published by HMRC for the first time on 18 July.

The basis of the article was HM Treasury’s press release “Scottish Income tax shortfall offset by UK funding”1. The press release states that “Scotland’s economy grew more slowly than the rest of the UK, hitting tax receipts and leaving the Scottish Government with a shortfall in funding”.

While it is correct that Scotland’s economy grew more slowly than the rest of the UK it is incorrect to suggest this is the primary reason for reduced Scottish income tax revenues and therefore the main reason for the adjustment to the block grant funded by UK Government. The main causes are the availability of more accurate figures on the income tax base in Scotland and divergences in the forecasted income tax revenues between the Rest of the UK and those for Scotland.

While the joint fiscal framework is a relatively complex, multi-stage process of transfers and balancing, the principal reasons for changes to Scottish income tax revenues and Block Grant Adjustment revisions after reconciliations are amenable to clear and unambiguous explanation. We recommend that future HM Treasury press statements based on these statistics provide better explanations of the causes of changes to the Scottish income tax revenues and the associated Block Grant Adjustment. This will reduce the risk that users draw misleading conclusions from the statistics and the statements that draw on them.

We recognise that this is the first time that HMRC has published these statistics and been subject to associated media scrutiny. While it is likely future reconciliations will not be as large, we consider that to fully inform the public it would be helpful to present this framework in line with our Code of Practice for Statistics, considering how the statistics can be best presented in a clear and unambiguous way.

I am copying this letter to Sean Whellams, Head of Profession for Statistics at HMRC, and Alyson Stafford, Director General at the Scottish Exchequer.

Yours sincerely

Mary Gregory
Deputy Director for Regulation

 

Related Links:

Sir David Norgrove to Derek Mackay MSP (August 2019)