Mark Pont to Sean Whellams: Statistics from the Survey of Personal Incomes

Dear Sean

Statistics from the Survey of Personal Incomes

I am writing to you following our review of the HMRC’s Statistics from the Survey of Personal Incomes (SPI). We reviewed the statistics against the Code of Practice for Statistics pillars of trustworthiness, quality and value.

I would like to take this opportunity to thank you and the statistics team for engaging so effectively with us throughout the review process. I am pleased to confirm that these statistics should continue to be designated as National Statistics. We found several positive examples of the way that HMRC produces and presents these statistics, including:

  • the extent to which HMRC has established these statistics as a rich resource answering important questions about people’s incomes and earnings. The statistics cover income and tax liability analyses for individuals in the UK, and helpfully include breakdowns for its constituent countries, other sub-UK geographic areas such as counties, and on individual characteristics of taxpayers, such as age. Understanding trends in income and earnings in our society helps us see how living standards may be changing and allows policy to be developed based on the data
  • the extent to which HMRC wants users of these statistics and data to be at the centre of statistical production. People we contacted who are members of a Devolved Income Tax Working Group found their engagement with HMRC very useful. We also found excellent examples from the Scottish Fiscal Commission (SFC) in which HMRC improved the ways in which it met SFC’s needs from the personal incomes data. In the 2019 edition of its annual Statement of Data Needs, SFC said “our three top priorities for income tax data from HMRC, ….have been delivered. These include ….a significant improvement to the main underlying survey data source we use in our forecast. …We appreciate the ongoing engagement from HMRC, and the work that has gone in to fulfilling our data requests”
  • the quality assurance of the statistics. The quality of statistics can be seriously compromised when analysts do not fully understand the nature and quality of administrative source data they are working with. HMRC statisticians working on statistics from the SPI have a good appreciation of the wider organisational context in which they work. We found well-designed and managed systems and processes, and established relationships between the statisticians and data suppliers and with the Personal Tax modelling team within HMRC
  • clear explanations of the strengths and limitations of the statistics. One of the ways that statistics producers build users’ trust is through explaining clearly the strengths and limitationsof the statistics and data in relation to different uses. HMRC provides comprehensive and detailed supporting information about personal incomes statistics identifying limitations of the methods and their application, explaining these to users. HMRC gives information to users about the underlying data sources, which they can take into consideration when weighing how best to use the data. As estimates of personal incomes are derived from a sample, HMRC also sets out the uncertainty in the results using confidence intervals

We have identified some areas where we consider that the statistics should be enhanced in order to ensure that the highest standards of the Code of Practice continue to be met:

Public Value

  • While we found proactive and productive engagement around these statistics with users from other government departments and allied bodies such as official forecasters, HMRC should consider how to better take account of the needs of analytical users outside of Government. Examples are users in think tanks and academia and those who use the statistics from the SPI in a data archive called the Public Use Tape (PUT), which is maintained by the UK Data Service. PUT contains more-disaggregated data than the statistics in the tables accompanying the personal incomes statistics bulletin and users of PUT data tend to feedback queries about the data to the UK Data Service. Consequently, HMRC personal incomes statisticians might be unaware of some of the needs of such users. To ensure that the statistics and data are maximising public value, HMRC should engage proactively with a wider group of users
  • Information requests from users can often provide helpful feedback about how users’ experiences of using the personal incomes statistics and data might be enhanced. For example, the personal incomes statisticians improved the data in Table 3.1 of these statistics after a user request. We commend HMRC for its responsiveness to users’ needs. HMRC should publish a summary of user feedback showing how feedback has influenced the development of these statistics to help prompt other users to respond to statisticians’ invitations to feed back their requests
  • Because there is a wide range of official statistics on income and earnings, the Office for National Statistics (ONS) has recently published an updated guide on sources of data on income and earnings. HMRC could helpfully provide a link to this guide from its personal incomes statistics web landing page
  • We recognise that the statistics are not very timely, largely because taxpayers who are required to submit a self-assessment tax return have 10 months after the tax year to do so. To help mitigate this, HMRC should provide alongside the personal incomes statistics a link to the monthly PAYE Real Time Information (RTI) statistics that it publishes jointly with ONS

Quality

  • HMRC’s estimates of personal incomes are subject to sampling error. In the detailed supporting information about personal incomes statistics, HMRC provides some guidance about the precision of its estimates for sub-UK areas. In the guidance HMRC presents examples of the precision of data for parliamentary constituencies and for large and small counties. HMRC told us that many of the data requests it receives are for local authority breakdowns, some of which are subject to quite wide sampling variability. To support users’ understanding of the levels of uncertainty in these small area estimates HMRC should provide some indication of their sampling variability and keep under review the sampling arrangements for these statistics in response to user needs

Trustworthiness

  • HMRC states that personal incomes statistics are published annually, usually in January or February. Over recent years, the first set of tables has been released as late as March or April. The uncertainty over timing of publication does not make for an orderly release of the statistics. HMRC should commit more decisively to a regular publication date for the first release and any revisions and update its metadata to reflect dates of release that it is confident it can commit to
  • HMRC releases Tables 3.16 and 3.17, covering data on various data on income tax liabilities on non-savings and non-dividend income due for Scottish taxpayers and non-Scottish taxpayers as official statistics and the statistics in all the other tables as National Statistics. We are satisfied that the statistics in Tables 3.16 and 3.17 are produced to the same standards as the others in this suite and should therefore be labelled as National Statistics for consistency

Our economy team will continue to engage with you and your team in the coming months to follow up on the actions highlighted in this letter and, if you wish to discuss any aspects of this letter, we are happy to do so. I am copying this letter to the responsible team: Mel Whent, Anthony Rourke and Gordon McGregor.

Yours sincerely

 

Mark Pont

Assessment Programme Lead

Ed Humpherson to Sean Whellams: Coronavirus Job Retention Scheme and Self-Employment Income Support Scheme statistics

Dear Sean

Coronavirus Job Retention Scheme (CJRS) and Self-Employment Income Support Scheme (SEISS) statistics

I am writing to endorse the approach you have taken to produce statistics about the Coronavirus Job Retention Scheme (CJRS) and the Self-Employment Income Support Scheme (SEISS). These statistics estimate the number and value of claims made to the CJRS and to the SEISS administered by HM Revenue and Customs (HMRC). I would like to congratulate everyone involved for their work to produce these important statistics.

My team has conducted a rapid regulatory review of the published information. We have reviewed the extent to which they have been produced in accordance with the Code of Practice’s Trustworthiness, Quality and Value pillars, while taking account of the pressures you and your teams have faced to deliver timely statistics about an important topic. A summary of our findings is set out below and we have already provided more detailed feedback to you.

Value

  • These statistics and data help us understand the scale to which employers, employees and the self-employed have relied upon these schemes. There has been considerable press interest in the UK and in the devolved countries in the estimates particularly highlighting the take-up and value of the schemes at UK, regional and local authority levels
  • The releases include information such as glossaries, background information about the schemes, the strengths and limitations of the statistics and links to other related statistics which help users to use and interpret the statistics. There are good breakdowns for each set of statistics which draw from the datasets available. The presentation of charts engages readers’ interests and where data are ranked it’s easy to see where or what is highest and lowest
  • Your approach to engaging with devolved countries’ statisticians about improvements that they would like to see in the statistics is praiseworthy. You have set out in the CJRS bulletin that there’s scope to include additional analysis in further releases. For example, you have mentioned including breakdowns of the CJRS claims by age and sex of employees. This engagement helps the statistics meet users’ needs as well as they can, and provide a strong foundation for future developments
  • These statistics have been developed from management information that HMRC was tweeting on a daily basis following the launch of the CJRS on 20 April. Releasing the management information so promptly about the CJRS including numbers of unique applicants, the number of furloughed jobs and the value of claims was a good example of transparency. It is particularly good that your team went on from releasing the management information to match data from its Real Time Information (RTI) system with Coronavirus Job Retention Scheme data to produce breakdowns of claims providing even more insight in the official statistics now being produced
  • The estimates of take-up of the SEISS relative to the estimated number of people eligible is helpful in making comparisons, we encourage you to do something similar for the CJRS, using a suitable denominator to enable easier geographic comparisons of the impact of furlough
    • Your decision to publish sub-regional statistics will helpfully provide further insights from the wealth of data in tables accompanying the main statistics. We suspect that as part of managing the recovery from the pandemic at a local level, users would benefit from further sub-UK insights. For example, in relation to SEISS there might be benefit to users in providing some background on the plausible reasons for significantly lower take-up rates of SEISS in some of the London local authorities and significantly higher than average take-up in some south Wales local authorities

Quality

  • The rationale for the assumptions made are well explained and reasonable. For example, that PAYE schemes provide a reasonable proxy for employers for the purposes of the release
  • To allocate scheme applicants into regions and countries you have used the best available address data, for example in CJRS statistics, people’s residential addresses. There are many reasons why someone’s residential location would be in a different region or country to where their job is. CJRS bulletin already provides an explanation about the use of residential addresses in section 4 of the CJRS bulletin but it may be worth repeating the explanation in section 5 for people who go directly to that section. Similarly, in the SEISS statistics the claimant’s self-assessment address is used which may be different to their residential address, for example the self-assessment address could be the claimant’s accountant’s or solicitor’s address. Using a self-assessment address might result in some distortions in the figures. You might urge greater caution on placing too much weight on statistics on eligibility and take-up of SEISS at Local Authority and Parliamentary Constituency levels due to factors such as this
  • There is interest in the numbers of people eligible for SEISS, for example the House of Commons Treasury Select Committee and the Institute for Fiscal Studies. IFS for example suggested that the numbers of people eligible for SEISS are around 3.12m whereas the estimated numbers in SEISS are given as 3.4m. We consider that acknowledging and explaining differences, where possible, would offer further reassurance to your users about the quality of the HMRC estimates of totals numbers eligible for SEISS support

Trustworthiness

  • You have pre-announced through the HMRC forward release calendar the release date for these statistics and has given a specific release date for the next release at least four weeks in advance

We look forward to seeing these statistics develop as circumstances change. As set out in our guidance on changes to statistical outputs you can include a statement to accompany information about these statistics such as “These statistics have been produced quickly in response to developing world events. The Office for Statistics Regulation, on behalf of the UK Statistics Authority, has reviewed them against several key aspects of the Code of Practice for Statistics and regards them as consistent with the Code’s pillars of Trustworthiness, Quality and Value.”

I am copying this letter to Jane Whittaker, Director of HMRC’s Knowledge, Analysis and Intelligence directorate.

Yours sincerely

 

Ed Humpherson

Director General for Regulation

Ed Humpherson to Jim Harra: Strengthening the quality of HMRC’s official statistics

Dear Jim

 Strengthening the quality of HMRC’s official statistics

Today we publish the findings of our review of the quality of HMRC’s official statistics. HMRC invited us to carry out this review after identifying a significant error in its Corporation Tax receipt statistics. The aim of the review was to provide an independent assessment of HMRC’s quality management approach for official statistics and to identify improvements.

HMRC produces many official statistics and most of the time it does this without issues or errors. We found much to admire in how the Knowledge, Analysis and Intelligence directorate (KAI) manages the quality of HMRC’s official statistics, but several areas where it could strengthen its approach. We make nine recommendations to address these issues. The most immediate recommendations relate to maintaining effective relationships between those producing official statistics and other parts of HMRC they receive data from; understanding data journeys; and quality assurance of data, analysis and publications. Further recommendations focus on strengthening leadership of quality management; building more effective knowledge sharing about quality; and improving information published about the quality of official statistics.

Our report notes that, like many other large government departments, most of HMRC’s IT systems have been designed for operational delivery, rather than for statistical analysis, and have been in place for many years, so may not have the functionality or flexibility of newer systems. This means it can be difficult for analysts in KAI to ensure that they have access to the right data, in the correct format, for their analytical work. Long-term improvements to quality management will rely in part on IT system and data owners in HMRC understanding and prioritising the Department’s statistical requirements alongside operational considerations. Investment in underlying IT systems will also be important to enable HMRC to continue to confidently exploit the value of the data it holds.

By inviting this review, HMRC has taken a proactive approach to strengthening the quality of its official statistics, and I’d like to thank HMRC for its open and positive engagement with my team throughout. I look forward to future engagement with HMRC as it takes steps to strengthen the quality management and assurance of its official statistics. I am copying this letter to Ruth Stanier, Director General for Customer Strategy and Tax Design, Jane Whittaker, Director of HMRC’s Knowledge, Analysis and Intelligence directorate, and Sean Whellams, HMRC Head of Profession for Statistics.

Yours sincerely

Ed Humpherson

Director General for Regulation

 

Related links:

Ed Humpherson to Ruth Stanier: Strengthening the quality of HMRC’s official statistics

Ed Humpherson to Sean Whellams: Review of HMRC statistical quality management

Review – Strengthening the quality of HMRC’s official statistics

Ed Humpherson to Ruth Stanier: Strengthening the quality of HMRC’s official statistics

Dear Ruth

Strengthening the quality of HMRC’s official statistics

Today we publish the findings of our review of the principles and processes underpinning the quality of HMRC’s official statistics.

We welcome HMRC inviting OSR to carry out this review after HMRC identified a significant error in its Corporation Tax receipt statistics, which affected the period from April 2011 to July 2019. The aim of the review was to provide an independent assessment of HMRC’s quality management approach for official statistics and to identify improvements that lower the risk of errors, especially errors that have a large impact on the statistics and their interpretation by users.

HMRC produces many official statistics and most of the time it does this without issues or errors. We found much to admire in the way that HMRC manages the quality of its statistics, but several areas where it could strengthen its approach: we make nine recommendations to address these issues. Our findings highlight the issues that big operational departments working with administrative data can face with respect to quality and will ring true for other Government departments.

The most immediate of our recommendations relate to establishing and maintaining effective relationships between those producing official statistics in HMRC and other parts of HMRC they receive data from; understanding data journeys; and quality assurance of data, analysis and publications. Further recommendations focus on strengthening the leadership and oversight of quality management; building more effective routes for knowledge sharing about quality; and improving the information HMRC publishes about the quality of its official statistics. Implementing these recommendations will strengthen the quality culture in HMRC and make sure that the structure and tools are in place to manage statistical quality effectively, minimising risk.

Our report also notes that, like many other large government departments, most of HMRC’s IT systems, from which the administrative data that underpins almost all of HMRC’s official statistics, and other analysis, are drawn, have been designed for operational delivery, rather than for statistical analysis, and have been in place for many years, so may not have the same functionality or flexibility as newer IT systems. Along with the recommendations about statistical quality management, investment in underlying IT systems to ensure they keep pace with these modern expectations (while also being of wider benefit to HMRC in fulfilling its duty of collecting taxes), will be important to enable HMRC to continue to confidently exploit the value of the data it holds. I have written separately to Jim Harra, First Permanent Secretary and Chief Executive for HMRC, about the need for this wider investment.

I’d like to thank HMRC for its open and positive engagement with my team throughout this review, and I look forward to future engagement with HMRC as it takes steps to strengthen the quality management and assurance of its official statistics. I am copying this letter to Jane Whittaker, Director of HMRC’s Knowledge, Analysis and Intelligence directorate, and Sean Whellams, HMRC Head of Profession for Statistics.

Yours sincerely

Ed Humpherson

Director General for Regulation

 

Related links:

Ed Humpherson to Jim Harra: Strengthening the quality of HMRC’s official statistics

Ed Humpherson to Sean Whellams: Review of HMRC statistical quality management

Review – Strengthening the quality of HMRC’s official statistics

Review of HMRC statistical quality management

Dear Sean

Review of HMRC statistical quality management

I am writing to thank you for inviting the Office for Statistics Regulation to carry out a review of the quality assurance principles and processes underpinning HMRC’s published statistics. The aim of our review is to provide an independent assessment of HMRC’s quality management approach and to identify improvements in the way it produces official statistics.

This review was initiated after HMRC identified a significant error in published Corporation Tax receipt statistics, which affected the period from April 2011 to July 2019: estimates for this period were corrected in your Corporation Tax statistics publication on 24 September. We note that this was a statistical reporting error, which does not affect the amount of tax paid by companies and received by HMRC, or the figures reported in HMRC’s accounts; and that the overall trend of increasing onshore Corporation Tax receipts remains the same following this revision.

We know that HMRC very much regrets this error, and we are pleased to help you take a proactive approach to improving quality management processes across your statistics. We intend that our final recommendations will be forward looking and focus on improvements HMRC can make to help avoid issues in the future. The review will consider:

  • existing quality assurance (QA) processes and current improvement plans within the Knowledge, Analysis and Intelligence (KAI) directorate of HMRC;
  • how QA of official statistics works in practice within KAI;
  • the processes in place for handling errors and managing risk during statistical production;
  • the levels of risk of different sources of error in KAI’s statistical production processes.

We welcome your approach to us to undertake this review and look forward to engaging with you and your teams over the coming months. We aim to publish our initial findings and recommendations at the start of next year.

Yours sincerely

Ed Humpherson
Director General for Regulation

Scotland’s Block Grant Adjustment

Dear Tom

As you are aware, the Office for Statistics Regulation was contacted about the use of HMRC statistics on Scottish Income tax receipts in 2017-18 in an article by Liz Truss MP, in the Scottish Daily Mail on 19 July. This article followed the publication of income tax reconciliations for Scotland, which were published by HMRC for the first time on 18 July.

The basis of the article was HM Treasury’s press release “Scottish Income tax shortfall offset by UK funding”1. The press release states that “Scotland’s economy grew more slowly than the rest of the UK, hitting tax receipts and leaving the Scottish Government with a shortfall in funding”.

While it is correct that Scotland’s economy grew more slowly than the rest of the UK it is incorrect to suggest this is the primary reason for reduced Scottish income tax revenues and therefore the main reason for the adjustment to the block grant funded by UK Government. The main causes are the availability of more accurate figures on the income tax base in Scotland and divergences in the forecasted income tax revenues between the Rest of the UK and those for Scotland.

While the joint fiscal framework is a relatively complex, multi-stage process of transfers and balancing, the principal reasons for changes to Scottish income tax revenues and Block Grant Adjustment revisions after reconciliations are amenable to clear and unambiguous explanation. We recommend that future HM Treasury press statements based on these statistics provide better explanations of the causes of changes to the Scottish income tax revenues and the associated Block Grant Adjustment. This will reduce the risk that users draw misleading conclusions from the statistics and the statements that draw on them.

We recognise that this is the first time that HMRC has published these statistics and been subject to associated media scrutiny. While it is likely future reconciliations will not be as large, we consider that to fully inform the public it would be helpful to present this framework in line with our Code of Practice for Statistics, considering how the statistics can be best presented in a clear and unambiguous way.

I am copying this letter to Sean Whellams, Head of Profession for Statistics at HMRC, and Alyson Stafford, Director General at the Scottish Exchequer.

Yours sincerely

Mary Gregory
Deputy Director for Regulation

 

Related Links:

Sir David Norgrove to Derek Mackay MSP (August 2019)