Compliance Check on HMRC Measuring Tax Gaps statistics – Response

Dear Anthony


Thanks for your letter of 30 October setting out the progress you have made in enhancing the trustworthiness, quality and value of HMRC’s Measuring Tax Gaps statistics since the publication of our compliance check on 16 May 2019. We welcome the work that has been accomplished and the efforts of you and your team to develop these important statistics.

We appreciate that the improvements made so far are just part of the ongoing innovation in presenting tax gaps statistics that HMRC has become recognised nationally and internationally for initiating. We look forward to seeing further public value arising from HMRC Measuring Tax Gaps statistics in the future.

This letter will be published alongside my previous letter setting out the findings of the compliance check once the election period is concluded.

I am copying this letter to Sean Whellams, Head of Statistics profession in HMRC.

Yours sincerely,

Mark Pont
Assessment Programme Lead


Related Links:

Letter – Anthony Burke to Mark Pont (October 2019)

Compliance Check – Measuring Tax Gaps statistics (May 2019)

Review of HMRC statistical quality management

Dear Sean

Review of HMRC statistical quality management

I am writing to thank you for inviting the Office for Statistics Regulation to carry out a review of the quality assurance principles and processes underpinning HMRC’s published statistics. The aim of our review is to provide an independent assessment of HMRC’s quality management approach and to identify improvements in the way it produces official statistics.

This review was initiated after HMRC identified a significant error in published Corporation Tax receipt statistics, which affected the period from April 2011 to July 2019: estimates for this period were corrected in your Corporation Tax statistics publication on 24 September. We note that this was a statistical reporting error, which does not affect the amount of tax paid by companies and received by HMRC, or the figures reported in HMRC’s accounts; and that the overall trend of increasing onshore Corporation Tax receipts remains the same following this revision.

We know that HMRC very much regrets this error, and we are pleased to help you take a proactive approach to improving quality management processes across your statistics. We intend that our final recommendations will be forward looking and focus on improvements HMRC can make to help avoid issues in the future. The review will consider:

  • existing quality assurance (QA) processes and current improvement plans within the Knowledge, Analysis and Intelligence (KAI) directorate of HMRC;
  • how QA of official statistics works in practice within KAI;
  • the processes in place for handling errors and managing risk during statistical production;
  • the levels of risk of different sources of error in KAI’s statistical production processes.

We welcome your approach to us to undertake this review and look forward to engaging with you and your teams over the coming months. We aim to publish our initial findings and recommendations at the start of next year.

Yours sincerely

Ed Humpherson
Director General for Regulation

Compliance Check of Measuring Tax Gaps statistics

Dear Sean


We recently completed our check of the compliance with the Code of Practice for Statistics of HMRC’s Measuring Tax Gaps (MTG) statistics. While these statistics are not National Statistics because some of the components are not sufficiently robust for National Statistics publication, they are important official statistics valued by users and there would merit in conducting a check of compliance against the elements of Code of Practice for Statistics particularly:

  • in respect to trustworthiness, we looked at orderly release and transparent processes and management;
  • in quality, we reviewed aspects of using sound methods to produce the statistics and how you assured quality; and
  • in the public value, we considered the relevance of the statistics to users, the clarity of the insights which the statistics offer to users and how HMRC is innovating and improving them

Your team has proven to be highly committed and engaged when working to enhance the trustworthiness, quality and value of these statistics. We found that HMRC’s leadership of the measurement of tax gaps was cited by eminent bodies such as the International Monetary Fund (IMF) and the National Audit Office. These are important statistics making a vital contribution to informing the national discussion around tax fairness and tax compliance.

We commend the preparation, production and publication of the MTG statistics, for example:

  • The main headline statistic – the overall scale of the gap – is front and centre of MTG and is quoted to the nearest billion pounds to help users understand the approximate accuracy of the estimates
  • The MTG statistics are valuable tools in helping HMRC make the best use of its resources. The MTG statistics help HMRC make detailed analysis of risks such as those to maximising tax yield. These analyses then contribute to the development of HMRC policy, strategy and operational activity in relation to the different taxpayer customer groups
  • The annual MTG statistics publication includes an estimate of the hidden economy and in MTG 2017 you introduced a revised hidden economy methodology using new data from a hidden economy survey commissioned by HMRC. HMRC’s model for estimating the scale of the hidden economy helps gain access to a hard-to-reach population and achieve honest responses to a sensitive topic
  • To enhance the quality of these statistics, HMRC proactively engages with external stakeholders by presenting findings at relevant external forums and working alongside experts on methodology improvements. For example, HMRC is collaborating with the Danish authorities on risk-based audits exploring variables for the entire population and management of HMRC’s random enquiry programme. We commend you on your international outreach
  • HMRC has chosen appropriate data sources to calculate the MTG statistics. HMRC has assessed and minimised the impact of any data limitations for use, and has explained these well in a detailed methodological annex.

We identified some areas where we consider that you could reinforce the trustworthiness of these statistics through enhanced transparency, demonstrate more openly the quality of the estimates and seek wider engagement with a view to further exploiting the considerable public value that the statistics offer. The detailed areas are:

  1. Showcasing the value of collaboration on the methods for measuring tax gaps with international and academic topic and methods experts can contribute to greater trustworthiness of these statistics when stakeholders, commentators and users see that HMRC is world-leading in measuring tax gaps and is setting the bar for others to follow
  2. HMRC could take steps to test whether there is an appetite from users for more nuanced reporting of tax gaps in line with the IMF proposals; and if there is, re-examine its technical capability to produce such estimates. This could provide opportunities to widen the uses of the statistics to new and existing users and potentially offer new insights into the effectiveness of the UK tax system
  3. HMRC should report to its methods governance body on the outcomes of the review of its continued use of US Internal Revenue Service research for multipliers. Particularly whether these continue to represent international good practice, scientific principles, or established professional consensus. This might better reflect in the statistics, the current behaviours of taxpayers in the UK and potentially supplying more-robust estimates of tax gaps
  4. HMRC should review whether its assumptions about under-reporting of alcohol consumption continue to represent established professional consensus. This could offer more assurance that the alcohol tax gaps statistics represent the current behaviours of taxpayers in the UK and lead to more-robust estimates of tax gaps
  5. HMRC to consider linking in the MTG publication to research based on analysis of the tax gap data. This recommendation could help HMRC not only draw attention to the valuable insight that the Institute for Fiscal Studies brought out of HMRC’s data but also publicise the availability of data to approved researchers in Datalab. The greater use that can be derived from HMRC’s microdata can enhance the public value of the data
  6. HMRC to consider whether the public value of these statistics and data could be enhanced within the budgets available, through examining the potential to meet users’ priorities beyond HMRC staff. Widening engagement might help HMRC to better meet the needs of a wider sets of users when considering where to place its limited development time and funds for these statistics. Additionally, externally verified evidence improves the prospects for people seeing the findings as high-quality, robust and independent.
  7. Where appropriate, provide comparisons to support interpretation of the tax gaps statistics and data, signpost to relevant statistics, and explain consistency and coherence. This might better help users with interests in fiscal impacts by helping them judge the prospects for future tax raising.

We have included more detail about our judgement and more detail about the recommendations in an annex to this letter. Our Economy Domain Lead, Iain Russell, has agreed with your team that they will continue to engage with each other on progress over the coming months. I would welcome a formal update from you by end of October 2019 about how you have addressed these areas.

Thank you for engaging effectively with us during this review. Please let me know if there is any aspect of this letter that you wish to discuss. I am copying this letter to Anthony Burke as responsible statistician for the Measuring Tax Gaps statistics.

Yours sincerely

Mark Pont
Assessment Programme Lead