Chapter navigation
The compliance review process
Overview
The diagram below sets out the key stages of a compliance review and the activities that occur during each stage. Activities under ‘All reviews’ are carried out for every review, regardless of their scope. Activities under ‘Assessment’ are additional activities that are carried out for all assessments. These activities are optional for other compliance reviews.
Accessible text for Compliance Review Process diagram
- Planning – All reviews involve a kick-off meeting with producers.
- Evidence gathering – All reviews involve desk research and an evidence meeting with producers. For assessments we also carry out data supplier and user engagement.
- Evidence review & judgements – All reviews involve compliance recording, sharing emerging findings with producers and drafting reports and letters.
- Quality assurance – All reviews involve OSR internal quality assurance (QA) and producer review of reports.
- Reporting findings – All reviews involve OSR internal sign-off and publishing reports and letters. Assessment reports are signed off by Regulation Committee.
- Follow up – All reviews involve producer engagement. Assessments also involve a producer action plan, reviewing evidence of meeting requirements, Regulation Committee sign-off of the accreditation decision, publishing the accreditation letter, and updating the list of accredited official statistics.
Planning
Our lead regulator for the relevant topic area will generally discuss potential compliance reviews with your HoP, Chief Statistician, or lead official as part of our business planning process. We will discuss any issues that have triggered the review and its specific focus and get the producer’s views. While most compliance reviews are pre-planned, our work programme is flexible enough to add reviews throughout the year.
The regulatory team in OSR will contact you to discuss the timing and scope of the review. If your organisation is new to compliance reviews, we will offer an initial meeting to explain the process and give you an understanding of the goals and potential outcomes of a review.
If you are seeking accreditation for official statistics, your HoP, Chief Statistician, or lead official must write to us to formally request an assessment. This process is set out in legislation (section 12(1) of the Statistics and Registration Service Act 2007). We will publish the letter alongside a short reply from us confirming that we will carry out the assessment. Only write the letter once we have confirmed that will be able to prioritise and resource the assessment.
Here are some examples of exchanges of letters:
- Letter from the Department of Health and Social Care requesting an assessment of the adult social care provider statistics and our reply
- Letter from Medr requesting an assessment of six statistics on higher education, further education, apprenticeships and adult community learning and our reply
- Letter from the Department of Infrastructure requesting an assessment of the Northern Ireland public transport statistics and road network and condition statistics and our reply
Compliance reviews of existing accredited official statistics or official statistics do not need a request from the producer; we can choose to initiate these at any time.
Evidence gathering
We seek to gather a broad range of evidence from different sources:
Desk-based research and document review:
Reviewing publicly available materials such as statistical outputs, website landing pages and statistical policies. For assessments, we may review internal documents about quality assurance or governance arrangements.
Engagement with producers:
Meetings or email exchanges to discuss aspects of statistics production and statistical developments.
Engagement with data suppliers:
Meetings or email exchanges to discuss aspects of the data collection.
Engagement with users:
Meetings or email exchanges to gather views on the trustworthiness, quality and value of the statistics.
We will first carry out desk research to evaluate the strengths and weaknesses of the statistics against the Code. We draw on material already in the public domain, along with anything we already know from users and stakeholders. For most compliance reviews, we do not require statistics producers to provide any additional documentary evidence.
For assessments, our main tool for gathering documentary evidence is the list of documents (Annex B). We ask you to complete this list at the start of the assessment. The documents include statistical policies and statements, methods and quality information, governance arrangements and user engagement plans.
You do not have to provide every document in the list. We are looking for existing documents to help us fill gaps in our knowledge or to demonstrate compliance with specific standards or practices of the Code. Feel free to refer to any individual document as often as it applies but highlight the specific part of any document that relates to each entry. It is fine if some documents are department-wide rather than specific to the statistics or topic.
For assessments, we need to understand all aspects of the data and statistics, so it is essential that we speak to users of the statistics and data suppliers. This engagement helps us form judgements about the quality of the data, where the statistics are adding value, and how trustworthiness, quality and value can be improved. We ask you to send us a list of known contacts, both users of the statistics and data suppliers, at the start of the review. Often, we supplement this list with our own understanding of users and potential users. The number of users we speak to varies from assessment to assessment, but we usually aim to speak to at least 10 users from different backgrounds.
Compliance reviews that are not assessments do not normally involve user and data supplier engagement, but we may speak to a small number of users and/or data suppliers if needed to help us form our judgements.
A critical element to the success of any compliance review is the relationship we build with you. While we are the experts on the Code of Practice, you are the expert on your statistics. It is our role to work together to get the most out of the compliance review.
We see our meetings and conversations with you as the most important opportunities to fully understand your statistical practices, how you manage known data issues, and plans for development. We will always be clear about the input we expect from you and will regularly update you on our plans and progress.
For assessments, there will usually be multiple evidence meetings. The first evidence meeting will be near the start of the assessment, after an initial review of the documentary evidence you shared with us. This is our main opportunity to explore with you how you produce the statistics, their context, the pressures you face and plans for development.
Evidence review and quality assurance
Our main project management tool for compliance reviews is the compliance record. We use this record to capture all key information, evidence and judgements about the statistics.
For assessments, after an internal quality assurance review, we will hold a second evidence meeting, where we will go through areas of your practice in greater depth. We will also discuss the next steps for the review, including timings for sharing a draft report and our publication plan. Timings will depend on the scope and scale of the review, but the table below gives a rough indication of timings for an assessment where the producer had six months to meet the requirements:
| Stage | Month |
|---|---|
| 1. Planning | January |
| 2. Evidence gathering | February - April |
| 3. Evidence review & judgements | April - May |
| 4. Quality assurance | May - June |
| 5. Reporting findings | June - July |
| 6. Follow-up | January - February |
Once we have some emerging findings and have identified any gaps in our understanding, we discuss these with you to get a complete picture of the trustworthiness, quality and value of the statistics, and to give you an indication of our conclusions and likely recommendations for improvement.
Most assessments will have requirements, which are the improvements that you must make to gain or retain accreditation. These are non-negotiable changes that are essential for meeting the standards of the Code.
Other compliance reviews will generally have recommendations. Recommendations are suggested changes that you should consider making. These are not mandatory but are strongly encouraged to improve the statistics in line with the Code.
Although we tend to make requirements in assessments and recommendations in compliance reviews, it can sometimes be the other way around, and a compliance review may contain both requirements and recommendations.
For assessments, we will discuss likely requirements during evidence meetings so that you can begin to address any compliance issues. We will also discuss the feasibility of meeting the requirements and agree with you a sensible timescale over which the requirements should be implemented. If you make improvements to the statistics during the period when we are drafting our report, we can reflect these actions in our report.
Reporting findings
Once the draft report has been internally quality assured, we will share it with you for comment. We will generally give you at least five working days to review the report, and we will give you an early indication of when you can expect to receive it. We are primarily looking for comments on the factual accuracy of the report, but this is an important time for you to consider if the requirements or recommendations we have set out are achievable within the proposed timescale.
We will inform you of any substantive changes to the report contents during the sign-off stages. You can then let us know if you identify any issues with the changes made.
Assessment reports are cleared by the Head of Regulatory Compliance and the Director General for Regulation before being submitted to the Regulation Committee for a final decision on the accreditation of the statistics. The Regulation Committee is the group of non-executive directors of the UK Statistics Authority that oversees our regulation activities on behalf of the Authority.
All assessment and compliance review reports are accompanied by a cover letter to your HoP, Chief Statistician or lead official. This letter summarises our key findings and requirements or recommendations and sets out the next steps for the producer team. The report and letter are published on the publications and correspondence areas of our website, respectively. Your HoP, Chief Statistician or lead official should support you in implementing the requirements or recommendations.
Follow-up
Assessment follow-up occurs during the ‘meeting requirements’ phase. The evidence we gather during this phase informs our recommendation for accreditation. Because the outcome is a decision on awarding accreditation, our recommendation needs to be approved by the Regulation Committee.
Assessment reports include a deadline for completing requirements. The deadline will depend on the number and nature of the requirements. It might be as little as three months for assessments with a few small requirements, or a year or more for assessments with several substantial requirements. It is important that you bear this in mind during the assessment and let us know whether you expect any difficulties in meeting the deadline.
For assessments and some other large compliance reviews, consider publishing an action plan of the steps you plan to take to meet the requirements or recommendations. This would demonstrate transparency about your planned improvements.
Here are some examples of action plans that were published in response to assessment requirements:
- Office for National Statistics (ONS) development plan for private rental prices statistics
- ONS action plan for admin-based population estimates
- HM Revenue and Customs and ONS action plan for statistics on earnings and employment from Pay as You Earn Real Time Information
We are happy to meet with you after the assessment report has been published to discuss your plans for addressing requirements. We will also check in with you before the deadline to see how you are getting on. Feel free to contact us throughout this period if you would find that helpful.
We expect you to send us evidence of how you have addressed the requirements and provide any links or attached evidence to demonstrate compliance. We don’t have a specific template for this response, but you may find the annex structure in previous accreditation letters helpful (see our correspondence page for examples). This evidence is required in advance of the deadline for completing requirements, but we are happy to receive evidence on an ongoing basis as you make improvements.
If we consider that a requirement has not yet been met, we may ask you to provide additional evidence. Once we are satisfied that all requirements have been met, we send our evidence to the Regulation Committee, which decides whether to accept our recommendation to accredit the statistics. If our recommendation is accepted, a letter confirming the accreditation, with an annex summarising the evidence that supports our judgement, will be published on our website and sent to your HoP, Chief Statistician or lead official before publication. Your HoP, Chief Statistician or lead official does not need to respond to this letter.
If after receiving additional evidence we are still not satisfied that all requirements have been met, we may extend the deadline for meeting requirements or choose to pause or close the assessment.
Compliance review reports include a similar deadline for reporting publicly on addressing or making progress against our recommendations, and, where relevant, requirements. This update can be in the form a letter from your HoP, Chief Statistician or lead official.
Accredited official statistics
Accredited official statistics are called National Statistics in the Statistics and Registration Service Act 2007.
You are encouraged to include the following information on your published statistics when compliance with the Code has been confirmed by OSR:
“These accredited official statistics were independently reviewed by the Office for Statistics Regulation in [month/year]. They comply with the standards of Trustworthiness, Quality and Value in the Code of Practice for Statistics and should be labelled ‘accredited official statistics”
You must display the accredited official statistics badge alongside accredited statistics. The badge should be shown at the top of the first page of a report. On a web page, the badge is generally displayed in the top right of the page. The badge does not need to be shown in other supporting documentation, but these documents should make clear their relevance to the accredited official statistics.
Accredited official statistics can be presented alongside official statistics or with other data. In these situations, it is essential that you explain the provenance of the different types of data. Clear information about the relative strengths and limitations of the data is vital.
We maintain the List of Accredited Official Statistics (previously known as the List of National Statistics) on our website. It is a comprehensive list of all statistical outputs that OSR has independently reviewed and accredited. Under the Statistics and Registration Service Act 2007, you must maintain compliance with the Code when producing and publishing accredited official statistics.
Suspension and cancellation of accreditation
From time to time, some statistics might not fully meet some standards of the Code, for example because of data quality issues or a change in data source. If you have concerns about the ongoing compliance of accredited official statistics, you can ask us to suspend or cancel their accreditation. We normally only consider suspension or cancellation where standards fall significantly below the expectations of the Code. Suspension and cancellation result in the removal of the accredited official statistics badge and other related language from the publication, and removal from of the publication from the List of Accredited Official Statistics.
We welcome producers raising Code compliance issues with us proactively. Requesting a suspension or cancellation of accreditation demonstrates transparency and lets users know that you are actively monitoring the Trustworthiness, Quality and Value of your statistics.
Note that a producer cannot remove accreditation from its own statistics; OSR has the final say as to whether accreditation should be suspended or cancelled, and accredited official statistics are not considered to have had their accreditation removed unless OSR has written to the producer stating so.
Whether you request suspension or cancellation will depend on the severity of the compliance issues and how long it will take you to make the necessary improvements. Cancellation should be considered if remedial action to address the issues cannot be completed within 12 months of deaccreditation. If you can make the necessary improvements within 12 months, suspension is appropriate.
Following a suspension or cancellation, a publication can regain its accreditation through an assessment by OSR against the standards of the Code. The process of re-accreditation is different depending on whether the accreditation was suspended or cancelled.
See our guidance on suspension and cancellation of accreditation for further information about the process for requesting removal of accreditation, the difference between suspension and cancellation, and re-accreditation.
We maintain a list of accreditations that have been suspended, cancelled and reinstated.
Official statistics in development
Occasionally we review statistics that are labelled official statistics in development. We have published separate guidance on producing official statistics in development. Note that you do not need involve OSR to remove an ‘in development’ label. Your HoP, Chief Statistician or lead official can make that decision independently.
Resources
We have produced a range of guidance on applying the standards and practices of Code of Practice for Statistics. Our regulatory approach policy sets out how apply the Code. Our Guide to the UK Statistical System gives an overview of the governance of the statistical system, including the various committees of the UKSA’s Authority Board.