The compliance review process

Overview

The diagram below sets out the key stages of a compliance review and the activities that may occur during each stage.

Activities under ‘All reviews’ are carried out for every review type, regardless of their scope. Activities under ‘Assessment’ are additional activities that are carried out for all assessments. These activities are optional for other compliance reviews.

Planning

Our lead regulator for the relevant topic area will generally discuss potential compliance reviews with your HoP, Chief Statistician, or lead official as part of our business planning process. We will discuss any issues that have triggered the review and its specific focus and get the producer’s views. While most compliance reviews are pre-planned, our work programme (Excel) is flexible enough to add reviews throughout the year.

A regulatory team in OSR will contact you to discuss the timing and scope of the review. If your organisation is new to compliance reviews, we will offer an initial meeting to explain the process and give you an understanding of the goals and potential outcomes of a review.

If you are seeking accreditation for official statistics, your HoP, Chief Statistician, or lead official must write to us to formally request an assessment. This process is set out in legislation (section 12(1) of the Statistics and Registration Service Act 2007). We will publish the letter alongside a short reply from us confirming that we will carry out the assessment. Here are some examples of exchanges of letters:

• Letter from the Department for Infrastructure (NI) requesting assessment of the Northern Ireland public transport statistics and road network and condition statistics and our reply.
• Letter from NHS England to OSR requesting assessment of the cancer waiting times statistics and our reply.

Evidence gathering

We seek to gather a broad range of evidence. The three main evidence sources for compliance reviews are:

• Desk-based research and document review: reviewing publicly available materials such as statistical outputs, website landing pages and statistical policies. For assessments, we may review internal documents about quality assurance or governance arrangements.
• Engagement with producers and data suppliers: meetings or email exchanges to discuss aspects of data collection and statistics production.
• Engagement with users: meetings or email exchanges to gather views on the trustworthiness, quality and value of the statistics.

We will first carry out desk research to evaluate the strengths and weaknesses of the statistics against the Code. We draw on material already in the public domain, along with anything we already know from users and stakeholders. In most cases, we do not require statistics producers to provide any additional evidence.

For assessments, our main tool for gathering documentary evidence is the list of documents (Annex A). We ask you to complete this list at the start of the review. It includes statistical policy documents, methods and quality information, governance arrangements and user engagement plans.

You do not have to provide every document in the list. We are looking for existing documents to help us fill gaps in our knowledge or to demonstrate compliance with specific standards or practices of the Code. Feel free to refer to any individual document as often as it applies but highlight the specific part of any document that relates to each entry.

For assessments, we need to understand all aspects of the data and statistics, so it is essential that we speak to users of the statistics and data suppliers. This engagement helps us form judgements about the quality of the data, where the statistics are adding value, and how trustworthiness, quality and value can be improved. We ask you to send us a list of known contacts at the start of the review. We supplement this list with our own understanding of users and potential users.

Compliance reviews that are not assessments do not normally involve user and data supplier engagement, but we may speak to a small number of users and/or data suppliers if needed to help us form our judgements.

A critical element to the success of any compliance review is the relationship we build with you. While we are the experts on the Code of Practice, you are the expert on your statistics. It is our role to work together to get the most out of the compliance review.

We see our meetings and conversations with you as the most important opportunities to fully understand your statistical practices. We will always be clear about the input we expect from you and will regularly update you on our plans and progress.

For assessments, there will usually be multiple evidence meetings. The first evidence meeting will be near the start of the assessment, after an initial review of the documentary evidence you shared with us. This is our main opportunity to explore with you how you produce the statistics, their context, the pressures you face and plans for development.

Evidence review and quality assurance

We will review all the evidence and compile our internal compliance record, which forms the basis of our published reports. For assessments, after an internal quality assurance review, we will hold a second evidence meeting, where we will go through areas of your practice in greater depth. We will also discuss the next steps for the review, including timings for sharing a draft of the report and our publication plan.

Once we have some emerging findings and have identified any gaps in our understanding, we discuss these with you to get a complete picture of the trustworthiness, quality and value of the statistics, and to give you an indication of our conclusions and likely recommendations for improvement.

Most assessments will have requirements, which are the improvements that you must make to gain or retain accreditation. These are non-negotiable changes that are essential for meeting the standards of the Code. Other compliance reviews will generally have recommendations. Recommendations are suggested changes that you should consider making. These are not mandatory but are strongly encouraged to improve the statistics.

For assessments, we will discuss likely requirements during evidence meetings so that you can begin to address any compliance issues. We can reflect any action taken in our assessment report should you implement changes during the period when we are drafting our report.

Reporting findings

Once the draft report has been internally quality assured, we will share it with you for comment. We will generally give you at least five working days to review the report, and we will give you an early indication of when you can expect to receive it. We are primarily looking for comments on the factual accuracy of the report, but this is an important time for you to consider if the requirements or recommendations we have set out are achievable within the proposed timescale.

We will inform you of any substantive changes to the report contents during the sign-off stages. You can then let us know if you identify any issues with the changes made.

Assessment reports are cleared by the Assessment Programme Lead and the Director General (DG) for Regulation before being submitted to the Regulation Committee – the group of non-executive directors of the UK Statistics Authority that oversees the regulation activities on behalf of the Authority – for a final decision on accreditation of the statistics.

Reports from all compliance reviews are accompanied by a cover letter to your HoP, Chief Statistician or senior statistician. The report and letter are published on the publications and correspondence areas of our website, respectively.

Follow-up

Follow-up for assessments (the ‘meeting requirements’ phase) is more formal and structured than follow-up for other compliance reviews. Because the outcome is a decision on awarding accreditation, the decision needs to be approved by the Regulation Committee.

The assessment report will include a deadline for completing requirements. The deadline will depend on the number and nature of the requirements. It might be as little as three months for assessments with a few small requirements, or a year or more for assessments with several substantial requirements. It is important that you bear this in mind during the assessment and let us know whether you expect any difficulties in meeting the deadline.

For assessments and some other large compliance reviews, consider publishing an action plan of the steps you plan to take to meet the requirements or recommendations. This would demonstrate transparency about your planned improvements.

We are happy to meet with you after the assessment report has been published to discuss your plans for addressing requirements. We will also check in with you before the deadline to see how you are getting on. Feel free to contact us throughout this period if you would find that helpful.

We expect you to send us evidence of how you have addressed the requirements, providing any links or attached evidence to demonstrate compliance. We don’t have a specific template for this response, but you may find the annex structure in previous accreditation letters helpful (see our correspondence page for examples).

If we consider that a requirement has not yet been met, we may ask you to provide additional evidence. Once we are satisfied that all requirements have been met, we send our evidence to the Regulation Committee, which decides whether to accept our recommendation to accredit the statistics. If our recommendation is accepted, a letter confirming the accreditation, with an annex summarising the evidence that supports our judgement, will be published on our website and sent to your HoP, Chief Statistician or lead official on the morning of publication.

Follow-up for smaller compliance reviews is usually more informal and ad hoc. The two most common outcomes are that we:

• follow up on developments and improvements as part of our normal domain engagement, or
• we ask you to send us a formal response to our recommendations. The letter to your HoP, Chief Statistician or lead official will provide a deadline for responding to recommendations.

Accredited official statistics

Accredited official statistics are called National Statistics in the Statistics and Registration Service Act 2007.

You are encouraged to include the following information on your published statistics when compliance with the Code has been confirmed by OSR:

“These accredited official statistics were independently reviewed by the Office for Statistics Regulation in [month/year]. They comply with the standards of trustworthiness, quality and value in the Code of Practice for Statistics and should be labelled ‘accredited official statistics’”.

You must display the accredited official statistics badge alongside accredited statistics. The badge should be shown at the top of the first page of a report. On a web page, the badge is generally displayed in the top right of the page. The badge does not need to be shown in other supporting documentation, but these documents should make clear their relevance to the accredited official statistics.

Accredited official statistics can be presented alongside official statistics or with other data. In these situations, it is essential that you explain the provenance of the different types of data. Clear information about the relative strengths and limitations of the data is vital.

We maintain the  List of Accredited Official Statistics (previously known as the List of National Statistics) on our website. It is a comprehensive list of all statistical outputs that OSR has independently reviewed and accredited. Under the Statistics and Registration Service Act 2007, you must maintain compliance with the Code when producing and publishing accredited official statistics.

We also maintain a list of accreditations that have been suspended, cancelled and reinstated.

See our Accredited Official Statistics Policy for further information.

Resources

We have produced a range of guidance on applying the standards and practices of the Code of Practice for Statistics. The OSR website also contains guidance on producing official statistics and working in line with the Code.