What can I expect if I raise a concern with you?
1. Concern identified by the Authority
Concerns may be identified by the Office for Statistics Regulation or raised with us via Regulation@statistics.gov.uk. We aim to send an acknowledgement within one working day of receiving any concern.
2. Initial review
The Office for Statistics Regulation (OSR) completes an initial review to consider if a case is in scope and how urgent it is.
If a case is in scope OSR will look into the concerns raised. OSR or the office of the UKSA Chair will usually engage with the relevant producer or the office of a political figure to learn more about the issue raised.
OSR, involving the UKSA Chair where appropriate, will form a judgement on the issues raised.
5. Outcome and follow up
We may decide to make our view public on our website via a letter (to the person or organisation who contacted us or who the enquiry related to), a statement or a blog. We will also share the outcome with whoever raised the issue with us.
We do not need a complaint to be raised with us in order to intervene.
Regulators across our domains keep abreast of current issues and identify concerns. We also undertake cross-cutting monitoring. For example, we reviewed manifestos from the major political parties for the 2019 General Election, primarily checking for repeated use of any statistics we had previously raised concerns about. We are also developing automated monitoring to support our understanding of users’ primary concerns about statistics and their use.
Whether self-identified or raised with us by somebody else, we are guided by our commitment to statistics that demonstrate trustworthiness, quality and value and will be proportionate in judging when we intervene.
Our Casework Annual Report shows the proportion of our cases which are self-generated. Of the 109 cases considered in 2019/20, internally generated casework made up 39% of all cases (43 cases). This figure varies, for example 19% of cases were internally generated the previous year.
We consider issues related to the production and misuse of statistics.
However, we take a broad view on what falls within our scope. We will consider issues relating to data and analysis produced by UK governments whether they are formally defined as official statistics or not.
We cover the whole UK including relevant bodies in Scotland, Wales and Northern Ireland. Our expectation is that all data published by government should be published in line with the principles set out in the Code of Practice for Statistics.
We have published guidance summarising our position and expectations on the use of management information by government and other official bodies and endorse the National Statistician’s guidance on handling management information (October 2019). We will consider issues related to use of management information based on the following three principles:
- Equality of access: When management information is used publicly to inform Parliament, the media and the public, it should be published in an accessible form, with appropriate explanations of context and sources.
- Regulatory intervention: In cases where management information is quoted and is material to public debate, we will consider a public intervention to highlight the failure to preserve equality of access.
- Proportionality: We recognise there will be occasions when ministers refer briefly or in passing to the management information they have access to in responding to questions. We will be proportionate in judging when we step in, and we will distinguish between a one-off use that is marginal to the issues of interest, and those figures that are material to public debate.
Forecasts and projections
Forecasts are not routinely treated as official statistics due to the potentially subjective nature of the assumptions and models. However, in some cases producers are electing to adopt the Code of Practice to guide their preparation and orderly release, and are therefore making them official statistics. Whether official statistics or not, we support transparency, accountability and orderly release. Where figures from forecasts or projections are quoted publicly by government we expect clear information on the data sources, methods and assumptions. We also expect clarity on the limitations and uncertainty around the estimates.
We recognise that economic forecasts can draw on a wide range of potential assumptions and methodologies, so that different forecasters can arrive at quite different conclusions. Nevertheless, our basic expectations around transparency of methodology and assumptions still stand.
Impact Assessments and Policy Models
We do not routinely comment on governments’ internal analysis, including models to inform government policy or impact assessments. However, where information from these is referred to publicly we may comment in line with the transparency practices set out in our interventions policy.
We take the view that published annual accounts of Government entities, which are prepared under International Financial Reporting Standards and are audited by the National Audit Office, do not fall within our scope. There are already rigorous arrangements to establish the reliability of these types of statement. However, we may comment on issues related to government funding or spending. We have previously published guidance for statements about public funding.
In order to look into a claim it is helpful to have details of:
- The statistics you are raising a concern about – ideally including a link or clear reference to the publication if it is a concern about the production of official statistics.
- The date and details of the claim in question – ideally including a link to where the claim has been made or reported e.g. if it is a concern about a claim made by a public figure.
- What aspect of the statistics or claim you are concerned about.
- Whether you have raised the concern with another relevant body (e.g. another regulator)
- Where concerns relate to actions by a producer of statistics, we encourage direct dialogue with producer before contacting us. It is helpful if you can tell us whether or not you have spoken to the relevant producer organisation when contacting us.
All issues raised with us externally or identified internally are recorded in our issues log. Inclusion in the issues log does not necessarily mean that we share the concern, but does demonstrate that the matter has been brought to our attention.
On occasions an issue will be raised with us which would be more appropriately considered by a different organisation. In these cases we encourage the correspondent to engage with the relevant organisation. For example:
- Advertising Standards Authority (ASA): Look at the content of television and radio advertisements.
- Information Commissioner’s Office (ICO): The UK’s independent authority set up to uphold information rights in the public interest, promoting openness by public bodies and data privacy for individuals.
- Independent Press Standards Organisation (IPSO): The independent regulator for the newspaper and magazine industry in the UK.
- Ofcom: The regulator for communications services, including responsibility for what people see and hear on TV and radio.
Response times vary depending on the complexity and profile of the issue. During 2019/20 45% of cases were closed within 10 days and 9% of cases took more than 50 days to close.
We aim to send an acknowledgment of all queries within one working day and will send further interim updates for cases which take a long time to resolve.
We aim to respond to any individual or organisation that raises a concern with us whether or not we consider the concern to be within our scope.
In considering whether documents or statements are liable to mislead, we judge whether, on a question of significant public interest, the way statistics are used is likely to leave a reasonable person believing something which the full statistical evidence would not support. We take into account:
- the context in which the statement is made (for example, distinguishing speeches and publications from off-the-cuff remarks in interviews or debate)
- the standing of the person responsible for the misleading statement (for example, whether that person, by virtue of holding a public office would be expected to make authoritative statistical statements)
- the extent to which the statement might be misleading – for example, we will consider if we think the statement is materially wrong to the point it could change the conclusion of a reasonable person.
- the extent to which the statement might undermine public confidence in the integrity of official statistics, and
- whether this is the first time the issue has arisen, or whether it is a repeated issue.
We have published a short think piece on misleadingness which gives further details on how we approach our thinking.
In deciding whether and how to intervene we will consider the extent to which something is misleading or potentially misleading. We will be proportionate in judging when we step in, we will distinguish between a minor misquote which retains the same sense of narrative and a statement which has potential to change the conclusion of a reasonable person.
When looking into a concern it is important that we understand the issue before forming a judgement. Speaking to the analysts involved in producing statistics and briefing supports us in understanding the issues.
We may share letters or statements with relevant individuals and organisations before publication to ensure factual accuracy or to make the recipient aware of our intention to publish.
Once we have looked into a concern raised with us we may conclude that there is an issue which needs to be addressed.
In deciding how to the address the issue we will consider what intervention is most likely to be effective. For example, if the issue is a one off we may informally raise the concern with the relevant individual or organisation and then monitor for any repeat misuse or poor presentation. If the issue is high profile and widespread then we are likely to intervene more formally including publishing our view through a letter, statement or blog.
In determining what to publish in relation to each case we consider the following points:
- We will make our judgement public where we consider there is a likelihood of the public being misled on an issue of significant public interest. In these circumstances we consider the need for the issue to be clarified. This is more likely where the use of statistics involves a public figure (such as Ministers or Members of Parliaments), or where misuse has been repeated (for example in media reporting).
- Where an issue is resolved informally, we may decide not to write publicly, or to write and welcome the actions.
- If the issue is outside our formal scope but we have chosen to look at it informally, we are less likely to share our response publicly. However, we would still publish our judgement if we deem the issue is of significant public interest and may be viewed in the same way as official statistics by a member of the public, even if they are not official statistics.
- If an issue has been raised with us in a formal manner by a public figure (such as a Member of Parliament) or organisation we would usually publish the incoming correspondence and any of our own subsequent correspondence. We will only publish correspondence we receive from others where we have permission from the correspondent.
- Where we think a number of issues raised with us contribute to a broader issue we may decide to publish a statement on the broader issue rather than publish responses relating to each individual concern raised with us. In these circumstances we will respond privately to all those who have raised relevant issues with us.
All issues we have considered through casework are recorded in our issues log – whether internally generated or externally raised and whether our response is published or not. We also produce an annual summary of our casework.
All concerns raised with us are logged anonymously on our issues log.
We will only publish correspondence we receive from others where we have permission from the correspondent.
We would typically only publish letters we receive if an issue has been raised with us in a formal manner by a public figure (such as a Member of Parliament) or organisation. In these cases we would usually publish the incoming correspondence and any of our own subsequent correspondence.
Where our intervention involves recommending an action by a producer of statistics, OSR will follow up to monitor implementation. If we feel the issue has not been adequately addressed then we will follow up with the producer and may raise our further concerns publicly.
We also seek feedback from those who have raised concerns with us to support improvements to our work.