Key themes
Caseload remained high
We have continued to see a high number of concerns raised with us. In 2021/22 we investigated 241 cases. This is a reduction in casework when compared with 2020/21 but it is still more than double pre-pandemic levels (see Figure 1).
Figure 1: Increase in annual caseload
Total annual number of cases investigated between 2008/09 to 2021/22
Figure 1 shows the number of cases investigated from April 2008 to March 2022 with a spike in the number of cases during 2020/21. It also highlights what our caseload may have looked like without cases that were linked to the pandemic.
Download Figure 1: Increase in annual caseload data source
The pandemic has continued to play a part in case load. However, as shown by the orange line in Figure 1, if pandemic related cases are excluded, 2021/22 has seen the highest number of cases for OSR since we started recording case load.
The month with the most cases was December 2021 (63 cases) (Figure 2). It is important to note that in December 43 identical pieces of casework on Intensive Care Unit (ICU) hospitalisations were submitted within a few days of each other. If these cases are excluded, then the number of pieces of casework was relatively consistent across the year. We provide more detail on the impact of these identical cases on our management information in the section on the source of casework.
Figure 2: Monthly casework load
Total cases logged per month between April 2021 – March 2022
There was an issue displaying the chart. Please edit the chart in the admin area for more details.Figure 2 shows the monthly number of cases investigated between April 2021 and March 2022 with a spike in the number of cases received in December. This spike is explained further on in the report.
Download Figure 2: Monthly casework load
Table 1 provides a high-level overview of our Management Information (MI) for casework during 2021/22
Table 1: MI headlines
| Number of cases | 241 |
|---|---|
| Length of time taken to close cases | Median: 11 days / Mean: 20 days |
| Most common source | Member of the public (49%) |
| Most common theme | Health and Social Care (54%) |
| Number of COVID related cases | 126 (52%) |
| Most common category | Use/misuse (41%) |
Looking ahead to 2022/23, we expect to continue to respond to higher numbers of cases compared with pre-pandemic years. We have already received more casework in quarter 1 of 2022/23 than we did in quarter 1 of 2021/22.
The majority of casework was about the pandemic
A large proportion of our casework continued to be directly related to the pandemic. Although the impact of the pandemic on casework has reduced when compared with 2020/21, pandemic related cases remained a large proportion of casework.
Figure 3 shows the general themes of casework from 2021/22.
In 2021/22, over half of cases (54%) were about health and social care statistics. Of those 131 cases, the majority (87%) were related to the COVID-19 pandemic. This is a drop from 97% in 2020/21 but demonstrates the continuing public interest in high quality statistics in this area.
Figure 3: Health and Social care received the most casework in 2021/22
Casework categorised by domain theme
There was an issue displaying the chart. Please edit the chart in the admin area for more details.Figure 3 shows the number of cases that were logged within each domain theme in 2021/22 and allows a comparison of these figures against the previous year.
Download Figure 3: Health and Social care received the most casework in 2021/22
Note: These themes broadly correlate to the OSR domains. Our domains have changed over the years whilst our database has not. Work for 2022/23 will include categorising casework under the domains as they stand in the link above.
The majority of cases about health and social care statistics were received from members of the public (65% – up from 55% during 2020/21). However, 43 of these cases were identical concerns raised with OSR as highlighted in the previous section.
The pandemic was a challenge for everyone, including statistics producers. As we discussed last year in our review of lessons learned during the pandemic, many producers were working at speed and in the face of considerable uncertainty and pressure in order to provide data and statistics about the pandemic. Overall, we found that health and social care statistics producers across the UK demonstrated a clear commitment to transparency through their efforts to put a huge amount of valuable information in the public domain. While we have continued to intervene this year through our casework to improve data and ensure that statistics are released in line with the Code of Practice, we have found that producers are quicker to recognise and respond to issues. Below are some of our public interventions which resulted from the ongoing data needs of the pandemic, highlighting some of the issues we looked into.
Case study – Statistics on COVID-19 vaccinations
In May 2021, Ed Humpherson wrote to the Department of Health and Social Care, NHS England, Public Health England and the ONS highlighting the importance of intelligent transparency in relation to statistics on vaccinations. Whilst the wealth of COVID-19 information made available for the public was exceptional, further work was needed to support the public’s understanding of the data with clearer, more accessible presentation and commentary.
We also commented on the need to ensure that data producers across organisations worked together to fill data gaps.
To ensure a broad uptake of the information, the UK government’s COVID-19 dashboard was updated to include the number of vaccinations administered and the proportion of the population who had been vaccinated, enabling people to drill down to find information about their local area.
In November 2021, Ed Humpherson wrote to the UK Health Security Agency (UKHSA) regarding its COVID-19 vaccine surveillance statistics as the data were being misused to argue that vaccines were ineffective. We welcomed changes to the presentation of UKHSA’s analysis, which made important caveats clearer, and an accompanying blog published by UKHSA to guide appropriate interpretation of the data. In April 2022, following the implementation of the UK government’s Living with COVID-19 plan, which ended free universal testing in England, it was good to see the UKHSA remove rates per 100,000 from the reports due to increased uncertainties in the data resulting from the policy change.
This was an example that received media attention from a number of sources and was also covered by Full Fact.
Case study – Use of unpublished data in a press briefing
In early March 2022, Ed Humpherson wrote to the UK Health Security Agency (UKHSA) and the Cabinet Office regarding the use of unpublished data at a February COVID-19 press briefing that estimated the cost of the UKHSA test and trace programme. We reiterated our expectation that any data used publicly by government should be published in an accessible form with the appropriate explanations. The UKHSA worked very quickly during the casework discussions and published the data before we formally wrote to them. The UKHSA also implemented a new process to rapidly publish ad hoc releases should they be needed and UKHSA and the Cabinet Office agreed to work together to ensure that the necessary processes were in place to reduce the risk of this happening again.
Case study – Quarantine requirements for arrivals into England
In July 2021, Mary Gregory wrote to the Joint Biosecurity Centre regarding the data released to support the decision on quarantine requirements for fully vaccinated arrivals from France to England. We raised concerns that the data to support this decision were not easily accessible and it was also not clear which data informed which decision.
We reiterated our expectations when it comes to the transparency and accessibility of data to inform decisions.
Following our letter, the Joint Biosecurity Centre updated its published methodology document to more clearly reference relevant data sources and provide information on analysis undertaken – including being clearer on how overseas territories were being treated.
Looking ahead to 2022/23 we expect to continue to see a decline in COVID-19 casework as the public interest shifts to other important issues such as the cost-of-living crisis and the war in Ukraine.
Use/misuse of statistics was the most common type of concern raised
As highlighted in Figure 4, the largest proportion of cases in 2021/22 related to use/misuse of statistics (41% of all cases). These cases often related to concerns about statistics being used in a way which someone considered to be potentially misleading. Most commonly the use was by a public figure or in a press release from a public body.
Figure 4: Most casework categorised as use/misuse of statistics
All casework divided by category for, 2020/21 vs 2021/22
There was an issue displaying the chart. Please edit the chart in the admin area for more details.Figure 4 shows the number of cases that were logged within each category of casework in 2021/22 and allows a comparison of these figures against the previous year.
Download Figure 4: Most casework categorised as use/misuse of statistics data source
Note: The categories in this report are provided to give an indication of the areas and issues covered by casework. There may be cases which could be classified as multiple categories, in these cases a decision has been made on which category is most relevant. Definitions of these themes can be found in our assumptions.
Some examples of casework categorised as use/misuse of statistics are highlighted in the case studies below.
Case study – Use of employment figures
Throughout 2021/22 we intervened on the public use of employment figures. Casework originally raised with us by Full Fact stated that the Prime Minister had incorrectly claimed that there are more people in work now than there were at the start of the pandemic. This statement was based on the number of payrolled employees rather than on the total number of people in paid work, which would also include the self-employed. When the whole of the workforce was considered there had been a fall in employment when compared with pre-pandemic data.
In line with our interventions policy we wrote to the Director of Data Science at 10 Downing street to raise this issue with the teams who brief the Prime Minister. Through our engagement we were able to understand that the Prime Minister was referring to the number of workers on payrolls and we started to see the correct references specifically to payrolled employment being used in statement.
However, the statement that employment had increased, without the caveat that this was based on payrolled employees only, started to be used again. So, in line with our interventions policy and the ability to escalate interventions, Sir David Norgrove (then Chair of the Authority) wrote directly to the Prime Minister making it clear that selective data is likely to give a misleading impression of trends unless the distinction is carefully explained.
This is another example of casework that received media attention. The press and other users often referenced our public letters when criticising the use of the statistic. As a result, the public were well informed about what the relevant statistics actually said and so less likely to be misled by the statements.
Case study – Use of crime statistics
Another example of casework on use of statistics relates to Sir Keir Starmer’s claim that crime is going up, raised with us by Kit Malthouse.
Sir David Norgrove wrote to Sir Keir Starmer highlighting that the Crime Survey for England and Wales (CSEW) is a better indicator of long-term trends in crime than police recorded crime, on which Sir Keir Starmer’s claim was based. The CSEW shows that total crime levels have been falling since the mid-1990s, including between the years ending March 2019 and March 2020. The letter also highlighted the risks that this statement could affect behaviour and lead to pressure for policy changes that are not supported by the data.
Case study – More use of crime statistics
Alistair Carmichael MP raised concerns with the Authority regarding the Prime Minister, the Home Secretary and the Home Office who claimed that crime continues to fall under their Government. In fact he argued that the ONS publication on Crime in England and Wales: year ending September 2021 showed a 14% increase in total crime.
Sir David Norgrove responded to Alistair Carmichael advising that we agreed that the way the Home Office had presented the data was misleading. Although the Home Office had stated that fraud and computer misuse had been excluded from its analysis, this was not consistently referenced.
This case received media attention and also started a public discussion on the impact that fraud and computer misuse has on victims and how the types of crime that people experienced changed during the pandemic.
Case study – Use of the success rate of Cart judicial reviews
In June 2021, the Public Law Project wrote to us with concerns about the accuracy of an analysis of the success rate of Cart judicial reviews in the Independent Review of Administrative Law report. The analysis found that there were only 12 successful Cart judicial reviews since 2012, representing 0.22% of all cases brought. The Public Law Project claimed that this figure is misleading because the methodology used artificially deflated the actual success rate.
We looked into these concerns, even though it is not usually within our remit to comment on the use of statistics in independent reviews. We agreed with the Public Law Project that the methodology used was overly simplistic.
We engaged with the Ministry of Justice on this matter. It agreed to review how the data and associated caveats are presented in its publications and examine the possibility of collecting improved data in the longer term.
Use/misuse of statistics has also been the most common category for almost all previous years. The only exception is 2020/21 when quality, reliability and trustworthiness was the most common theme.
The increase in the quality, reliability and trustworthiness theme last year could reflect a greater public interest in statistics. However, we do not have the MI to support the notion that the pandemic was (or was not) directly behind the switch in the most common category last year. There is no clear link between casework categories and other aspects of the data that explains the change last year to quality, reliability and trustworthiness and the reversion back to use/misuse of statistics this year.
We will continue to stand up against the misuse of statistics and intervene to minimise any misuse or to reduce the impact of it on decision-making and public understanding.
Intelligent transparency continued to be an issue
Over the past year, intelligent transparency has continued to be a cross-cutting issue in our casework. Intelligent transparency aims to support public confidence in statistics and the organisations that produce them by ensuring that statistics are available, easily accessible and clearly explained.
Intelligent transparency is informed by three principles:
- Equality of access: data should be made available to all in a transparent way. This includes providing sources and appropriate explanation of context, including strengths and limitations.
- Enhancing understanding: data shouldn’t just be released for the sake of releasing it. Data should enhance understanding of societal and economic matters, including the impacts of policy.
- Leadership: organisations need strong analytical leadership that has the authority to make decisions regarding the publication of statistics and data independently of political and policy processes.
By meeting these three principles statistics and data should serve the public good, allowing individuals to reach informed decisions, answer important questions and hold governments to account. Whilst our State of the Statistical System Report 2021/22 highlights that there is now a greater understanding of the need for intelligent transparency, this remains a key focus for OSR and one in which the casework function is able to highlight issues and work with producers to support improvements.
Equality of access
A key principle of intelligent transparency is equality of access, i.e., data quoted publicly by governments, for example in parliament or the media, should be made available to all, including links to sources and appropriate explanation of context. Availability of statistics accounted for 18% of our casework this year.
In cases where a statement has been made and the underlying data is not already publicly available, we expect an ad hoc statistical release to be published as soon as possible after the statement. We have continued to highlight our expectations around transparency and published a statement on data transparency and regulatory guidance on the transparent release and use of statistics and data. Intelligent transparency is also highlighted as a key theme in our State of the Statistical System 2021/22; we would like to see intelligent transparency being the default for all statistics and data.
Some examples of cases related to intelligent transparency are highlighted in the case studies below.
Case study – Small boat crossings
In November 2021, we raised concerns with the Home Office regarding the public use of statistics on small boat crossings that were not already included as part of an existing publication. Migrant English Channel crossings are an area of strong public interest. Ministers and the media were regularly quoting statistics that were not available to the public.
We made clear our expectations on data transparency as well as the Home Office’s obligations to meet the requirements of the Code of Practice for Statistics. The Home Office replied agreeing with the points and acknowledging the need for transparency. The Home Office committed to publish statistics on this topic and began to publish quarterly data on small boat crossings from February 2022, with the publication of daily provisional data on small boat crossing being led by the Ministry of Defence as they took on a greater role for the operational response. The two Departments work closely together to ensure coherence in these publications.
Case study – Critical care beds
In a press conference on 7 January 2022 the Welsh First Minister stated that there were a number of people in critical care beds who had not been vaccinated.
These data were not publicly available and could not be reproduced or fact checked.
We engaged with the Welsh Government and they produced an ad hoc release providing the source of the data that was used to make this statement. We wrote to the Chief Statistician for Wales to thank them for their engagement but also to reiterate our expectations when it comes to intelligent transparency.
Case study – Homes for Ukraine sponsorship scheme
Towards the end of 2021/22, we received concerns regarding statements made in the media and parliament about the number of households or organisations that had signed up as potential sponsors under the Homes for Ukraine sponsorship scheme.
We raised these concerns with the Department for Levelling up, Housing and Communities (DLUHC). Our concerns were that the public did not have access to the underlying data and that the statistic had been used but was not available. Even though information had been publicly quoted on a topic that was very much of interest to the public, it was not released in an intelligently transparent way. We highlighted this and asked for forward analysis plans to be published so that the public would be informed about the availability of additional data.
We received a response from DLUHC in which it recognised the importance of intelligent transparency and committed to publishing information as soon as possible. DLUHC quickly worked with Home Office and the devolved administrations to develop and publish from April 2022, weekly sponsorship scheme visa data broken down by local authority for England, Wales, Scotland and NI. This provided detailed breakdowns of Home Office weekly headline national visa figures on the number of applications to the Ukraine sponsorship scheme, and the outcomes of those applications.
DLUHC supported intelligent transparency in the information that it published by providing guidance notes to highlight the limitations of the data to support its accurate interpretations by users. DLUHC also voluntarily applied the Code of Practice for Statistics when producing the data. Its statement of voluntary compliance with the Code is available as a tab in each data spreadsheet, while the webpage states that: ‘the principles of transparency of high-quality analytical outputs to inform decision making and the public underpin this data release.’
DLUHC has since separately also published new management information on homelessness duties owed to Ukrainian nationals and visa data by age and sex of applicants. Each of these contain guidance notes to enable an intelligent interpretation of the data by users.
We will continue to champion and encourage improvements to making data and statistics easier to find, accessible and understandable for users. We will also continue to monitor and log examples of intelligent transparency in our casework to build up our evidence base and to support our intelligent transparency campaign.
Most casework came from the public, with social media playing a bigger role this year
Through our casework function we monitor the public use of data and analysis by UK governments and respond to issues raised with us by a range of users. Understanding who is raising casework issues with us, as well as responding appropriately, is a key part of widening our reach and ultimately serving the public good.
As Figure 5 shows of the 241 cases investigated in 2021/22, the highest proportion of cases came directly from members of the public who emailed us with their concerns (49%). This is a slight drop from 2020/21 when casework from the public represented 53% of all cases. Internally generated casework accounted for 22% of all cases, a larger proportion but similar number to 2020/21 (16%). In 2021/22, there was a drop in the number and proportion of cases (11%) from the user/academic community compared to 2020/21 (17%).
Figure 5: Most casework was received from members of the public
Casework categorised by source
Figure 5 shows the number of cases by source of the complaint for 2021/22 and allows a comparison of these figures against previous years.
Download Figure 5: Most casework was received from members of the public data source
Social media is increasingly impacting our casework, including how we share information about our casework, how we identify potential new casework and how people raise concerns with us. Although we do not respond directly to anyone on Twitter about casework, we do use our @StatsRegulation account to highlight public interventions we have published as well as to horizon scan for issues that the public are concerned about. Often these concerns are logged as internal cases as our team has picked up an issue from the media or from social media that we might want to explore further. Additionally, for the first time, we received cases through a social media campaign, as shown in the case study below.
Case study – Patients in specialist care by vaccination status
In December 2021, we received 43 identical pieces of casework from 43 different individuals. The casework was regarding a tweet by the Secretary of State (SoS) for the Department for Digital, Culture, Media and Sport and an associated media article which contained unpublished statistics on the proportion of unvaccinated people in Intensive Care Unit beds. The individual who originally raised the case with OSR had started a Twitter campaign which gave correspondents a template letter to use and encouraged them to write to us.
Normally when members of the public write to us we do not publish our correspondence. However, for this case we did as it was clear that there was a high level of public interest in the issue and we wanted to remain transparent with our response.
We wrote to NHS England thanking it for publishing the relevant data and the SoS tweeted an update using the most up to date statistics.
Given that the cases we received through this social media campaign make up almost 18% of casework this year, it is important to highlight them and the impact they had on the statistics presented in this report. Though casework affected by social media campaigns has an impact on our casework management information, it follows the same process and time scales that would be followed had we only received one complaint. The number of complaints that we receive on an issue does not impact the importance we place on that casework. All casework concerns are considered and addressed based on the contents of that complaint.
If we had received only one complaint from the original complainant regarding unvaccinated people in ICU beds, our total number of cases would have been 18% lower. Our median and mean response times increase when we take away these duplicate cases, increasing our time taken to respond from 11 median days to 17 median days and from 20 mean days to 24 mean days. The most common source for cases remains members of the public at 38%, despite the reduction in 42 cases. The most common theme remains health and social care statistics, down from 54% to 45%. The percentage of cases that were pandemic related also decreases from 52% to 42%.
We will continue to monitor the impact of social media on our casework function. This includes the opportunities that social media offers, for example the greater scope for identifying issues for us to consider, and the potential risks such as the echo chambers that social media can create.
We saw impact from our interventions
We want to maximise the impact of our casework to ensure that statistics serve the public good and that the public can have confidence in the statistics produced and used by government. We have been working on (and will continue to develop) our understanding of how we can maximise and secure impact for each case.
In 2021/22, we developed an impact chain that enables us to understand the intermediate and final outcomes that lead to our overarching impact to protect the use of statistics and prevent misuse of statistics in public debate. Key outcomes identified include where we have made significant impacts on statistical production within organisations and references to our casework in the media, demonstrating that we are taking appropriate action to support public confidence.
Impacts that we have identified through tracking our casework have included that: data have been released following our intervention; misleading tweets or blogs have been removed; improvements have been made to methodology information or referencing of the source of statistics; and analysts have been involved in the drafting of blogs and press releases.
We monitor our media coverage as a way to understand the impact of our interventions. We also work with media partners to promote our public interventions to ensure that any misunderstanding regarding official data can be reduced. During 2021/22 there were several cases that received strong media coverage. Where relevant these have been highlighted in the case studies within this report with links provided to some of the stories in the media.
A good indicator of our impact on protecting the role of statistics in public debate are mentions of OSR, our public casework interventions and letters that the Director General for Regulation or the Chair of the UK Statistics Authority have written to producers or users of data in Parliaments across the UK. Annex B provides further details of specific references to Authority interventions in Parliaments across the UK, including excerpts from Hansard and the Official Report.
Table 2 below highlights the number of cases we have received from members of Parliament and devolved parliaments. This includes cases raised by current members of Westminster and Scottish Parliaments. In 12 of these cases (92%) the Authority’s response was published and is available on our website.
Health and Social Care accounted for 38% of the cases raised by members of parliaments across the UK, and 38% of cases related to the pandemic. Crime, poverty and education were other topics raised by members of parliament.
Table 2: Cases raised by members of Parliament
| Administration | Number of cases received |
|---|---|
| England | 8 |
| Wales | 0 |
| Scotland | 5 |
| Northern Ireland | 0 |
During 2021/22, our casework interventions were mentioned in the House of Commons and House of Lords 15 times. There was one mention of an intervention during this period in the Scottish Parliament. There were no mentions of the Authority, OSR or our interventions in the Welsh Parliament or the Northern Ireland Assembly.
One of the outcomes that we are seeking is that the public has confidence in the statistics used by government. One source that can give some insight into levels of trust in official statistics is the 2021 study of public confidence in official statistics. It found that, amongst people who responded, there was high confidence in the statistical system.
While respondents did not necessarily know about the Authority or the OSR, there was strong support for our role, with 96% of respondents agreeing there should be an independent body to speak out against the misuse of statistics and 94% agreeing that such a body should ensure that statistics are produced free from political interference.
Although this is only one source, and we are careful not to place too much weight on a single survey result, we do consider that this provides some reassurance around public confidence in official statistics.
We will continue to identify and find ways to increase the impact of our casework for the public good.
Back to top