Executive Summary

This is the final report on our review of statistics on gender identity based on data collected as part of the 2021 England and Wales Census.

Following the first release of census statistics on gender identity in England and Wales in January 2023, concerns were raised publicly with ONS and with us about the published estimates of the trans population. As additional census data were published, these concerns extended to the relationship between gender identity and proficiency in English. We also received concerns about the level of methodological information published.

Learning from new evidence in Scotland’s Census, the Office for National Statistics (ONS) wrote to us on 5 September 2024 to request that the gender identity estimates from Census 2021 should no longer be accredited official statistics and should instead be classified as official statistics in development.

There is great value in ONS having undertaken this work on developing these statistics. We recognise the challenges in collecting data at scale on a sensitive topic for the first time. While there are lessons to be learned, ONS has worked with good intentions aiming to address an important data gap on gender identity.

This commitment is demonstrated by ONS recognising that it can do more to support users to interpret the census gender identity data and to understand any limitations, and has led to its request to cancel the accreditation of the gender identity estimates from Census 2021. ONS’s decision to label these data as statistics in development supports its desire to publicly reflect the evolving understanding of gender identity and to undertake further research. We welcome statistics producers approaching us when they have any concerns about the statistics they produce.

We have written to ONS to support its proposal to publish the Census 2021 gender identity statistics in England and Wales as official statistics in development rather than accredited official statistics. We found that the issues highlighted in this report are unique to the statistics on gender identity and therefore all other outputs from the Census 2021 in England and Wales are unaffected by this decision and remain designated as accredited official statistics.

This report sets out the details of our regulatory decision and shares our recommendations to ONS on the steps it must take to help users of the census gender identity statistics understand their strengths and limitations. In addition, we have also set out the development work we consider is required on the Government Statistical Service (GSS) gender identity harmonised standard. This includes a need to publish additional advice for statistics producers who are currently using the standard and for those who are thinking about adopting it. The GSS Harmonisation Team must consider these recommendations as it further develops its workplan. This report has been reviewed and approved by the Authority’s Regulation Committee, which oversees the work of OSR.

While we recognise ONS’s commitment throughout the census operations to obtaining sound evidence, we have found that that the Census 2021 gender identity statistics published by ONS do not fully comply with the following areas of the Code of Practice for Statistics:

Quality – Suitable data sources

As ONS has now recognised, the question developed to determine the size of the trans population of England and Wales did not work as intended. The evidence indicates that people may have found the question confusing and therefore gave a response that did not reflect their gender identity. This appears to be more likely for people who do not speak English as their first language. It is important to note that neither ONS, nor we at OSR, identified in the development or delivery process of the census that there was a significant risk that certain groups of people were more likely than others to misunderstand the question.

Assured Quality

There are a number of areas where ONS could have better communicated with users of the census gender identity statistics. Our interim report found that ONS’s communications did not sufficiently manage users’ expectations on what areas ONS was exploring and what the research might show. Additionally, until recently, ONS’s approach has been characterised by a focus on defending the estimates of the size of the trans population. ONS has not shown sufficient willingness to engage constructively with the emergent evidence. The publication of the Scotland Census data on trans and trans history in June 2024 has been instrumental in a shift in ONS’s approach.

We have confidence that those working on the data and research have been working in line with the Code, but consider that stronger leadership from ONS, particularly on clearer public messaging when concerns were raised and on setting expected outcomes for ONS’s subsequent research work, was needed.

Communicating uncertainty

We have found that ONS has not done enough to communicate uncertainty and that users who use the census outputs at a national level would struggle to understand the impact of ONS’s published research on the use of the gender identity data. This problem is replicated at the local level, where the research indicates that the variability of the estimate may be greater.

ONS has conducted a range of research activities to explore the quality of the estimates, but these efforts have been hampered by a reluctance to communicate the limitations of the gender identity statistics. ONS’s headline message in the research report around the validity of the census data, and how users should interpret and use the data at national and smaller area levels, is unclear.

We have also considered three further factors relating to the status of these estimates:

ONS’s approach to criticism of the gender identity data

We noted in our interim report that the content and tone of the published communication from ONS has been somewhat closed and at times defensive. Some users have raised concerns that ONS has been captured by interest groups, leading to a lack of objectivity. We found no evidence of this form of bias through any of our work. We consider it regrettable that ONS’s defensiveness has created an impression of bias to some external observers.

Whether the statistics are materially misleading

We also considered whether the statistics are materially misleading. The purpose of the Code’s Quality pillar is to ensure that statistics are a fair representation of what they seek to measure and are not materially misleading.

At the national level, triangulation with other sources, including data from the Scottish Census, suggests that an estimate of around “1 in 200” triangulates with other sources and is not likely to be materially misleading. While the information on the size and nature of all the potential biases is incomplete, it is hard to draw the same conclusion for some more-detailed breakdowns, including for local areas where the data indicate a higher concentration of people misunderstanding the question.

Our regulatory approach

When developing new statistics or introducing a new data source or methodology, initial publications are often labelled ‘official statistics in development’ by producers. We would usually assess the official statistics against the Code of Practice once the development is complete.

However, the census is a unique case. Our regulatory accreditation process was undertaken in tandem with the development and first publication of the statistics in the respective countries of the UK. While this process allows us to provide users of these statistics with the reassurance of quality that comes with accreditation at first publication, it means that if we have any subsequent concerns about any aspect of the Code of Practice for Statistics, we may need to reconsider our accreditation decision in light of new evidence.

In this case, new evidence has emerged that has demonstrated that the statistics do not comply with important quality aspects of the Code of Practice for Statistics. With the benefit of hindsight and this new evidence, our conclusion is that the census gender identity statistics, which were collected from a novel question in an area where measurement practices are still emerging, would be better described as official statistics in development, and should not be labelled as accredited official statistics.

We acknowledge that this is a limitation with our approach, and as a result of this review, we are reviewing what changes might be needed to our assessment approach in circumstances where accreditation at first publication is in line with serving the public good.

Required future work by ONS and the GSS

There is great value in ONS having undertaken this work on developing how people are able to describe society. We recognise the challenges in collecting data at scale on a sensitive topic for the first time.

Our recommendations focus on the steps ONS must now take to help users of the census gender identity statistics understand their strengths and limitations and the development work we consider is required on the GSS gender identity harmonised standard.

Guidance for use of the census gender identity statistics

• It is important that ONS communicate clearly across all its census outputs what ‘official statistics in development’ means in practice for the census gender identity statistics.
• In doing this, the varying levels of information across the gender identity publications and data tables must be addressed by ONS across its suite of publications and data on gender identity.
• ONS should provide more practical information, according to the needs of stakeholders, to help users understand the uncertainty inherent in the data and how this affects their use and interpretation. This information could include sensitivity analysis, and example use cases published at different levels of geography and population highlighting any likely anomalies or implausibilities.
• ONS should consult with us as it develops these communications, which should be treated as a priority workstream.

Gender identity question development

These data were identified as being important for society, and so it is important that the original user need for robust data on gender identity be addressed for future statistics. ONS should consider how these data needs are best addressed in future as part of its further testing of the question.

We consider that there is sufficient correlational evidence that the gender identity question in the census has underperformed, and some users, producers and stakeholders have lost confidence in the suitability of the gender identity question.

This issue is important as the census gender identity question is also the question used in the GSS gender identity data harmonised standard. As we set out in a letter to the Deputy National Statistician with oversight of the Government Statistics Service (GSS) in February 2024, we are seeing in our regulatory work the implications of the lack of confidence in the harmonised question by producers, who are unsure what gender identity question they should use and feel unsupported by the Government Statistical Service. In particular:

• We can confirm that our interim recommendation remains: namely that, as part of the ongoing development of the GSS harmonised standard for gender identity, ONS should carry out further testing of the question. ONS must give consideration to developing and testing a harmonised question that commands the confidence of producers and users. The merits of the Scottish Census question should also be considered in this process.
• This work on the question should be considered as part of the outcome for the Future of Population and Migration Statistics consultation, in determining the future production of these statistics in an administrative data-based framework.
• Given that the census gender identity question is also the GSS harmonised standard, ONS should prioritise this work, ensuring that adequate resources are made available.
• We consider that there is an immediate need for the GSS to publish advice for statistics producers who are currently using the gender identity question and for those who are thinking about adopting the standard. This guidance should set out the limitations of the question, reference the Scottish Census approach and caution against knee-jerk changes to established collections until ONS has undertaken further research.
• We recommend that stakeholders be fully involved and engaged in future work, to help build and restore confidence.

The GSS should publish its initial plans for developing the gender identity harmonised standard within three months of our report’s publication.