Executive Summary

ES. Introduction

ES.1 The gender identity question was asked, on a voluntary basis, for the first time in the 2021 England and Wales Census, conducted by the Office for National Statistics (ONS). The question represents the current Government Statistical Service harmonised standard in development for collecting data on gender identity.

ES.2 The first release of statistics on gender identity based on Census data was in January 2023. When publishing the results ONS said: “Gender identity refers to a person’s sense of their own gender, whether male, female or another category such as non-binary. This may or may not be the same as their sex registered at birth.”

ES.3 Measuring gender identity is undoubtedly challenging. It is inherently personal and therefore can be a sensitive topic. Those whose gender identity differs from their sex represent a small proportion of the population which creates challenges for data collection. The concept may be unfamiliar to many people and there are few robust alternative sources of data to cross-check against.

ES.4 Following the first release, concerns were raised with ONS and the Office for Statistics Regulation (OSR) about the published estimates of the transgender population. As additional breakdowns were published, these concerns extended to the relationship between gender identity and proficiency in English (or Welsh). Users also raised concerns about the level of methodological information published.

ES.5 ONS initiated research in response to these concerns in April 2023. We then announced our plans to carry out a short review of these statistics and the findings of ONS’s research, formed around the principles within the Code of Practice for Statistics. The review considers ONS’s approach to responding to questions raised by users about the results of the gender identity data. It does not consider gender identity as a concept or the decision to collect data on gender identity.

ES.6 On 30 August ONS cancelled the publications and released a statement advising that the research is ongoing and that it will publish further updates in due course. This delay is likely to have disappointed and impacted some users.

ES.7 In light of this delay, we have produced an interim report. This interim report sets out our expectations of what is needed from ONS to provide assurance on the gender identity data and harmonised question in line with the Code. We have based these on what we know so far about how ONS tested, quality assured, communicated and engaged with users, both in advance of publishing the results and where appropriate from the research to date.

ES. What we found so far

ES.8 The review considers ONS’s approach to responding to questions raised by users about the results of the gender identity data. This interim report looks at four aspects: how ONS tested the question; how this compares with the approach taken in the Scottish Census; the approach to quality assurance; and how ONS responded once some users raised concerns about it, including addressing uncertainty.

ES.9 ONS worked with good intentions when seeking to address an important data gap on gender identity. These intentions guided its approach to the question design and testing process, and to the processing and dissemination of the resulting data.

ES.10 ONS considered language proficiency as part of its wider approach to testing the questionnaire and its own research pointed to the need for more targeted language testing on gender identity. ONS should explore ways to improve participation of those with lower English language proficiency as part of any further testing of the harmonised standard.

ES.11 There were strong working relationships between ONS and National Records of Scotland (NRS) in sharing updates and learning as they went through testing, processing and quality assuring the gender identity data. We recognise that different contexts and user need shaped the choice of questions used by ONS and NRS. However, on some aspects of the questions, ONS and NRS received conflicting feedback from similar user groups. To inform future development and use of the gender identity question, we consider that further investigation is required to understand these different user perspectives and needs in more detail.

ES.12 The general approach taken by ONS to quality assure the data, when considering it alongside the other variables, was proportionate particularly given the lack of data on this topic area. The approach involved independent scrutiny and expertise when choosing what areas to probe and analyse. In hindsight, given it was a new question, ONS could have prioritised analysis of the agreement rates with the Census Quality Survey as part of its quality assurance process, before releasing the data. Further clarity is also needed to help users understand how the sex and gender identity questions were asked and relate to each other, given the correlation has been perceived by some as an indication that the question has been misunderstood.

ES.13 Given that this is a contested area, with little data available for triangulation, it would be reasonable for users to have expected clearer messaging on the impacts on quality and uncertainty. This gained added importance when users raised challenges and offered alternative hypotheses to explain the data. In this context ONS now needs to undertake further analysis and be clear on what can and cannot be concluded from the data.

ES.14 ONS has been keen to deliver public value by releasing these new data on gender identity in a timely manner. ONS has taken steps to publish updates on its research around these data and we have seen positive engagement on these issues in our interactions with ONS. We recognise the complexity and sensitivity of the research ONS is conducting and agree that this should not be rushed. However, the published communication to date does not sufficiently manage users’ expectations of what areas ONS’s research into these data is exploring and what it then might show. A more open approach would manage users’ expectations around the use of the data and lay the ground for any changes in advice on the use of this data resulting from ONS research.

ES.15 The message that ONS provides upfront in a box on the gender identity data harmonised standard page is a good example of a clear and honest acknowledgment of uncertainty without the need to quantify it. This approach should be adopted more broadly for ONS’s outputs and quality information surrounding the gender identity data to ensure the data are not open to misuse and misinterpretation.

ES. Expectations for Further Work

ES.16 Throughout our review, it has been clear that the ONS team leading the quality assurance and additional exploration of the data is committed to understanding the quality of the data and how this can best be communicated to users.

ES.17 ONS’s research on gender identity should seek to understand to what extent, if any, has a misunderstanding of the gender identity question in the 2021 England and Wales Census led to a significant misestimation of the transgender population. When the target group is small, the effect of incorrect responses on the data will be amplified.

ES.18 ONS should consider the testing and operationalisation of the question as well as the production and analysis of the results. If there is evidence that there has been a significant misestimation of the transgender population, to a degree that might affect usage, the research should set out both the implications for the use of the results from the Census and the use of the GSS harmonised question in future data collection. We expect ONS to take on board the following recommendations as it works on developing its research.

ES.21 As this interim report makes clear, there was always good reason to expect a significant degree of uncertainty around estimates of the transgender population derived from the gender identity question in the 2021 Census. A willingness on the part of ONS to provide further context and clarity on the initial estimates would be evidence of the strength of the Census process and not a weakness. ONS providing updates to the interpretation of these estimates should be regarded as a normal part of ongoing statistical production for a new area of data collection. The same would be true of any changes to the initial estimates, as long as they are fully and transparently explained. Such updates should not undermine user confidence in the robustness of the Census results as a whole, in which most questions have been asked many times before.

ES.22 We are satisfied that the overall approach to the 2021 Census complies with the standards of trustworthiness, quality and value in the Code of Practice for Statistics in order to achieve ‘accredited official statistics’ status. We will consider the accreditation of this subset of the Census dataset when ONS publishes its research findings. We will publish a follow up report with our findings after we have reviewed, and engaged with users on, the full set of research.