Executive Summary
The census is one of the most important sources of data and statistics, informing decisions about almost every aspect of life within the UK. It allows users – including government, local authorities, academics, and commercial businesses – access to important information on the people and households of the UK and helps people get a better understanding of the places in which they live and work. It is essential that the data and statistics from the censuses are reliable and provide valuable insights, meeting the rigorous standards of trustworthiness, quality and value outlined in the Code of Practice for Statistics.
What we found
The National Records of Scotland (NRS) has undertaken many years of planning and development ahead of delivering census operations for Scotland in 2022. When it became clear that return rates from census collection were not as high as anticipated, NRS worked swiftly to improve returns and is reviewing and revising its statistical methods, on the advice of international census experts, so it can produce the best possible estimates of the population of Scotland. The OSR assessment team recognises the concerted efforts and commitment of NRS at an especially pressured time.
The steps NRS has taken to adapt its statistical methods are examples of good practice when faced with challenging circumstances – it is drawing on expertise and exploring new solutions in a flexible and considered way. Users have highlighted some concerns about the quality of elements of census data such as for certain geographical areas. NRS has published information on the plans it has for census data and statistics.
However, NRS has not yet published information for users on how the level of data quality will be improved through its adapted methods or the likely quality of resultant census estimates.
When NRS publishes its census outputs, from autumn 2023, it will accompany these with information and guidance for users on data quality. It is likely that more-detailed quality information will be produced as NRS’s programme of outputs are delivered. Given issues faced by NRS during its census collection phase, concerns raised by users, and known areas of change, it will be particularly important for NRS to ensure accessible and prominent information is available to support users – for example, on data for new or amended census topics and where comparability across UK censuses is a factor.
NRS has amassed a wealth of experience and opportunity from delivering Scotland’s Census. For example, it has delivered a large scale predominately online data collection system, it has developed new flexible data access tools for users, and it has worked to face challenges such as lower response rates – which is an issue facing many statistical surveys in recent years. NRS is an active participant in the statistical and census community in Scotland, the UK, and internationally. These communities would benefit enormously from NRS sharing its knowledge and experiences, advancements in technology and statistical methods, and the lessons it has learned from Scotland’s Census 2022.
Back to topRequirements
We have identified several areas that NRS needs to address to support appropriate use of the data and meet users’ needs to comply with the highest standards of the Code.
NRS demonstrated its trustworthiness and was transparent in the early stages of census planning by providing users with its expectation of the quality for census outputs through, for example, its KPIs on accuracy. While specific quality measures will only be possible as part of its census outputs, NRS should be open ahead of those outputs about how its developments in data and methods are likely to affect the quality of estimates that will ultimately be produced.
While NRS has been open to discussing data quality concerns with users, its engagement strategy may benefit from review to ensure it reaches a broad range of users.
Requirement 2: NRS should be transparent about its approach to public engagement with regard to its confidence in data quality. NRS should review its communication and engagement plans, prioritising users where concerns over data quality are greatest and where there is most risk from inaccurate census estimates.
NRS introduced a new trans history question for Scotland’s Census 2022 and there have been concerns raised over the collection of data about sex. The trans history question used in Scotland and the guidance for the sex question differs from the equivalent in the 2021 Census in England and Wales.
Requirement 3: NRS should ensure that, when it publishes statistics on trans history or produces demographic breakdowns by sex, these are accompanied by clear information on the definitions and guidance used to produce those statistics. NRS should explain the coherence and comparability of its statistics from the 2022 census with those from other censuses, especially given the differences in guidance for the census sex question.
NRS’s plans to provide quality information alongside its census outputs should be delivered to support and assure users of the value and quality of the data and statistics. These plans should take account of key areas of interest or concern raised.
Requirement 4: NRS should ensure that supporting documentation, guidance and information on data quality, including bias and uncertainty, is communicated in its outputs. This information should be refreshed and added to as its programme of outputs is delivered. For known areas of user interest, for example for data on sex, NRS should provide detailed information on quality indicators, such as the outcomes from its quality assurance processes, in assessing census estimates against other data sources.
UK census data will be affected by various factors including differences in the timing of census across the countries of the UK. While responsibility for UK estimates lies with ONS, users are likely to reach out to both NRS and ONS for information and support.
Requirement 5: NRS should take further steps to communicate plans and provide more-detailed information to users of UK census data in Scotland. NRS should continue to work together with the other census offices to explain any impacts on UK census data quality and describe where user needs may or may not be met as a result.
Judgement on National Statistics Status
We have identified five requirements for NRS to address in order to ensure the high standards of public value, quality and trustworthiness associated with National Statistics designation are met. Once NRS has demonstrated that these requirements have been addressed, or provided sufficient assurance that our expectation for the data and statistics will be met, OSR will recommend to the UK Statistics Authority that National Statistics status for these statistics be confirmed. NRS is aiming to meet the requirements of this report in the coming months so a designation decision can be made ahead of first census outputs in autumn 2023.
OSR expects to publish its findings and judgements on how NRS has responded to these assessment requirements and the outcome of the UK Statistics Authority’s decision on National Statistics designation in late summer 2023.
Back to top