Dear Roger
Scottish Vacant and Derelict Land Survey Statistics
As you are aware we recently completed our compliance check of your Scottish Vacant and Derelict Land Survey (SVDLS) statistics against the Code of Practice for Statistics. Our review found a range of positive features that demonstrate the trustworthiness, quality, and value of the statistics:
- The statistical bulletin has a well-presented key findings section and uses a mix of charts, tables and maps to aid user interpretation
- The strong links the team has with policy colleagues help ensure the relevance of the statistics, and links with local authorities that supply the data (accompanied by clear published survey guidance for authorities to follow) help ensure data quality
- Transparent communication about the decision to not publish 2020 SVDLS statistics, due to data quality concerns related to the Covid-19 pandemic
- The statisticians sharing knowledge of new desk-based surveying approaches developed by some authorities through the pandemic, with other authorities where challenges to data collection remain, to enable data returns for those authorities
- The team’s plans to increase the public value of the statistics to reflect planned changes to the National Planning Framework, which are due to be laid before the Scottish Parliament during autumn 2021. These include plans to review and redesign the bulletin to present more insightful commentary upfront, including on the main users and uses of the statistics, and to reduce the volume of tables currently presented. They also include plans to develop new interactive content to support use by wider range of users, making the data accessible through a new digital platform in line with a drive towards digital e-planning, while drawing on users’ and policy colleagues’ feedback on the changes
We also identified some areas where more could be done to improve the trustworthiness, quality and value of the statistics:
- The Highland authority has not undertaken a survey since 2015, reducing the quality and representativeness of the SVDLS estimates. Highland council has put plans in place to complete an updated return for 2021 and the statistics team informed us that planned changes to the National Planning Framework and new desk-based approaches may also help with achieving this
- The bulletin could be clearer about the nature and extent of uncertainty associated with the estimates, to aid user interpretation, including in relation to regular revisions when surveys are not carried out in some authorities in certain years, and due to post hoc definitions changes. Including the impact of the latest year’s revisions in the table of revisions for previous years would help enable a better understanding of the overall impact of revisions on previous estimates for users.
- Set out the quality assurance steps the team undertakes and review its assurance processes for the source data, ideally using our Quality Assurance of Administrative Data (QAAD) framework. In line with this, we would expect producers to communicate their knowledge of data suppliers’ quality assurance checks alongside a description of producers’ own checks on the data, to demonstrate transparency and enable users’ appropriate interpretation of statistical quality
- The team told us about its approach to engaging with users of the statistics and has agreed to review and refresh the outdated information on users and uses of the statistics and make it more prominent in the bulletin. Publishing details about the team’s approach to engaging with users from a wider audience, and details of the team’s future development plans, will increase the transparency of planned developments to the statistics, and help foster wider and better engagement with a broader range of users. We recently published guidance on user engagement which the team may find useful in planning future user engagement activities
- Take advantage of the work being undertaken in digital e-planning to explore the potential of SVDLS data to be linked to other data sources, such as those related to flood risks and contaminated land to add further insight. We recommend that the team explores the potential to innovate or add insight by collaborating with developments across government, such as new approaches to measuring land use stock developed by the Ministry for Housing, Communities and Local Government.
We understand that the statisticians are currently discussing with local authorities, arrangements for conducting a 2021 survey, and a decision concerning the publication of 2021 statistics will be made later this year, once a fuller assessment of data quality has been completed. The statisticians explained that subject to the quality of the data, it could be feasible to produce an interim publication to reflect the latest position, making clear it is provisional and providing appropriate caveats to enable its appropriate interpretation by users. We would welcome this approach should you decide that it is feasible.
I would like to thank your team for their positive engagement on this review. Please do not hesitate to get in touch if you would like to discuss any aspects of this letter further. I am copying this letter to Adam Krawczyk, Head of Housing, Homelessness and Regeneration Analysis at the Scottish Government, and Charles Brown and Janice Blanc, the responsible statisticians.
Yours sincerely
Mark Pont
Assessment Programme Lead