Dear Sandra
Statistics on Statutory Homelessness in England: Confirmation of accredited official statistics status
We have reviewed the actions that your team has taken to address the requirements in Assessment Report 364: Statistics on Statutory Homelessness in England. A summary of this review is presented in Annex A. On behalf of the Board of the UK Statistics Authority, I am pleased to confirm that Statistics on Statutory Homelessness in England comply with the standards of trustworthiness, quality and value in the Code of Practice for Statistics and should be labelled as accredited official statistics. Accredited official statistics are called National Statistics in the Statistics and Registration Service Act 2007.
The improvements that you have made to these statistics will help better answer users’ key questions on homelessness. The statistics are important to a wide range of users within and outside of government. Your statisticians’ commitment to a new annual user engagement cycle will proactively ensure that a wider range of users’ needs is understood, their use of the statistics is supported, and the value of future developments is maximised.
The insights that the statistics provide for answering users’ key questions has been considerably enhanced. The annual flows analyses and dashboard demonstrate the real value of case-level data for monitoring homeless households’ outcomes, while a range of other dashboards support users to further interrogate the data. Additional value for answering users’ key questions has been achieved by making annual datasets available through the ONS Secure Research Service.
I also commend your team’s approach to working closely with local authorities and DLUHC’s Homelessness Advice and Support Team, to overcome difficulties that some local authorities have experienced in submitting accurate returns, particularly on temporary accommodation. This has resulted in the proportion of authorities providing temporary accommodation data rising to 94 per cent in the latest collection. Supplementary narrative about data imputation along with additional notes and guidance helpfully set out how the quality of the statistics is assured and factors that may impact their uncertainty. This guidance further supports the appropriate interpretation and use of the statistics.
Finally, I would like to thank your statisticians for their positive engagement with my team during this assessment. I am copying this letter to Madeha Asim and Simone Capp, the responsible statisticians.
Yours sincerely
Ed Humpherson
Annex A: Review of actions undertaken in response to Assessment report 364: Statutory Homelessness in England Statistics produced by the Department for Levelling Up, Housing and Communities (DLUHC)
To ensure that the statistics are useful, easy to access and support understanding of important issues, DLUHC should improve the clarity of the statistics’ main messages to help illustrate what the statistics show in terms of their value for answering key questions on homelessness, as well as their main strengths and limitations. DLUHC should also improve the accessibility of the current statistics package and website to aid navigation by a broader range of users, and so that the statistics can be appropriately accessed and interpreted.
Actions taken by DLUHC to meet the requirement
DLUHC has reviewed the range of current homelessness outputs to ensure key messages are presented more clearly. It has produced an infographic for users with visual representations of the data to illustrate key messages and trends. This also includes ‘Notes on usage’ and ‘Uses and limitations’ sections, which provide a steer to users on the questions can be answered through the statistics and those that cannot. This information is also within the statistical release, with further detailed explanations presented in the technical note.
DLUHC has added key definitions into the release for a better user experience and has used signposting to ensure clearer navigation between the different output products. The statistics are now published in html to support accessibility for a wider range of users.
A range of data dashboards now also supports interrogation of the data in new ways. This includes quarterly statutory homelessness, and local authority (LA) performance dashboards, and annual flows and support needs dashboards.
OSR’s evaluation of evidence
DLUHC has considerably improved the clarity and presentation of the statistics and the signposting and links between the different homelessness products. Improving the website layout, removing outdated content, and publishing the statistics in html has increased accessibility for users and helps them navigate to the right data for their needs.
The new annual flows analyses and dashboard highlight the value added by the case-level data for monitoring homeless households’ outcomes, and allow users to further interrogate the data. The other dashboards allow users to understand key trends on other specific homelessness variables of interest.
The new infographic is a welcome addition and helpful in getting some of the key findings across with supporting visualisations, including on the strengths and limitations of the statistics. This will support appropriate use of the statistics.
To ensure that users’ needs are fully understood, and their use of the statistics is supported, DLUHC should engage with a greater number and broader range of homelessness statistics users on an ongoing basis. DLUHC should review and update its user engagement strategy and set this out publicly, drawing on the updated GSS user engagement strategy. It would also be helpful for the team to update the GSS user forum web pages to include information of interest to a broad range of users of the statistics, beyond LA users.
Actions taken by DLUHC to meet the requirement
DLUHC has published a new user engagement strategy and has engaged with relevant Government Statistical Service (GSS) teams to ensure its plan aligns with the GSS user engagement strategy.
In 2022 DLUHC introduced a new continuous user engagement survey to collect feedback from users to inform further improvements to the statistics.
DLUHC plans to publish a new ‘Use, Improvements, and User Engagement’ page alongside its annual release in October 2023, to keep users updated with progress on developments annually and set out its commitment to undertake a yearly engagement cycle. This will include an annual newsletter, and inviting user feedback on new features and forthcoming ideas on further improvements.
DLUHC updated the old GSS user forum pages to make them more accessible. It intends to use these pages for data suppliers and to engage with a wider set of users through its new user-focused web page and annual engagement cycle.
OSR’s evaluation of evidence
We judge that this requirement will be met when DLUHC publishes its new ‘Use, Improvements, and User Engagement’ page in October 2023, which will set out its commitment to an annual user engagement cycle.
The new user engagement strategy outlines DLUHC’s approach to ensuring ongoing contact and feedback from a broader range of users. The new user survey provides an opportunity for users to continuously feedback views on how the statistics can be used.
DLUHC’s commitment to a new annual user engagement cycle will proactively ensure that a wider range of users’ needs are understood, and their use of the statistics is supported on a continuous basis. This will help ensure the value of future developments is maximised.
To realise the public value of the statistics and support the information needs of a broad range of users, DLUHC should ensure that users have opportunities to contribute to development planning; are aware in advance of developments being launched; and have opportunities to contribute their views on new developments once published. DLUHC should be clear and transparent in its communications about which user requests for further developments it plans to address, and those it does not, and the rationale for its decisions.
Actions taken by DLUHC to meet the requirement
DLUHC has engaged with the relevant GSS teams for advice on how best to engage with users and request feedback. DLUHC has sought feedback from users ahead of upcoming developments through its continuous user survey.
DLUHC has added a section in the technical note and in previous releases to highlight recent developments to users, for example, the annual homelessness dataset now being available to approved researchers through the ONS Secure Research Service (SRS).
This approach will be further reinforced through DLUHC’s new annual user engagement page, which will further highlight forthcoming developments and seek users’ feedback on further improvements. DLUHC has committed to being open about those requests it can and cannot address in subsequent publications.
OSR’s evaluation of evidence
We judge that this requirement will be met when DLUHC publishes its new ‘Use, Improvements, and User Engagement’ page in October, and the associated commitment to an annual user engagement cycle.
The additional information included in the technical note and previous bulletins on forthcoming and recent developments will ensure that users have opportunities to contribute to future development planning.
Additional value for answering various questions of homelessness statistics users has been supported by a de-identified annual homelessness dataset being made available to approved researchers within the SRS. This is publicised within the technical note.
To enable all LAs to return data through H-CLIC, and to help drive improvements in the quality of the data returned, DLUHC should:
- work with LAs that are unable to provide accurate H-CLIC data, including those not currently engaging with the team, to gain a better understanding of the specific barriers that they face, and overcome outstanding issues
- facilitate the sharing of best practice between LAs in terms of successful approaches to submitting data for those with similar IT systems, or those considering alternative systems, so that lessons can be shared more widely
- review its assurances around the quality of data collected from LAs, including variability in quality across different variables, informed by engagement with LAs about their data quality management approaches, and the practice areas within the QAAD toolkit
- publish a plan setting out its proposals and timelines for addressing 1), 2) and 3).
Actions taken by DLUHC to meet the requirement
DLUHC statisticians have worked closely with DLUHC’s Homelessness Advisory Service Team to develop a plan for engaging with LAs that have been unable to consistently submit H-CLIC data, and explored tailored approaches for specific barriers faced by individual LAs. DLUHC has engaged with LAs to put them in touch with others who use a similar IT system. This has led to improvements in communication, guidance, as well as the data specification and submission process.
From April-June 2023, new funding conditions for the Homelessness Prevention Grant (HPG) were introduced which meant that LAs not providing data of high quality and publishable will have a review of circumstances at the end of the year, and may have future years’ HPG funding reduced by up to 10%. This incentive has also been combined with more focused engagement with LAs that have previously experienced greater difficulties providing publishable H-CLIC data.
DLUHC has also reviewed its quality assurance as a whole from the data collection to publication, looking at specific priority variables to have a more formal sense of variability in quality. The H-CLIC performance dashboard provides key metrics on data quality and performance in relation to homelessness duties for individual local authorities. DLUHC has engaged with the GSS Data Quality Hub and references the QAAD toolkit to ensure that it has followed best practice.
DLUHC has updated a published workplan with each quarterly publication to provide progress updates on addressing the requirements to users.
OSR’s evaluation of evidence
The proportion of LAs providing case-level temporary accommodation data through H-CLIC has risen from 23% in Q2 2018 to 94% in the latest collection. A case-level weighting method is used to estimate and account for missing LA data, drawing on returns from previous quarterly submissions. The weighting process estimates the amount of missing data by observing quarter-on-quarter change in related groups of similar LAs.
More LAs have missing data for Temporary Accommodation (TA) than for other variables due to legacy TA cases often being held on older systems which are incompatible with the new collection approach. Six LAs were unable to return TA data in 2022/23 and their data has been imputed in national and regional totals. DLUHC is working with these LAs to overcome technical difficulties in moving TA reporting to an accurate, case-level data return. This is being achieved through working closely with authorities alongside DLUHC’s Homelessness Advice and Support Team to understand the barriers to submitting accurate data, in line with new funding conditions of the HPG.
The H-CLIC performance dashboard which provides key metrics on data quality for individual LAs is used to identify and facilitate local conversations to improve practice and data quality in future collections.
To enhance user understanding on the quality of the statistics, DLUHC should expand the published information on data quality, in line with broader quality measures covered in the DLUHC quality strategy, and its learning from applying the QAAD toolkit, to include:
- further information, in line with what was previously published on comparability between P1E and H-CLIC data, and the impact that removing P1E has had on data quality and the levels of imputation used, including on temporary accommodation figures
- clarity around any limitations or quality issues identified through further engagement with LAs and from applying the QAAD toolkit, and how these have, or will be, addressed or mitigated
- clear communication around the extent of uncertainty for different H-CLIC variables, and how these relate to the red, amber, and green dashboard quality indicators, to help to ensure appropriate interpretation and use.
Actions taken by DLUHC to meet the requirement
DLUHC completed a self-assessment against the QAAD framework and produced an action plan for quality issues that needed to be addressed or mitigated.
DLUHC has engaged with the GSS Data Quality Hub for guidance on effective communication of limitations and quality issues.
DLUHC has included information on the comparability of H-CLIC and the old P1E form and impacts of switching the data collections in the technical note.
DLUHC has ensured more-coherent communication on quality in the technical note, linking to the quality measures in the performance dashboard.
DLUHC has provided a more expansive narrative on potential uncertainties in data quality for its forthcoming annual technical note. It has also added details on the rationale for cut off points for its performance dashboard and updated the definitions and calculations section in the dashboard and technical note.
OSR’s evaluation of evidence
We judge that this requirement will be met when DLUHC publishes its annual technical note.
The technical note provides an overview of the processes used to ensure the quality of the statistics, before, during and after data collection.
For each section of the release, the technical note sets out the proportion of missing data which has been subject to imputation and weighting process. This supports user needs to understand how much data is missing and how this may affect overall figures, including on TA figures. It also sets out a range of reasons why data may be missing.
Footnotes at the end of tables give details on which LAs have missing data, either because data was not submitted, or potential quality issues were identified.
Notes and caveats are highlighted as footnotes in the data tables, and in the technical note to ensure that users understand how quality has been assured and factors which may affect their use and interpretation.
To support users in the appropriate interpretation of the statistics, the team should provide:
- clarity on when revisions are counted as scheduled or non-scheduled in line with what happens in practice
- clear information on how far back revisions are made, and the nature and extent of revisions, for example by providing a revisions table.
Actions taken by DLUHC to meet the requirement
DLUHC updated the technical note from the October to December 2021 publication to be clearer on its approach to scheduled revisions and unscheduled corrections. It now sets out what revisions are made, how far back they go, and the average impact on the numbers. DLUHC has illustrated if there is a greater range or more variability in the revisions made for some variables over others.
OSR’s evaluation of evidence
The new table in the technical note now highlights which data are revised at which points during the year, and how far back revisions are made.
The text helpfully illustrates that some variables are subject to a greater degree of revision that others and quantifies the average extent of revision made.