Dear Ed,
I am writing to you following the recent letter from Sir David Norgrove, Chair, UK Statistics Authority to Lord Bourne about the use of statistics on the impact of the Rough Sleeping Initiative[1], and also Sir David’s recent reply to John Healey MP, Shadow Secretary of State for Housing who expressed concern about the quality of the rough sleeping statistics[2]. In these letters, Sir David raised a number of matters including improvements to the rough sleeping statistics, consideration of using a CHAIN (Combined Homelessness and Information Network) approach beyond London, the impact of local authorities switching from an estimate to a count (or vice versa) and conclusions about the impact of the Rough Sleeping Initiative drawn from the Rough Sleeping statistics. We are also aware of your letter to Clive Betts MP, chair of the Housing, Communities and Local Government Select Committee this week. This letter reiterates the UKSA’s concerns in the light of the Committee’s recent evidence session.
By way of response to the recent letters from Sir David, I would like to set out the improvements we have made to the annual Rough Sleeping statistics following the UKSA Assessment in 2015 and also our plans for improving the evidence base on rough sleeping, which is broader than the annual statistics.
Accurately counting or estimating the number of people sleeping rough within a local authority area is inherently difficult given the fluctuating nature of rough sleeping, with individuals regularly moving into and out of rough sleeping, and its hidden nature. The latest statistical release[3] makes clear there are a range of factors that have an impact on the number of people seen or estimated to be sleeping rough on any given night. The methodology we use has been in place since 2010 when it was developed after consultation with local authorities and the voluntary sector with the objective of strengthening the accuracy of the figures. The annual snapshot has never claimed to be a total figure of all those sleeping rough in the country and is a pragmatic approach that provides a representation of those sleeping rough, alongside a reliable and consistent measure of change. The methodology used in England aligns with the approach which is now standard in many parts of the world including Canada, the United States and a number of other European countries including France, Ireland, Italy and Spain.
All counts and estimates are independently verified by Homeless Link who actively challenge areas in situations where the veracity of the count or estimate is questionable. They oversee the whole process and issue clear guidance on how to conduct a count and where it may be appropriate to move from a count to an estimate, or vice versa. Hundreds of local partners and voluntary sector organisations actively engage in the count and producing the estimate which also gives us confidence in the reliability of the statistics.
Since the UKSA Assessment of the Homelessness and Rough Sleeping Statistics in 2015, we have addressed the range of requirements set out in the report, as detailed in Annex A and summarised in my letter[4] to you of 26 June 2018. We have expanded the statistical release to include demographic information about young people, gender and nationality, as well as providing a time series table. The release has much clearer information about the methodological issues which affect the data and the role played by the lead statistician in their production. We held a user event in November 2016 supported by the UKSA. Further information about all the latest developments in both homelessness and rough sleeping can be found on the homelessness statistics user forum page[5]. Whilst we have a programme of work underway to improve the broader evidence base for homelessness and rough sleeping, we are not planning any major changes to the single night snapshot methodology but continue to encourage users to provide feedback on how these statistics are used and how well they meet their needs. We remain confident in the comparability of the time series to provide a way of assessing the extent of rough sleeping across local areas on a single night and measuring change. It has always been the case that each area choses the most appropriate approach to assess the extent of rough sleeping in their local area each year.
We routinely meet with Homeless Link to review the count and estimate process so that we can be confident that these statistics are a consistent measurement of change. We will keep users updated on any new developments via the user forum and in future statistical publications.
As recommended, we have included CHAIN (Combined Homelessness and Information Network) statistics within the Rough Sleeping Statistics Release, but, at this stage, have concluded that it would not be practical to extend a CHAIN multi-agency database approach beyond London and those other cities which have started to adopt similar approaches. CHAIN is a multi-agency database which records information about people seen rough sleeping by outreach teams in London across the year whereas the single night snapshot provides a way of estimating the number of people sleeping rough across all local authorities in England on a single night and is a way of assessing change over time. Although, CHAIN offers more information on the flows and characteristics of people rough sleeping, data quality is influenced by the number of outreach workers active on any one night. Thus a local authority which does not have a commissioned outreach team may appear to have fewer rough sleepers than an adjacent borough which does have an active outreach service. MHCLG is actively encouraging local authorities with significant numbers of rough sleepers to improve their year-round data on rough sleeping and is working collaboratively with local authorities to develop improved homelessness data and outcomes that can measure progress in reducing rough sleeping and homelessness. We have also funded a significant number of additional outreach posts through the RSI which increases an area’s capacity and understanding of rough sleeping and rough sleepers in their area.
Our broader programme of work will improve the evidence available for developing, implementing and monitoring policies in this area. Alongside our recent overhaul of homelessness statistics, introducing case level collections with far greater details, we are also establishing local data pilots which will seek to improve the data collected by a wide range of services which people who sleep rough access. These data pilots will not provide or contribute to a national measure of rough sleeping, but they will help us to develop and test a multi-agency outcomes framework, looking across a range of services, including key health services.
The data pilots, our programme of evaluations and new research studies as detailed in Annex B, are all part of an ongoing effort to ensure that our interventions are evidenced based and that we build up an evidence base on ‘what works’ to reduce rough sleeping. This was set out in our Rough Sleeping Strategy and we made a commitment that “…over the next nine years, we will plug evidence gaps and pilot, test and evaluate new approaches which will inform our plans for wider roll-out”. It is important to note that this relates to the broader range of evidence in this area, rather than the Rough Sleeping Statistics specifically – while Sir David’s reply to John Healey MP, conflates the two.
The latest published annual rough sleeping statistics included specific reference to the areas supported by the government’s Rough Sleeping Initiative (RSI). We will be publishing an evaluation of the initiative in the summer which will help us to better understand the impact of the initiative.
The evaluation is in two parts: an impact evaluation and a process evaluation. The impact evaluation, led by MHCLG analysts with independent peer review, will look at the impact of the range of activities in RSI areas on the overall numbers of people sleeping rough. This will include consideration of the impact of the change in choice of approach for assessing the extent of rough sleeping, as well as other factors such as the weather, levels of funding and, the types of activity within the areas who are part of the initiative. The process evaluation, carried out by Ipsos MORI will share learning about how the Rough Sleeping Initiative has worked, good practice and key challenges, including findings from a survey of initiative areas and case studies. We are intending to publish these findings in the summer and would welcome a meeting with the UKSA to discuss the findings and provide further advice, once this work is complete.
The UKSA have highlighted concerns about an ‘apparent methodology shift’ impacting the annual rough sleeping statistics. As stated above, the impact evaluation will provide information on this. However, we provide additional information in the annual statistical release on areas that have changed their approach to assessing the extent of rough sleeping in their local area this year compared to previous years. This is an established part of the methodology – local authorities, not central government, decide in conjunction with their local partners and based on advice from Homeless Link (who verify the rough sleeping statistics collection) to use the approach that they believe will return the most accurate figure to assess the extent of people sleeping rough in their local area. This decision is not linked to any requirement under RSI funding. There are a number of reasons for areas to change from an estimate to a count year to year as set out in guidance provided by Homeless Link. This includes:
- A significant change in the number, population or location of people sleeping rough where sites are visible
- High numbers of people sleeping rough in the area with no ongoing data collection e.g. no/limited outreach
- An increase or fluctuations in numbers of people sleeping rough where sleep sites are accessible/visible
- Difficulties forming an estimate on the basis of the information available
- Significant disagreement about the numbers between agencies
Overall, 74 areas used a count this year compared to 52 last year, and 252 used an estimate compared to 274 in 2017. In RSI funded areas, 46 areas used a count this year compared to 28 last year, and 37 used an estimate compared to 55 in 2017. Over three quarters of RSI areas used the same approach as last year. Given the number of new posts which have been funded through the RSI, there is also greater capacity to conduct street counts, which may explain some of the increase in areas doing street counts this year. As outlined above, in our view this does not affect the comparability of the time series as the methodology has always been that local areas decide the approach to use to assess the extent of rough sleeping in their local area to provide the most accurate figure. However, the RSI Impact evaluation will consider this issue in more detail.
We are in regular contact with your officials about both the homelessness and the rough sleeping statistics and we will discuss further once the RSI evaluations have been published. We will also be in contact to discuss reviewing the assessment of the homelessness and rough sleeping statistics following the overhaul of the homelessness statistics. As outlined in my letter of 26 June 2018 we expect it to take time for the new collection arrangements to become established and data quality to be assured and as yet we are just a few months on from our initial publication of experimental statistics in December 2018.
Yours sincerely,
Sandra Tudor
Head of Profession for Statistics, MHCLG
[1] UKSA letter to Lord Bourne, 18 April 2019
[2] UKSA letter to John Healey, 25 March 2019
[3] Rough sleeping in England: autumn 2018
[4] Letter from Sandra Tudor, Head of Profession for Statistics, MHCLG to Ed Humpherson, Director General for Regulation, Office for Statistics Regulation, 26 June 2018
[5] MHCLG homelessness statistics user forum page
Related Links: