Dear Elaine  

We recently completed a compliance check of Social Security Scotland’s Adult Disability Payment and Child Disability Payment statistics, which are both published as official statistics in development. To support their further development, we have carried out a high-level investigation into whether the statistics are being improved in line with the Code of Practice for Statistics. In this letter, we outline our findings and improvements to consider as your team continues its work on these statistics. 

These statistics show information on applications and payments for child and adult disability payments. The main messages from the statistics are clearly explained, with appropriate commentary. Suitable charts and tables are used to illustrate the statistics. We judge that the statistics are presented impartially and have been released in an orderly manner, in line with the Scottish Government’s statistics publication timetable. To inform users of any changes to past data, the outputs include a description of the revisions made. 

Suitable data sources are used to compile the statistics, and the team is planning to further improve how it explains the limitations of the data. The statistics are sourced from the person-level dataset held in Social Security Scotland’s own internal case management system. The data are extracted and securely transferred to the statistics team. There is some information published on the limitations arising from the methods used to compile the statistics, including bias and uncertainty. The statistics team told OSR that it is planning to publish more information about uncertainty in the data, potential biases and the resulting impact on the statistics, which is good practice. 

Currently, part of the data processing is manual, and the statistics team told OSR that it is in the middle of transferring to an automated statistics production pipeline. The plans to automate the data processing process are in line with good practice and should improve the quality of the statistics. Once the transition to automation is complete, Social Security Scotland should publish detailed information about the new quality assurance steps and stages. It should also consider publishing the software code used to produce the statistics. 

Engaging with users to understand their needs will enable Social Security Scotland to consider and prioritise developments in line with user need. The statistics team indicated that it is engaging with a range of users in central government, local government and the NHS. We also welcome the fact that users approach the statistics team in an ad hoc way and that the team asks for feedback to gather insight on how to develop the statistics. 

To further enhance the Trustworthiness, Quality and Value of these statistics, we have identified the following improvements for your team to consider: 

  • To provide users with a wider view of the disability benefits landscape, Social Security Scotland should consider providing information about other UK disability payments statistics in the release and commenting on how the statistics should or should not be compared. 
  • Publishing more detail about the quality assurance processes would provide greater assurance to users. For example, Social Security Scotland should include a process map, identify potential sources of bias and error, and describe the actions taken to minimise risks to data quality. 
  • It would help potential new users if Social Security Scotland published more information about its statistics user engagement plans, so that users know that it are willing to hear their views. 
  • To support the development of these statistics, and in line with our guidance about official statistics in development, Social Security Scotland should consider publishing more detail about its plans to develop these statistics, including timelines and how users will be involved the process. 

I would like to thank your team for its positive engagement with us during this review. We will continue to engage with you as you develop these statistics. Once you consider that these statistics, or other statistics in your portfolio, are ready for a full assessment against the Code, please do not hesitate to get in touch. 

I am copying this letter to Alastair McAlpine, Chief Statistician of the Scottish Government. 

 

Yours sincerely 

Siobhan Tuohy-Smith 

Assessment Programme Lead

Related Links:

Letter: Ed Humpherson to David Wallace: Transparency and user engagement in the production of official statistics