Findings

Maximising insight through user engagement

1.8 Throughout the statistical bulletin the commentary is presented impartially and is supported by appropriate diagrams and charts to aid user understanding. Further insightful information on the different measures used within the PIP statistics, changes made to the benefits processes during the Covid-19 pandemic and the devolution of benefits to Scotland are provided within the background quality and methodology report along with a detailed glossary of all the terms used.

1.9 DWP engages with stakeholders taking a proportionate approach through a combination of its internal quarterly stakeholder user group and quarterly newsletter, using the most appropriate means depending on user need and based on the level of insight the team has to share with internal users.

1.10 DWP also uses other means to engage with users such as through X (formerly Twitter), the online user discussion forum statsusernet as well as asking users for feedback through the PIP statistics bulletin and on the PIP statistics landing page. Since we started the assessment process the team has reached out to its counterparts in Northern Ireland and as part of sharing best practice, DWP is considering adopting the Northern Ireland Statistics and Research Agency’s (NISRA’s) approach to capturing user feedback on its PIP statistics through a user questionnaire. This gives more opportunity for users to engage with the PIP statistics team, especially those outside DWP.

1.11 An innovative DWP-wide initiative currently in development is the web-scraping of X feeds to help understand how DWP’s statistics are being used. This is a great example of seeking to engage further with users. While this project is still subject to having access to the relevant IT tools, should it be successful it could have wide-reaching positive implications across official statistics producers and not just DWP.

1.12 Users we spoke to as part of our review recognise the value that these statistics offer such as providing helpful insights into the characteristics of PIP claimants and being able to use the statistics to monitor the impact of the welfare system. They also identified where they would like to see more data in order to provide further insight into people’s experiences of PIP to better understand the impact of the welfare system on disabled claimants in order to provide more-targeted help to those in need.

1.13 A common theme from speaking to users was that they would like to understand more about the processes and outcomes around Mandatory Reconsiderations (MR) – where claimants who wish to dispute a decision on their PIP claim at any stage can ask DWP to reconsider the decision ­– particularly from charities who support claimants going through the PIP process. Topics of interest include whether the outcome of an MR leads to a claimant receiving a more or less favourable new award; how many claimants who were unsuccessful at MR stage go on to appeal; at which point of the client journey the claimant challenges the MR decision; and the MR success rates at each point of the client journey where the claimant challenges the MR decision.

1.14 The team has avenues available for users to contact them with feedback which are signposted such as on the PIP statistics landing webpage and within the statistical bulletin and supporting documentation. While some users may not be aware of how to engage with the PIP statistics team at DWP, the team has welcomed the additional user insights we have been able to provide as part of this assessment as they cover views that have not been previously shared with the team and we are assured by the team’s commitment to proactively reach out to these users.

1.15 Feedback from users helps inform the development of DWP’s statistics and DWP has recently updated its statistical work programme, as well as its PIP statistics release strategy, providing users with transparency around the development of its statistics.

1.16 DWP has also published further developments to the annual statistics on DLA to PIP reassessments in its statistical release in December 2023 to include second and subsequent attempts to claim PIP following a DLA claim, rather than just the first attempt, as has been published in previous years. These developments include a count of the number of attempts, the outcomes of both the first and latest attempts, and the time elapsed between first and latest attempts. This extra information should help to provide clarity around clients who had been on DLA previously and are classed as being re-assessed when they move on to PIP rather than being a new applicant.

1.17 It would be helpful to users if DWP was to explore with users what further insights could be generated from the data it has available, to help address these areas that users we spoke to identified. The team has confirmed that it plans to use the user feedback from our assessment to not only understand the disconnect in perceived levels of user engagement but to also help inform the list of future developments of the statistics, while recognising the need to balance user requirements with the limitations of the administrative systems used to record the PIP data.

1.18 The team has also committed to promoting the work programme and release strategy documents more widely to increase user awareness of them which should help manage users’ expectations about what further value can be obtained from the data and how future developments are being prioritised.

Accessibility of the data

1.18 DWP publishes some data tables in a reusable format whilst the majority of the data is made available to interrogate further and download through DWP’s Stat-Xplore tool. Most of the users we spoke to use Stat-Xplore to view the data rather than using the statistical bulletin and Excel data tables. While some users spoke positively of their experiences using Stat-Xplore some found Stat-Xplore difficult to navigate and were unable to access the data they needed, with some users unsure as to whether the data they needed were actually available. This uncertainty is also reflected in the team receiving user queries asking for help in finding the data they need. While a user guide for Stat-Xplore is available it is not intuitively accessible from the Stat-Xplore home page – access is currently through clicking on a small question mark icon in the top right-hand corner of the home page. The team has committed to amending the bulletin to better signpost the Stat-Xplore user guidance in a bid to help users find the support they need.

1.19 Given that all DWP statistical data and not just the PIP statistics are held on Stat-Xplore, addressing the accessibility of the user guide and improving the user experience of Stat-Xplore would have much wider benefits. The team told us that there are potential plans in the pipeline to develop a video tutorial for Stat-Xplore. We consider that it would add significant public value from DWP’s entire data holding if DWP were to make any necessary changes to improve the accessibility and usability of Stat-Xplore with user feedback helping to shape any plans.

1.20 The House of Commons Library publishes its own interactive PIP statistics constituency data dashboard for England and Wales and a dashboard for Scottish local authorities on people claiming Adult Disability Payment drawing on data from Stat-Xplore and other published government data sets such as the Ministry of Justice’s quarterly tribunal statistics and ONS census data. Some users identified this as a helpful way to be able to access the data. DWP has its own dashboard. However, due to de-prioritisation of some work, the data available are only for 2017. Further work on developing the dashboard has also been paused due to the de-prioritisation of projects and website hosting issues which would require a disproportionate amount of IT resource. The team has told us that since the dashboard stopped being updated, it has not received any user feedback requesting it to be reinstated.

Methods and quality

1.21 The DWP statistical team publishes clear and informative information in the background quality and methodology report on the methods used and quality assurance processes for the PIP statistics. This information is further supported by a comprehensive quality assurance of administrative data (QAAD) report that has been produced in line with our QAAD guidance. Users we spoke to were generally happy with the quality of the statistics. The team also talked us through the quality assurance checks that it carries out including tabulations and sense checks as well as working with its quality assurance group and the final outputs being signed off by the Head of Profession for Statistics at DWP.

1.22 During the course of this assessment the team has committed to further strengthen its knowledge of the end-to-end quality assurance process by considering the findings from our Quality Review of HMRC statistics. The team is thinking about producing a process map for use within DWP, which shows the steps in the production of its official statistics from data collection to publication, identifying any risk points throughout the process, and the mitigating actions that are taken to ensure the quality of the statistics at these points. This can be applied across all statistical outputs within DWP and not just the PIP statistics, providing a consistent approach.

1.23 Appropriate statistical disclosure controls are applied to the data to prevent identification of individuals. Users are advised within the statistical bulletin and background quality and methodology report that the data are rounded in line with DWP’s rounding policy, (for example for the number of claimants over one million and one these are rounded to the nearest one hundred thousand).

1.24 Information is provided within the statistical bulletin advising users that the data are subject to minor revisions (retrospection) within the claims process. DWP has told us that it times the data extraction from the data system to minimise the extent of these changes. The background quality and methods document also explains that errors can occur when entering data onto the PIP data collection system and sets out the steps taken to mitigate these errors.

1.25 In line with our expectations around communicating these types of uncertainties within the data, DWP has told us it will more clearly explain to users within the statistical bulletin the effect of the retrospection and data entry errors on the statistics and their interpretation.