User engagement

26. The KS4 performance statistics are used for a wide range of purposes and DfE has a comprehensive understanding of the different users of the statistics as outlined in the ‘About the output’ section on the KS4 performance methodology page.  
27. DfE aims to meet the needs of different users through providing different statistical products. The first of these is the statistical bulletin which provides a commentary as well as interactive elements. The second product is the downloadable data files and the third product is the table building tool which is available for all releases on the EES platform. The fourth product is the KS4 school-level data provided on the Compare School Performance website.  
28. DfE is in regular contact with its key users and receives feedback from them regarding the statistics. DfE also provides contact details for users to submit their feedback. The views from users are addressed where possible, for example, DfE has recently introduced more-detailed breakdowns for ethnicity at local authority (LA) level.  
29. DfE conducts regular user research on the Compare School Performance website. This enables DfE to systematically test aspects of the website and the statistics presented. This research has been valuable in understanding how users interpret the Attainment 8 and Progress 8 measures and identifying improvements that can be made to their presentation.   
30. Although the key users that we interviewed were generally satisfied with their engagement with DfE, this was not fully reflected in the online user questionnaire (see Annex A). While some users reported that DfE had been quick to respond to their queries and that the information had been useful, other respondents raised several issues with their interactions with DfE. These included: long delays between responses or no response at all; and queries being passed to multiple different teams or people. Most users reported that they were able to get requested data from DfE relatively easily, however a small number reported that they felt that they had to raise formal requests under the FOI Act in order to get the data rather than being able to engage with DfE more informally. DfE should consider how it can better engage with these types of users to ensure that their needs are met.  
31. DfE is currently in the process of engaging with key users and stakeholders to discuss their proposed changes for the 2022 publication. Examples of planned changes as a result include the structure of the institution pages on the Compare School Performance website and clear guidance around how to interpret the data. These changes will help communicate the impact that the pandemic has had on the KS4 data and methods and the appropriate comparisons that can be drawn. 
32. DfE is planning to refresh its user engagement strategy in light of the wider GSS work on user engagement in statistics, and reflecting its plans to use a variety of feedback mechanisms to understand how users interact with its statistics, and what information they need. We encourage the KS4 statistical team to use the updated DfE engagement strategy as a basis to update the KS4 statistics user engagement strategy. We would also encourage the KS4 statistical team to consider new and innovative ways in which it could engage with users or promote the statistics to increase awareness of the statistics and communicate effectively with the widest possible audience.  
33. The Code of Practice for Statistics sets out that statistics producers should collaborate with other producers within the UK. DfE works closely with The Office of Qualifications and Examinations Regulation (Ofqual) and The Joint Council for Qualifications (JCQ), who publish related statistics, to ensure that the key messages are coherent across the different publications. Although the comparisons that can be made across the different countries in the UK are limited due to the devolution of education, there are potential benefits to be gained from sharing best practice and knowledge. We welcome the KS4 statistics team’s plan to engage more regularly with the devolved administrations. 
34. DfE provides detailed advance notice about changes to methods, often consulting with the public and expert users on proposals. For example, Progress 8 and Attainment 8 were introduced following a consultation on secondary school accountability in 2013. Where possible, DfE has provided a consistent time series, for example following the introduction of the disadvantaged pupil gap index.  
35. DfE has a detailed work programme for KS4 statistics. However, information on this is not always shared beyond its key users. The Code of Practice for Statistics states that statistics producers should be open about progress towards meeting priorities and objectives and that users and other stakeholders should be involved to help priorities statistical plans. We would encourage DfE to make more information available regarding the development plans for the KS4 statistics so that all users are kept informed. It would also allow a greater number of users to be involved in the ongoing development of the statistics which will help to maximise public value from this rich and important data source.