Dear Roger
Scottish School Leaver Attainment and Initial Destinations: statistics
We have recently carried out a compliance check against the Code of Practice for Statistics on Scottish Government’s School Leaver Attainment and Initial Destinations: statistics. Following a recent consultation about how to better meet user needs, a revised publication schedule has been implemented. This has resulted in two publications: Summary Statistics for Attainment and Initial Leaver Destinations published in February 2019 and Summary Statistics for Follow-up Leaver Destinations published in June 2019. I am pleased to confirm that both these statistics can continue to be designated as National Statistics.
Throughout this compliance check we identified both areas of good practice, and areas for potential improvement. We have discussed these with your team and summarised below.
We commend the statistics team’s collaboration with Skills Development Scotland (SDS) in investigating methods to develop a new measure of long-term outcomes for school leavers using methodology based on that used for its Annual Participation Measure. A weakness of the leavers’ destinations data is that the statistics are based on a snapshot at a particular point in time. If plans progress, the proposed new long-term outcomes measure should allow users to see the proportion of school leavers going into positive destinations, based on longitudinal data. This would create a better understanding of the destinations of young people based on their activity over longer time periods which users would find beneficial.
We consider that the current landscape around school leaver statistics in Scotland is somewhat incoherent as both the Scottish Qualifications Agency (SQA) and SDS produce complementary statistics albeit at different times. Although these organisations are not currently official statistics producers, they are phasing in compliance as they will be included in the forthcoming Official Statistics (Scotland) Amendment Order. The SQA produces its Attainment Statistics in August soon after school students are told about their achievements. Similarly, SDS produces the Annual Participation Measure Statistics also in August. The School Leaver Attainment and Initial Destinations statistics are then published in February with the follow-up in June. It would be helpful if greater clarity was available about the most appropriate uses of the different statistics. This might be achieved through guidance on the webpage for each of these outputs. Further collaboration between Scottish Government, SQA and SDS, and with users of the different outputs, may give useful insight into how this suite of statistics could better meet user needs.
We are pleased to see the positive and innovative measures being taken to enhance the statistics. This serves to both bolster the value of the statistics including offering more choice to the user about how to access them. An example of this includes the new School Leaver Attainment and Destinations Dashboard where users can see both the attainment and destination trends by local authority, deprivation levels or by pupil characteristics.
However, we discussed with the statistics team that there are some potential enhancements to the publications that could support greater accessibility and understanding of the statistics. These include better referencing the methodology throughout the bulletin and aligning the table numbers in the follow-up June statistics with those used in the February statistics. We also identified that there could be greater transparency about what the attainment levels mean in the statistical bulletin. Users may be unfamiliar with the Scottish Qualifications Framework outlined in Chapter 4. It would be helpful if there was further explanation of what lower levels of attainment could mean for the pupils’ future. Similarly, further information on what constitutes a ‘positive destination’ could be outlined. We welcome that you are further clarifying ‘personal skills development’ within the destinations’ categories which will help to better inform the users.
The Government Statistical Service’s Good Practice Team may be able to provide further guidance on enhancements to the publications prior to producing the next release in February 2020.
As you know, the quality of these statistics is highly dependent on the quality of administrative data sourced from the SQA for the attainment data and from SDS for the destinations data. We are encouraged to hear that the statistics team has documented instructions for the internal quality assurance processes for both. To provide greater assurance related to the quality of these statistics we suggest that the statistics team applies the Authority’s Quality Assurance of Administrative Data (QAAD) guidance to demonstrate more clearly that your QA approach is appropriate to the data risks.
We are grateful for how effectively the statistics team engaged with us during this review. We will keep in touch with the statistics team as part of our ongoing monitoring of the developments in your statistics relating to the learning of young people in Scotland.
I am copying this letter to Andrew White and Anna MacKinnon at Scottish Government, to James Morgan at SQA and Alison More at SDS.
Yours sincerely
Mark Pont
Assessment Programme Lead