Children, Education and Skills

Leaving school during a global pandemic

What are the consequences for young people leaving school as a result of the pandemic?

How can more detailed statistics about school leavers help us understand and effect real change for our young people?

Last year we published our UK wide report – Exploring the public value of statistics about post-16 education and skills. This was an in-depth look at the post-16 education sector and covered statistics on workforce skills, universities and higher education, further education and colleges and apprenticeships. Doing a multi country, multi sector report of this nature was for me, a challenge in many ways, not only due to the fact that we were engaging with multiple producers and users all with diverse viewpoints, but also because there was a myriad of different statistics as well as data gaps to consider. We also wanted to ensure that areas of good practice and shared learning opportunities were highlighted across the four nations.

Our research highlighted the following areas as being of greatest importance to the sector:

the coherence of the available statistics, how they inform the bigger picture
the accessibility of the statistics to users
how well the current statistics fully meet the information needs of users and understanding where there may be information gaps

They say that timing is everything and of course by July last year we were in the midst of the global pandemic with the post-16 sector like many others facing immense challenges. We still felt however, that it was important to publish and share what we had found.

One year on…

Across the country this month, young people are leaving compulsory education and making decisions on their future career prospects. As both a parent and regulator in the children, education and skills domain, I think of the tough decisions young people are making, with the stakes seemingly higher than ever in a world of increased uncertainty during the pandemic. We need to ensure that the data available to help them is timely, relevant and accessible to those that need it.

We have been encouraged that, even with the challenges faced by the post-16 education sector, we have seen many of the recommendations we made in our report progress, but there is more to do.

Statistics to make a real difference

Leading the user engagement of the Scotland statistics, I remember how passionately some researchers spoke about the need for good quality statistics to track individuals from their early years in the education system, through to the choices they make in their post-16 years and beyond. They felt this could make a difference, building an evidence base to support targeted interventions at the right time.

It was also an eye opener for me to find out about the complexities around linking this data using a common unique identifier between schools, colleges and universities as well as other post-16 options. Again, the real value comes when the linked datasets tell the stories and thus allow progress and change within the education system. This has benefits beyond those who have been linked in the data as it enables researchers understand issues and develop appropriate solutions for the future.

As we continue our engagement with the relevant statistics producers, we will encourage them to address issues around data granularity, quality and linkage so those working within this sector can understand and effect real change for our young people. As the effects of COVID-19 may affect their outcomes for decades to come – they deserve it now, more than ever.

If you wish to discuss user views for post-16 education and skills statistics please get in touch with us.

The people behind the Office for Statistics Regulation in 2020

This year I’ve written 9 blogs, ranging from an exploration of data gaps to a celebration of the armchair epidemiologists. I was thinking of making it to double figures, setting out my reflections across a tumultuous year. And describing my pride in what the Office for Statistics Regulation team has delivered. But, as so often in OSR, the team is way ahead of me. They’ve pulled together their own year-end reflections into a short summary. Their pride in their work, and their commitment to the public good of statistics, really say far more than anything I could write; it’s just a much better summary.

So here it is (merry Christmas)

Ed Humpherson

Donna Livesey – Business Manager

2020 has been a hard year for everyone, with many very personally affected by the pandemic. Moving from a bustling office environment to living and working home alone had the potential to make for a pretty lonely existence, but I’ve been very lucky.

This year has only confirmed what a special group of people I work with in OSR. Everyone has been working very hard but we have taken time to support each other, to continue to work collaboratively to find creative solutions to new challenges, and to generously share our lives, be it our families or our menagerie of pets, all be it virtually.

I am so proud to work with a team that have such a passion for ensuring the public get the statistics and data they need to make sense of the world around them, while showing empathy for the pressures producers of statistics are under at this time.

We all know that the public will continue to look to us beyond the pandemic, as the independent regulator, to ensure statistics honestly and transparently answer the important questions about the longer term impacts on all aspects of our lives, and our childrens’ lives. I know we are all ready for that challenge, as we are all ready for that day when we can all get together in person.

Caroline Jones – Statistics Regulator, Health and Social Care Lead

2020 started off under lockdown, with the nation gripped by the COVID-19 pandemic and avidly perusing the daily number of deaths, number of tests, volume of hospitalisations and number of vaccines. This level of anxiety has pushed more people into contacting OSR to ask for better statistics, and it has been a privilege to work at the vanguard of the improvement to the statistics.

To manage the workload, the Health domain met daily with Mary (Deputy Director for Regulation) and Katy, who manages our casework, so we could coordinate the volume of health related casework we were getting in. We felt it important to deal sympathetically with statistic producers, who have been under immense pressure this year, to ensure they changed their outputs to ensure they were producing the best statistics possible. It’s been rewarding to be part of that improvement and change, but we still have a lot of work to do in 2021 to continue to advocate for better social and community care statistics.

Leah Skinner – Digital Communications Officer

As a communications professional who loves words, I very often stop and wonder how I ended up working in an environment with so many numbers. But if 2020 has taught me anything, it’s that the communication of those numbers, in a way that the public can understand, is crucial to make sure that the public have trust in statistics.

This has made me reflect on my own work, and I am more determined than ever to make our work, complex as it can be, as accessible and as understandable to our audiences as possible. For me, the highlight of this year has been watching our audience grow as we have improved our Twitter outputs and launched our own website. I really enjoy seeing people who have never reached out to us before contacting us to work with us, whether it be to do with Voluntary Application of the Code, or to highlight casework.

As truly awful as 2020 has been, it is clear now that the public are far more aware of how statistics affect our everyday lives, and this empowers us to ask more questions about the quality and trustworthiness of data and hold organisations to account when the data isn’t good enough.

Mark Pont – Assessment Programme Lead

For me, through the challenges of 2020, it’s been great to see the OSR team show itself as a supportive regulator. Of course we’ve made some strong interventions where these have been needed to champion the public good of statistics and data. But much of our influence comes through the support and challenge we offer to statistics producers.

We published some of our findings in the form of rapid regulatory review letters. However, much of our support and challenge was behind the scenes, which is just as valuable.

During the early days of the pandemic we had uncountable chats with teams across the statistical system as they wrestled with how to generate the important insights that many of us needed. All this in the absence of the usual long-standing data sources and while protecting often restricted and vulnerable workforces who were adapting to new ways of working. It was fantastic to walk through those exciting developments with statistical producers, seeing first-hand the rapid exploitation of new data sources.

2021 will still be challenging for many of us. Hopefully many aspects of life will start to return to something closer to what we were used to. But I think the statistical system, including us as regulators, will start 2021 from a much higher base than 2020 and I look forward to seeing many more exciting developments in the world of official statistics.

Emily Carless – Statistics Regulator, Children, Education and Skills Lead

2020 has been a challenging year for producers and users of children, education and skills statistics which has had a life changing impact on the people who the statistics are about.  We started the year polishing the report of our review of post-16 education and skills statistics and are finishing it polishing the report of our review of the approach to developing the statistical models designed for awarding grades.  These statistical models had a profound impact on young people’s lives and on public confidence in statistics and statistical models.

As in other domains, statistics have needed to be developed quickly to meet the need for data on the impact of the pandemic on children and the education system, and to inform decisions such as those around re-opening schools. The demand for statistics in this area continues to grow to ensure that the impact of the pandemic on this generation can be fully understood.

The exams algorithm story is about more than just exams

“Dreams ruined by an algorithm”. This was the headline on the BBC Northern Ireland website on 13 August. It summarised one of the main stories of the summer in all four nations of the UK. The headline reflected the human and emotional impact of the 2020 exams results: with some students getting lower grades than they were expecting or felt that they deserved.

But the story is about more than using statistical models to award grades. The resulting negative backlash, specifically on the role that algorithms played, threatens to undermine public confidence in statistical models more broadly.

We are concerned about this. That’s why we’ve launched a review of the way the statistical models were built and overseen. It is not our intention to apportion blame or to launch criticisms with the benefit of hindsight. Instead, we want to focus on the future: how to implement models in a way that is consistent with public confidence.

Government statisticians have performed well during the pandemic. They have responded quickly to identify, develop and produce new data and statistics – statistics which are currently an integral part of our lives. The ONS, and its counterparts in Scotland and Northern Ireland, have provided clear analysis of the human tragedy of mortality. And, as OSR’s rapid reviews have shown, statisticians have produced new data on sectors as diverse as the economy, transport, school attendance, and people’s engagement with green space.

Over the last six months, these and other statistics have served the public’s appetite for clear, trustworthy information on the unfolding covid-19 pandemic. There is clear evidence that they are meeting a real need and serving the public good.

Our overarching role at OSR is to ensure that statistics serve the public good, and we do this by setting a Code of Practice that Government must follow. The Code has public confidence in statistics at its heart. However, confidence in statistics risks being undermined by the poor public reception that the exam algorithms received in all four countries of the UK.

From the perspective of statistics serving the public good, the exams story looks worrying. Not only has there been widespread criticism of the statistical models because of the perceived unfairness in the results they created; but there is a risk that future deployment of statistical techniques may be held back by the chilling effect of the poor publicity. That would be a real setback: it would limit statistical innovation and mean that the public sector cannot use new approaches to providing services to the public.

The exams issue has shone a light on how statistical models are used to make decisions. We want to learn from this experience. We want to explain how statistical models can be deployed and overseen. We want to set out some basic expectations for how these models can serve the public good. And we want to show how the principles of the Code of Practice – trustworthiness, quality and value – are highly relevant to situations like this, where complex models are used to support decisions that have impacts on individual human beings.

In short, the review aims to identify lessons for public bodies considering the use of statistical models to support decisions.

Our overall aim is simply stated. The use of statistical models to support decisions is likely to increase in coming years. We want to show how government can learn from this experience – and make sure that these statistical models serve the public good.

Think of the children

The Covid-19 pandemic is having a profound impact on all parts of society. While statistically those medically hardest hit by the disease are the older generations, children and young people are having to come to terms with significant, immediate and possibly long term changes to their lives.

More than ever it is important that statistics about children and young people reflect the lived experiences of children. Statisticians play a key role in ensuring that the data collected and published about children and young people accurately reflects their needs and helps to inform policy and services that work to support them.

Our review

Prior to the pandemic we started reviewing the availability of statistics about children and young people with a view to better understanding their value in society and to determine whether:

the current statistics are accessible, timely and help society to understand the experiences of children and young people in all aspects of their lives
improvements are needed to the ways in which decisions on what to collect and analyse are reached

the wider statistical system is responsive to the needs of users of statistics.

We want to see a step change in how the needs of children and young people are met by official statistics, where statistics producers consistently consider children and young people’s needs and voice during the design, collection, analysis and dissemination of statistics. The current pandemic and its aftermath make this all the more important.

Our initial research has looked at the strengths and weakness of the current statistics on children and young people. In doing so, we have identified three key lenses which, if applied through a structured framework, may support statistic producers to better meet users needs. This approach reflects the core principles set out in the UN Convention on the Rights of the Child.

Our proposed framework

We propose that producers of statistics consider children and young people through three lenses.

Visibility – Statistics are available on children and young people
Vulnerability – The experiences of vulnerable children can be analysed separately
Voice – Statistics reflect the views of children and young people and can be used by them

For each lens of the framework we propose some key questions for statistics producers to consider.

Visibility – Statistics are available on children and young people

Are children and young people visible in the statistics?
Is data collected about them and then made available to inform decisions in the best interests of the child?
Are decisions around what data to collect on and from children and young people transparent?

Vulnerability – The experiences of vulnerable children can be analysed separately

Are the most vulnerable children visible?
Is their experience identifiable to ensure that they are not being discriminated against?
Do the statistics and data help identify which groups of children and young people are the most vulnerable to having poorer outcomes?

Voice – Statistics reflect the views of children and young people and can be used by them

Are the views of children and young people represented in the statistics?
Are survey questions asked to children and young people themselves?
Do the statistics give them a voice on what is important to them by being understandable to them?

Your views are important to us

The next stage of our review is to test this framework approach with a wider set of users and statistics producers to see if this supports these aspirations. We hope also that sharing our initial thinking now may assist producers in their immediate decisions about what statistics and data they should be collecting and making available during and after the Covid-19 pandemic.

Are you a statistician trying to identify what data to collect and publish? Would this framework help you in making those decisions? Is there anything else that you feel could be considered? What would be the barriers to ensuring that children and young people are visible in the statistics, that the vulnerable can be separately analysed and that the statistics give children and young people a voice?

Are you a decision or policy maker using statistics to understand the lives of children and young people and the impact of decisions and policies on them? Does this framework cover the key elements that you feel are important? Is there anything else that you think statisticians should consider?

Are you a researcher using data and statistics to research children and young people’s lives and outcomes and the interventions that impact on them? Does this framework cover the key elements that you feel are important? Is there anything else that you think statisticians should consider? Are your needs adequately reflected by the framework?

Are you a child or young person or do you represent them? Are visibility, vulnerability and voice the key elements of statistics that are important to you? What are you most interested in when looking for statistics? What makes it difficult for you to find and use statistics?

Please get in touch to share your thoughts with us at regulation@statistics.gov.uk