A closer look at loneliness statistics

At OSR, we have always been aware of the importance of loneliness statistics on a national and local scale. In 2019, we started a systemic review of loneliness statistics to investigate the state of official statistics on loneliness in the UK. 

Initially, we found there were some significant gaps in loneliness data that were not being filled by official or national statistics. Statistics users we spoke to, such as charities focused on loneliness, told us this made it more difficult for them to carry out their core functions of preventing and tackling loneliness among the UK population.  

We heard that good quality statistics that covered local and regional geographies were needed in order for them to deliver their services, allocate funding, and in some cases, present evidence to their regional parliaments. Where official statistics were not meeting these needs, expert users were often stepping in and producing their own statistics to fill data gaps. Given this, we identified a range of specific recommendations to help improve official statistics on loneliness. 

Like many pieces of work during this period however, the pandemic made us re-think our approach. The pandemic has changed how we all think and act, including how we think about loneliness. Understanding and addressing loneliness among the population has become a focus for governments and policy makers. In response, statistics producers have had to develop their loneliness statistics to meet society’s need for information. As a result, many good developments have happened in this area and we’ve found that statistics producers have been filling in some of the key gaps we identified when we first started looking at these statistics. Our new report published today commends the efforts by statistics producers in creating statistics that better serve the public good in answer to these societal changes. 

This isn’t to say that improvements can’t still be made though. Users we spoke during the pandemic still identified some key gaps in the official statistics landscape on loneliness. We would encourage statistics producers to build on the work they had achieved in the last 18 months and to continue to take forward producing statistics that meet user needs and offer value for charities and academics in preventing and researching loneliness.  

Continuing the loneliness review was one of the first pieces of work I got when I started my placement year at the OSR last September. I’ve really enjoyed working on the report and having the opportunity to lead a review and conversations with producers. Seeing the report published on my last day at the OSR brings a wonderful and rather cyclical end to my year! The work isn’t ending with me though. As an organisation, we are looking forward to continuing working in this area and assisting producers to develop their statistics to better meet user needs. If you would like to contact us about this, please email my colleague, Emma Harrison. 

“Welp. We screwed up”: A lesson in admitting your data mistakes

A couple of months ago this tweet from the Governor of Utah caught my eye:  

The background was that the Utah Department of Health had identified an error in its published vaccinations figures – fewer people had been vaccinated against coronavirus than had been reported. In a public letter to his fellow Utahns, Governor Cox admitted the mistake.  

Here at the Office for Statistics Regulation we love this kind of openness and action. Five things stood out to us from Governor Cox’s letter, which we think all statistics producers in the UK should be thinking about when it comes to data errors. 

  1. Identify your mistake

You can’t fix what you don’t know is broken. This is why rigorous quality assurance is so important. Statisticians need to regularly look under the hood of their analysis to assure themselves and others that it is correct. In Utah, a healthy dose of scepticism about an unexpectedly high vaccination rate made the data team double-, triple- and quadruple-check their figures, until they found the mistake. So, as a statistics producer ask yourself: how do I assure myself that my analysis is correct? Do the numbers look as I expect, and why or why not? What are the risk points for errors? If the root cause of an error isn’t obvious then it can help to ask the five whys until you reach it. 

  1. Be open

One of the things that impressed us most about this example was how direct and open the communication of the error was. There was clear ownership of the mistake and a willingness to correct the figures quickly, publicly and with humility. In the UK, our Code of Practice for Statistics requires government statistics producers to handle corrections transparently. It’s also important that government officials and minsters who use statistics are open about mistakes. 

  1. Fix it

Of course, once you have identified the mistake, it needs to be fixed. As well as being transparent about corrections, the Code of Practice asks that they are made as quickly as is practical. 

  1.  Explain

In Utah, Governor Cox explained that while they had reported that 70% of adults had received at least one dose of a coronavirus vaccine, the actual figure was 67%. In a separate statement, the Utah Department of Health went into more detail about the numbers and the mistake. Statistics and data should help people understand the world around them. So, when admitting a data error, it’s important to clearly explain the impact of it – what has changed and what does this mean? 

  1. Improve

The last, but perhaps the most important, step is to learn from the mistake – so that you can avoid it, or something similar, happening again. In Utah, the data team re-examined their processes to prevent this general type of error from being repeated. Statistics producers should reflect on why a mistake was made and what can be done to avoid it in future – and then share what they have learned, and what action they are taking, with their users. 

Statistics and data produced by governments serve a vital purpose. They keep us informed, help us make decisions and allow us to hold our governments to account – so we must have confidence in them and the people who produce them. As Governor Cox said, “trust consists of two things: competence and ethical behaviours”. We agree. The Code of Practice places a strong emphasis on trustworthiness. We see that trustworthiness is demonstrated by organisations which are open, and where people who work on statistics are truthful, impartial, and appropriately skilled. We are all human, we mess up and we make mistakes – but we can retain trust by actively looking for our mistakes, being open when we find them and by learning from them.  

Data makes the difference

This is a guest blog from Jonathan Smyth, Head of Communications and Fundraising at Action Mental Health.

As an organisation, Action Mental Health has long campaigned for better mental health services in Northern Ireland. Alongside partners in the sector, a key part of our campaigning included calls to produce a fully costed and properly resourced mental health strategy that would deliver real change for people in Northern Ireland. We were the only region of the UK without such a strategy despite being the region with the most need, something borne out by the fact that we have the highest prevalence of mental health problems in the UK.

In June 2021 then, we very much welcomed the announcement by Northern Ireland’s Health Minister – Robin Swann, MLA, of Northern Ireland’s first ever Mental Health Strategy, a ten-year vision that outlines three key themes encompassing 35 actions, as well as recognising the need to invest £1.2bn over the life time of the strategy to deliver its recommendations.

In addition to the new strategy, we very much welcome OSR’s in-depth review of mental health statistics in Northern Ireland, which has confirmed that existing statistics do not meet current user needs and sets out expectations in this area to make real change.

Across the many discussions and interactions, we have had, and continue to have with other mental health campaigners and professionals, one of the key things we hear is frustration at the lack of robust data and statistics around mental health and mental health service delivery in Northern Ireland. Given the obvious pressures on the health budget due to Covid it is vital that precious investment is not wasted and unfocused due to incomplete or false data.

We hear regularly from professionals about the challenges they face in navigating Northern Ireland’s fragmented services, which are often entirely different from area to area, or maybe they are simply described differently depending on postcode.

We’re also aware of the impact this has on our clients and the confusion and frustration it causes as they have to re-tell their story to many different healthcare professionals.

With this differentiation in service delivery comes issues with data collection – there is very little standardisation of data, across what is such a relatively small area, both in terms of geography and population. How then do we plan for better services and better outcomes if we don’t know what we are comparing from area to area? As an organisation trying to develop innovative new projects it is frustrating that there is no easily accessible source of data to ensure our valuable resources are properly focused on client need.

The lack of robust statistics in such a complex area can also present challenges in the digital age when misinformation can be spread so easily. Being able to vigorously challenge potentially damaging or worrying claims with evidence based, factual information is vital to protect public confidence and support public health messaging.

Our anecdotal evidence is supported by the findings of the newly published OSR (Office for Statistics Regulation) review of Northern Ireland’s mental health statistics which found:

  • The scarcity of robust mental health data in Northern Ireland has hindered the development of statistics and led to significant data gaps.
  • The lack of regional standardisation and a fragmented technology infrastructure has led to poor data quality, with limited consistency or comparability across the region.
  • Users find it difficult to locate official statistics across a dispersed landscape. Data accessibility could be improved.

In many ways these issues will be a fundamental challenge to the successful delivery of the new Mental Health Strategy. We need timely and robust data to underpin everything we do.

As that famous old business consultancy cliché goes:

“What gets measured gets done”

We have a unique opportunity with the new strategy in Northern Ireland to change how we support those with mental health issues, and robust and reliable data that targets investment and ensures better outcomes must be our goal.

You can find out more about Action Mental Health’s work by visiting our website or follow us on Twitter.

The Code pillars: Value – bringing something to the party

Value for me is about why it all matters. Value means that statistics and data are useful, easy to access, remain relevant, and support understanding of important issues. These things mean that the statistics will be used. Without statistics being of value, they risk becoming irrelevant.

Statistics should bring something to the party.

But why should we be invited to the party in the first place?

The pandemic has demonstrated the crucial value of statistics and of statisticians being involved in decision making and debate.

The statistical community really stepped up in providing new, innovative and highly relevant analyses. We sought to provide answers to the questions that decision makers and society needed answered. We saw the power of statistics to inform, to paint a picture. That is what value is all about. We were at the heart of the party! Without us being there, statistics cannot serve the public good. We need to value ourselves, and our statistics to demonstrate that value.

The value of valuable statistics

Early in my career, my colleague and I organised a session about government statistics for a school ‘careers in maths’ day. We were going to spend all day talking to 14-year olds about maths – probably not the most exciting prospect to them.

To try to engage them we developed a session based around why (government) statistics are important. The session, called ‘King for a Day’, involved crowning a child king (or queen) and getting the children to develop the list of statistics that they would need to run the kingdom.

Nothing was out of bounds. If the children decided that knowing the number of goals scored by the top football teams was a priority for their king, then it went on the list. I didn’t realise it at the time, but we had decided to talk about the value of statistics rather than simply how to produce good quality ones.

Without valuable statistics, the children realised their kingdoms couldn’t run properly, and their ‘citizens’ couldn’t hold them to account. They learned the value of valuable statistics, and why they are essential for the public good.

So how do we ensure that we are invited to the party?

We keep getting invited by….

  1. Being relevant – engaging in conversation with others at the party, listening, understanding what they need and responding accordingly.
  2. Being accessible – recognising that different party goers need different things to get the most from the party.
  3. Being clear and insightful – clearly explaining to the others what we are bringing to the party and how it can be useful. Ensuring that what we bring compliments what others are bringing.
  4. Being innovative –keeping listening and improving what we bring.
  5. Being efficient – recognising that we can share resources. Providing a clear rationale for why we are asking for certain things to be supplied to the party, and not overburdening others by asking them to contribute too much.

In short, following the Code of Practice for Statistics, and adhering to its three pillars; Trustworthiness, Quality and Value, ensure that statistics serve the public good.

For more information on the Code and the three pillars, you can visit the Code website. There are also case studies that demonstrate how statistics producers have implemented different practices in the Code.

 

 

 

 

 

 

Empowering statistical leaders

As part of our work on statistical leadership, we are hosting a series of guest blogs. This blog is from Stian Westlake, Chief Executive at the Royal Statistical Society.

From remote working to online shopping, the pandemic has been a great accelerator of long-term shifts. It has done much the same thing to the role of data and statistics within public life.

This was really brought home to me in late 2020 when the RSS’s panel of distinguished experts sat down to decide our Statistic of the Year. In past years, our choice had usually generated a quirky news piece, intended to highlight how statistics could make sense of the big stories of the year. But when we looked through the nominations for 2020, we realised things were different: the statistics before us were the big stories of the year. The news of the pandemic, its spread, and its impact on lives and on society were being understood through the medium of statistics.

In much the same way, statistics became a central tool of our collective efforts to understand and to tackle Covid. Crucial projects like the Coronavirus infection survey, the UK Coronavirus dashboard, and the RECOVERY trial were as central to the UK’s Covid response as Nightingale hospitals or the vaccine procurement programme, and each was, in its different way, an exemplary statistical undertaking. Statisticians were in demand across government, and proved their value time and again.

So it is extremely timely that earlier this year the Office for Statistics Regulation published its major review, Statistical leadership: making analytic leadership count. Others have written eloquently about several of the main themes of the report, such as the importance of statistical skills, and of transparency and trust. While these are dear to the RSS’s heart, there is another theme in the report that I think is especially important: empowering statisticians to provide leadership, and ensuring they have strong career prospects.

One way of thinking about the crucial importance of empowered statisticians is to consider the counterfactual. What happens when if the other conditions for statistical leadership – such as technical skills and transparency – are met, but if organisations fail to give statisticians the right organisational roles, access and opportunities?

When this happens, we see a very particular failure made. Statisticians are left out of the loop of strategic planning. Data is seen as a specialist function to be commissioned as an afterthought, often to justify rather than inform a decision. And the commissioning process breaks down: statistical projects are assigned by leaders with limited statistical background, sometimes with unrealistic objectives and little chance to iterate during the project. The near-term results are projects that are frustrating to work on and disappointing for users. The longer-term results is that skilled statisticians are demoralised and drift away. We’ve all seen organisations like this; we may even have worked at some. Sometimes statistics or statisticians get the blame, and we hear talk of mutant algorithms or statistical errors. But the root cause isn’t in the data or the methodology: it is a problem of organisation.

But the good practice of the past eighteen months have shown to the world at large that there is a better way. In our experience, this relies on a few elements.

First of all, putting statistics, data and evidence at the heart of the organisation’s strategy. This means those senior leaders who aren’t statisticians gaining the skills to be users of statistics and to work well with statisticians, and statisticians being supported and trained to take senior leadership roles, rather than existing as a permanent auxiliary function. This helps make statistics and data intrinsic to the organisation’s workings.

Secondly, it requires strong career development opportunities for statisticians. Technical skills are important, but for true statistical leadership these need to be complemented with opportunities to learn general management and other operational skills. Crucial to this is mentorship. (This is why the RSS runs a mentorship scheme for candidates for our Chartered Statistician designation.) One of the silver linings for some statisticians of the immense workload imposed by the pandemic has been the exceptional opportunities to try out new roles in other organisations, as statistical and data skills have been at such a premium. Wouldn’t it be good if the volume and quality of these opportunities continued once the burden of Covid-related work has subsided?

Thirdly, it requires managers and heads of profession to be mindful about the make-up of the profession and to ensure it is open, diverse and growing. Research has shown that lack of opportunity and diversity is a big barrier to society’s scientific potential; it is likely that the same holds true for our discipline. This means redoubling our efforts to increase the diversity of the statistical workforce when it comes to protected characteristics. It also means promoting non-traditional routes into the profession, building on the GSS’s apprenticeship and degree apprenticeship scheme, and making the most of in-work skills schemes like the RSS’s Data Analyst, Graduate Statistician and Chartered Statistician designations, and the competency framework we have designed for them.

Getting these vital human-level, organisational questions right is essential for a thriving statistical profession. And that in turn is indispensable for anyone who cares about rigorous, useful, trustworthy statistics.

The Code Pillars: Quality

When I joined OSR as a placement student last September, the Code of Practice for Statistics was unknown territory. It certainly sounded official and important. Was it password protected? Would I need to decipher something or solve a puzzle to get in?

It soon became clear to me that this elusive ‘Code’ was at the heart of everything I would be doing at OSR. Not wanting to remain in the dark any longer, I dutifully dragged it to the front of my bookmarks bar and began to familiarise myself with its contents. (Thankfully no complicated code-cracking required).

The Trustworthiness and Value pillars appeared to be pretty straightforward. Yet, something about the Quality pillar didn’t seem quite so inviting. It sounded like the technical, ‘stats-y stuff’ pillar, that my degree background in economics and politics would surely leave me ill-equipped to understand.

*Spoiler alert* I was wrong.

It turns out that ensuring statistics are the highest quality they can be, isn’t as complicated and technical as I once feared. Quality simply means that statistics do what they set out to do and, crucially, that the best possible methods and sources are used to achieve that.

There are lots of ways that statistics producers can meet these aims. For example, quality can be achieved through collaboration. This can be with statistical experts and other producers, to arrive at the best methods for producing data. It can also be with the individuals and organisations involved in the various different stages of the production process – from collecting, to recording, supplying, linking, and publishing. Collaborating in these ways not only helps to ensure that statistics are accurate and reliable, but also that they are consistent over time and comparable across countries too.

There are lots of other important-sounding documents like our Code of Practice that set out national or international best practise and recognised standards and definitions for producing statistics and data such as the GSS harmonisation standards and the Quality Assurance Framework for the European Statistics System. These also help producers ensure that their statistics and data meet the highest possible standards of quality.

Quality is not only important at the producer-end of the equation, but at the user-end too. It is vital that producers are transparent with their users about how they are ensuring the quality of their statistics. This means telling users about the steps they take to achieve this, and being clear with them about the strengths and limitations of the statistics with respect to the various different ways in which they could be used.

For an indication of just how important quality is, the Quality Review of HMRC Statistics we conducted last year is a prime example. After identifying an error in its published Corporation Tax receipt statistics, HMRC asked us to assess its approach to managing quality and risk in the production of its official statistics. With the Code as our guide, we were able to review HMRC’s existing processes and identify potential improvements that could be made to reduce the risk of statistical errors in the future.

This is just one example of how high-quality data fulfils our vision of statistics that serve the public good. We have found many others across our work and we continue to support producers to consider quality when producing statistics. Last year, we published new guidance for producers on thinking about quality, which was inspired by the HMRC review and the questions we asked.

If you’re interested in finding out more about Quality and the other pillars of our Code, check out the Code of Practice website. I promise it’s not as scary or elusive as it sounds…

 

Did you know we have case studies on our Code website too? Here are some of our examples that highlight examples of good practice in applying the quality pillar of the Code.

  • Q1 – Ensuring source data is appropriate for intended uses
  • Q2 – Developing harmonised national indicators of loneliness
  • Q3 – Improving quality assurance and its communication to aid user interpretation

Leaving school during a global pandemic

What are the consequences for young people leaving school as a result of the pandemic?

How can more detailed statistics about school leavers help us understand and effect real change for our young people?

Last year we published our UK wide report – Exploring the public value of statistics about post-16 education and skills. This was an in-depth look at the post-16 education sector and covered statistics on workforce skills, universities and higher education, further education and colleges and apprenticeships. Doing a multi country, multi sector report of this nature was for me, a challenge in many ways, not only due to the fact that we were engaging with multiple producers and users all with diverse viewpoints, but also because there was a myriad of different statistics as well as data gaps to consider. We also wanted to ensure that areas of good practice and shared learning opportunities were highlighted across the four nations.

Our research highlighted the following areas as being of greatest importance to the sector:

  1. the coherence of the available statistics, how they inform the bigger picture
  2. the accessibility of the statistics to users
  3. how well the current statistics fully meet the information needs of users and understanding where there may be information gaps

They say that timing is everything and of course by July last year we were in the midst of the global pandemic with the post-16 sector like many others facing immense challenges. We still felt however, that it was important to publish and share what we had found.

One year on…

Across the country this month, young people are leaving compulsory education and making decisions on their future career prospects. As both a parent and regulator in the children, education and skills domain, I think of the tough decisions young people are making, with the stakes seemingly higher than ever in a world of increased uncertainty during the pandemic. We need to ensure that the data available to help them is timely, relevant and accessible to those that need it.

We have been encouraged that, even with the challenges faced by the post-16 education sector, we have seen many of the recommendations we made in our report progress, but there is more to do.

Statistics to make a real difference

Leading the user engagement of the Scotland statistics, I remember how passionately some researchers spoke about the need for good quality statistics to track individuals from their early years in the education system, through to the choices they make in their post-16 years and beyond. They felt this could make a difference, building an evidence base to support targeted interventions at the right time.

It was also an eye opener for me to find out about the complexities around linking this data using a common unique identifier between schools, colleges and universities as well as other post-16 options. Again, the real value comes when the linked datasets tell the stories and thus allow progress and change within the education system. This has benefits beyond those who have been linked in the data as it enables researchers understand issues and develop appropriate solutions for the future.

As we continue our engagement with the relevant statistics producers, we will encourage them to address issues around data granularity, quality and linkage so those working within this sector can understand and effect real change for our young people. As the effects of COVID-19 may affect their outcomes for decades to come – they deserve it now, more than ever.

If you wish to discuss user views for post-16 education and skills statistics please get in touch with us.

Letting the good data shine: The state of the UK system of statistics 2021

At OSR we’ve long been concerned about the risks that a world of abundant information and misinformation could lead to a catastrophic loss of confidence in statistics and data – and that our public conversation, cut loose from any solid foundations of evidence and impartial measurement, would be immeasurably weakened as a result. That is, at root, what we exist to prevent. I have written about this before as a form of statistical Gresham’s Law – how the risk is that the bad data drive out the good, causing people to lose confidence in all the evidence that’s presented to them in the media and social media.

I’ve also said that this is not inevitable, and indeed we can easily envisage a reverse effect: the bad data being driven out by the good data – that is, the trustworthy, high quality, high value data.

What it takes to secure this positive outcome is a public sector statistical system focused on serving the public good. A system that does not regard official statistics as just a Number, shorn of context, calculated in the way it always has been done, some kind of dusty relic. Instead a system that regards the production of statistics as a social endeavour: engaging with users of statistics, finding out what they want and need to know, and responding in a flexible and agile way to meet those needs.

The pandemic has really tested the public sector statistical system and its ability to provide the good data, the trustworthy, high quality, high value data. The pandemic could have seen us being overwhelmed with data from a wide range of sources, some less reliable than others. It could also have seen Government statistics retreating to the Just a Number mindset – “we just count cases, it’s up to others to decide what the numbers mean”. But the system has not done this. Instead, as our report published today shows, the statistical system has passed the test with flying colours.

Statistical producers (producers) across the UK nations and system-wide have responded brilliantly. They have shown five attributes. It’s easy to see these attributes in the work of public health statisticians and ONS’s work on health and social care statistics. They have done great things. But what’s clear to us is that these attributes are system-wide – appearing in lots of producers of statistics and across lots of statistical areas.

Letting the good data shine: The state of the UK system of statistics 2021 Responsive and proactive

Producers across the UK governments have been responsive, proactive and agile in producing data and statistics to support policy and to provide wider information to the public which really adds value. For example the ONS launched the Coronavirus (COVID-19) Infection Survey in swift response to the pandemic. The survey provides high-quality estimates of the percentage of people testing positive for coronavirus and antibodies against coronavirus. These statistics provide vital insights into the pandemic for a wide range of users, including government decision-makers, scientists, the media and the public that are essential for understanding the spread of the virus, including the new variants.

Letting the good data shine: The state of the UK system of statistics 2021 Timely

Producers have responded impressively to the need for very timely data to ensure that decisions around responses to the pandemic are based on the most up-to-date evidence. For example, the ONS published the first of its weekly Economic activity and social change in the UK, real-time indicators (previously called Coronavirus and the latest indicators for the UK economy and society) publications in April 2020, one month after the UK first went into lockdown and has continued to do so ever since. The publication contains a series of economic and social indicators (for example, card spend, road traffic and footfall), which come from a variety of different data sources. These assist policymakers with understanding the impact of the pandemic and gauging the level of overall economic activity. During the early weeks of the Covid-19 pandemic, the Department for Transport rapidly developed near-to-real-time statistics about Transport use during the coronavirus (COVID-19) pandemic. The statistics were regularly used by No10 press conferences (example in slide 2) to show the change in transport trends across Great Britain and gave an indication of compliance with social distancing rules.

Letting the good data shine: The state of the UK system of statistics 2021Collaborative

Collaboration and data sharing and linkage have been a key strength of both the UK statistical system and the wider analytical community over the past year. This more joined-up approach has improved our understanding of the impact of the pandemic both on public health and on wider areas such as employment and the economy. For example, during the pandemic, ONS and HMRC accelerated their plans to develop Pay as You Earn (PAYE) Real Time Information (RTI) estimates of employment and earnings. The Earnings and employment from PAYE RTI is now a joint monthly experimental release that draws from HMRC’s PAYE RTI system which covers all payrolled employees and therefore allows for more detailed estimates of employees, rather than a sample based approach, as well as information on pay, sector, age and geographic location.

Letting the good data shine: The state of the UK system of statistics 2021Clear and insightful

We have seen some good examples of clearly presented and insightful statistics which serve the public good. For example, Public Health England (PHE) developed and maintain the coronavirus (COVID-19) UK dashboard. This dashboard is the official UK government website for epidemiological data and insights on coronavirus (COVID-19). The dashboard was developed at the start of the pandemic to bring together information on the virus into one place to make it more accessible. Initially it presented information for the UK as a whole and for the four UK countries individually. Over time it has developed so that data are now available at local levels. We have seen the increasing use of blogs to explain to users how the pandemic has affected data collection, changes to methodologies and bring together information available about the pandemic. For example, the Scottish Government has blogged about analysis and data around COVID-19 available for Scotland. We have also seen examples of statisticians engaging openly about data and statistics and their limitations, both within and outside government, helping the wider understanding of this data and statistics. For example, Northern Ireland Statistics and Research Agency (NISRA) statisticians have introduced press briefings to explain their statistics on weekly deaths due to COVID-19. The Welsh Government Chief Statistician’s blog is a regular platform for the Chief Statistician for Wales to speak on statistical matters, including providing guidance on the correct interpretation of a range of statistics about Wales.

Letting the good data shine: The state of the UK system of statistics 2021Transparent and trustworthy

For statistics to serve the public good they must be trustworthy, and this includes statistics being used and published in an open and transparent way. We have seen efforts to put information in the public domain and producers voluntarily applying the Code of Practice for Statistics (‘the Code’) to their outputs. For example, the Department of Health and Social Care (DHSC) publishes weekly statistics about the coronavirus (COVID-19) NHS test and trace programme in England. DHSC has published a description about how the pillars of the Code have been applied in a proportionate way to these statistics. However, inevitably the increased volume of and demand for data has placed a greater burden on producers and led to selected figures being quoted publicly when the underlying data are not in the public domain.

But our report also shows how the system cannot take these 5 attributes for granted. What has been achieved in the high pressure environment of a pandemic must be sustained as we ease out of being a pandemic society. New challenges – like addressing regional imbalances, or moving to a greener economy or addressing issues like loneliness and inequality – cannot be understood using objective statistics if the system retreats into the Just a Number mentality.

So, our report sets a number of recommendations. The recommendations aim to make sure that the statistical system we have seen in the pandemic is not an aberration, but is – in the classic pandemic phrase – the new normal. A system that can harness these five attributes is one that serves the public good. It is the best way to ensure that the bad data do not thrive and the good data shine out.

 

 

 

 

Glimmers of light for adult social care statistics

I was very interested in a recent Social Finance report on how to secure impact at scale. One of their points is that, if you want to see impact at scale, you need to be willing to seize the moment. Change arises when supportive policies and legislation fall into place, and when a new public conversation starts.

This idea – new policies, and new public conversations – made me think of social care statistics. It’s very tragic that it has taken the disastrous impact of the pandemic in care homes to focus attention on this issue, but there seems to be some potential for progress on the statistics now.

The background is that we’ve been shouting about the need for better statistics for the last couple of years. We’ve done so through reports on social care statistics in England , Scotland and Wales . We’ve done it through presentations and I’ve taken the opportunity to highlight it when I’ve given evidence at the Public Administration Committee in the House of Commons.

Certainly, we found some excellent allies in Future Care Capital and the Nuffield Trust, yet it has sometimes felt like we’re in the minority, shouting in the wilderness.

What were our concerns? Our research in 2020 highlighted that there were several challenges and frustrations related to adult social care data that were common to England, Scotland and Wales. Our report summarising the common features of the statistics across Great Britain highlighted four key needs to help both policymakers and individuals make better informed decisions about social care:

  • Adult social care has not been measured or managed as closely as healthcare, and a lack of funding has led to under investment and resourcing in data and analysis.
  • There is an unknown volume and value of privately funded provision of adult social care. Although data is collected from local authorities, this only covers activities that they commission and fund, which constitute a smaller proportion of total adult social care activity.
  • Robust, harmonised data supply to ensure comparable statistics from both public and private providers is problematic, as data collection processes are not always standardised. Furthermore, data definitions might not always be consistent across local authorities and other providers.
  • Data quality is variable within and across local authorities, with inconsistent interpretation of data reporting guidance by local authorities. This means that data isn’t always reliable and so users have difficulty trusting it.

As data issues go, as the pandemic has highlighted, there is not so much a gap as a chasm, with consequences to our understanding of social care delivery and outcomes.

Most people we’ve talked to, inside and outside the UK’s governments, recognise these issues. But to date there hasn’t been much evidence of a sustained desire to inject energy into the system to effect change.

Maybe, though, there are glimmers of light. Whilst this list is not meant to be exhaustive, I would like to draw attention to some initiatives that have caught my eye.

  • The first comes from an extremely negative space. That is the pandemic’s impact on those in care homes. Not only has the pandemic highlighted the importance of care and care workers, it has also led to much more interest in data about the care home sector. The Care Quality Commission and the Office for National Statistics (ONS) collaborated to publish timely information on the numbers of deaths in care homes , to shine a light on the impact of the pandemic for this vulnerable population. And DHSC has commenced the publication of a monthly statistics report on Adult social care in England to fill a need for information on the social care sector itself. This means that COVID-19 has resulted in people listening to analysts and statisticians when we raise the problem with social care data now. Of course, the questions people are interested in go well beyond COVID-19.
  • The Department for Health and Social Care’s draft data strategy for England makes a significant commitment to improving data on adult social care.
  • The Goldacre Review for data in England may present a further opportunity for improvement.
  • I was pleased to restore the National Statistics designation to the Ministry of Housing, Communities and Local Government’s statistics report about local authority revenue.
  • Beyond the pandemic, ONS is working in collaboration with Future Care Capital to shine a light on one of the biggest data gaps here: the private costs borne by individuals and families for care. And ONS has recently published estimates of life expectancy in care homes prior to the pandemic.
  • Adult social care remains high on the political agenda in Scotland, with the recently published independent review of adult social care by the Scottish Government and the inquiry by Scotland’s Health and Sport Committee.
  • The Welsh Government remains committed to improving the data it captures on social care .

It’s far too early to declare “problem solved”, but we ought to be optimistic about improvements to data as a consequence. We’ll be reviewing the actions currently underway as statistics producers react to the gaps in social care statistics highlighted by the pandemic and publishing a comprehensive report of our findings in the autumn.

What I do think is that there is an opportunity here – if statistics producers across the UK are willing to take it, we can anticipate much better statistics on this sector. And a much better understanding of the lives and experiences of citizens who receive, and provide, social care.

The Code Pillars: Trustworthiness is about doing things differently

Trust can seem a no-brainer. It may seem so obvious, that of course it matters. It has often featured as the guiding aim of many a strategy for statistics.

I spend much of my time explaining about the Code of Practice for Statistics and our three pillars. I think of Quality as being the numbers bit – getting the right data, using the right methods. I think of Value as being why it all matters, making sure to serve the public and society. And Trustworthiness? Well, Trustworthiness for me is a lot like reputation – as Warren Buffett once said:

“It takes 20 years to build a reputation and five minutes to ruin it. If you think about that, you’ll do things differently.”

So, the Trustworthiness pillar is about ‘doing things differently’ – for analysts and their organisations. You can’t expect just to be trusted – you must earn it.

You have to behave honestly and with integrity – you can show that in the way that you use numbers. Anyone who spins data to make themselves look good, or cherry picks the numbers that tell the best story, will reveal themselves to be untrustworthy. But those that set out the facts plainly and objectively will be seen as someone to be trusted.

How you handle data and show respect to people and organisations giving their personal information can also prove that you are a safe pair of hands. But if you are seen to lose people’s data, or share it inappropriately, you’ll probably find people are not willing to share their information with you again.

And the way you release information matters – if you give any sense of being under the sway of political opportunism, the credibility of your statistics will be harmed.

So why isn’t the pillar called ‘Trust’ if that is what we are after?

Well, the answer is thanks to the seminal work of philosopher Baroness Onora O’Neill. She said that focusing on trust is fruitless – instead, you need to demonstrate that you are worthy of trust.

Basically, you can’t force someone to trust you. You can only show through the way you behave, not just once, but repeatedly, that you are honest, reliable, and competent:

  • tell the truth
  • do what you do well
  • and keep on doing these

Being reliable in these ways will mean that people will come to trust you. But the only bit you can do is show you are worthy of trust.

So, if you reflect on your reputation for being trustworthy and you want to be sure to keep it, do things differently.

Here are some case studies on our Code website that illustrate some ways that statistics producers show their Trustworthiness: