Ed Humpherson to Siobhan Carey: Assessment of statistics on planning activity in Northern Ireland

Dear Siobhan

ASSESSMENT OF STATISTICS ON PLANNING ACTIVITY IN NORTHERN IRELAND

Thank you for inviting us to assess the Department for Infrastructure’s Northern Ireland Planning statistics against the Code of Practice for Statistics. We have completed the assessment and have today published our Assessment Report covering these statistics. I am grateful for the positive contribution and engagement from your team throughout the assessment process, especially over the last few weeks at this particularly challenging time.

During our assessment we found that the statistics are produced independently and presented impartially, providing a good view of the overall trends in planning in Northern Ireland. Users are generally happy with the presentation, content, and quality of the statistics, and are positive about the level and forms of engagement provided by the statistics team as well as reporting that they have confidence in the quality of the planning statistics produced.

We have identified several ways that the statistics should be strengthened further, to ensure that they fulfil the expectations of the Code. There are four requirements we are asking the statistics team to focus on:

  • Enhance the impact and insight of the planning statistics for a wider range of users and facilitate their further reuse, for example through making use of more innovative forms of presentation including interactive content and maps;
  • Investigate the feasibility of linking planning approvals to house building completions data and collaborate with other statisticians as necessary to help answer key questions and enable a better understanding of the NI planning statistics in relation to the overall UK planning picture;
  • Improve the information it publishes on its future planned developments and details of relevant user meetings so that other users are kept informed and have opportunities to comment on proposals;
  • Demonstrate a higher level of assurance around data quality and any associated methods changes during the transition period to the new planning portal, setting out how any potential impact on data quality or methods will be mitigated.

As is the situation with many statistics producers at the moment, the impact of COVID-19 on statistical production over the coming months is uncertain. We understand there may be an impact on the capacity of planning offices to process planning applications, the effects of which may be seen in the forthcoming publications in June/July and September this year. There may also be a need to delay the publication of quarterly planning statistics, to allow councils more time to carry out their regular data quality assurance. We are reassured that the statistics team is keeping the situation under review and will identify any changes in quality or statistical trends that may be related to COVID-19, highlighting these in the published statistics.

We judge that the Northern Ireland Planning statistics can be designated as new National Statistics once we have confirmed that the four requirements set out in the report have been met. Addressing the requirements will demonstrate that your statistics meet the highest standards of trustworthiness, quality and public value and comply with the Code of Practice for Statistics. We have agreed that the team will report back to us by October 2020 outlining how they have addressed the requirements.

I am copying this letter to Michael Thompson, Lead Statistician DfI; Tracy Power, Director of Analysis NISRA; and Ruth Fulton, Chief Statistician’s Office.

Your sincerely

Ed Humpherson

Director General for Regulation

 

Related links:

Devolved Labour Market Compliance Check

Dear all

STATISTICS ON THE LABOUR MARKET

I am writing to you following our recent review of the key Labour Market statistical reports for the devolved nations against the Code of Practice for Statistics. The statistics published by the Welsh Government, Scottish Government and Northern Ireland Statistics and Research Agency (NISRA) that we reviewed are:

The statistics have been considered as part of a wider review of labour market statistics, along with our assessment of UK employment and jobs statistics produced by the Office for National Statistics (ONS). I am pleased to confirm that all three labour market reports should continue to be designated as National Statistics.

Labour market statistics are key economic indicators which are used by a wide range of users and are subject to high user interest. This review focussed mainly on the quality and public value of the data, statistics and supporting information. We recognise that the outputs we have reviewed differ between the three countries in terms of purpose, each team’s access to the underlying data and the time window available to produce them. These factors have been considered as part of our recommendations.

In reviewing the labour market reports, we found examples of clear supporting methodology information, effective sign-posting and presentation of uncertainty, which we detail separately for each country later in this letter. We have identified some common areas for improvement across the three producer teams, which also correspond to areas for improvements we’ve highlighted in our assessment report of ONS’s employment and jobs statistics. The recommendations in this letter build on those we have made to ONS and we encourage all four countries to continue to work together to ensure that labour market statistics across the UK continue to provide the necessary insights. In order to improve the quality and public value of these statistics, the teams should:

  • Consider how the statistics can be better presented to help improve users’ understanding of how the labour market is changing over time. We found some examples where the key labour market measures are defined but the relationship between these groups of people (for example, the unemployed and the economically inactive) could be more clearly explained. We encourage the three producer teams to also work with ONS to develop a way to understand the flows of people into, out of and within the labour market.
  • Build on existing collaboration between all the producer teams, including ONS, to enhance the coherence of labour market statistics. We found strong evidence of effective cross-producer collaboration through regular meetings and steering groups. However, discrepancies between the Labour Force Survey (LFS) and the Annual Population Survey (APS) data currently present issues with coherence of data sources. Greater collaboration could support a consistent approach in presenting data from the LFS or APS respectively and in turn, lead to a better read-across between the different countries’ statistics. This will require leadership and coordination from ONS and is highlighted in our assessment report of employment and jobs statistics (para 2.5). It could also prove an effective part of finding a solution to address the concerns raised by Scottish Government and Welsh Government about the future funding for APS which is explained in the ONS assessment report (para 1.9).

Welsh Government

  • Key Economic Statistics is well presented and the narrative provides relevant context to the statistics. The section on ‘Key quality information’ is appropriately detailed and provides useful information on data sources and methods. The bulletin includes links to supporting documentation and StatsWales data tables throughout. This could be further improved by signposting relevant sections from the ‘Key quality information’ within the main bulk of the bulletin to help aid understanding.
  • The statistics team has presented confidence intervals for the LFS estimates which provides some context for the level of uncertainty associated with the data. However, these are relatively inaccessible, and the language used in the narrative presents the latest figures as absolute, for example “The employment rate in Wales was x%”. This is particularly important when comparing data across the four countries, where estimated differences are not always statistically significant. Welsh Government should improve the way uncertainty is reflected in the narrative, following the lead of ONS as recommended in our assessment report, for example referring to the latest figures as estimates.
  • We were pleased to hear from the statistics team about its plans to potentially introduce a new bulletin covering protected characteristics in the labour market, which is an area of interest identified in its 2012 user consultation. We encourage Welsh Government to keep published statistical development plans up to date and to ensure users are aware of progress being made against these developments.
  • The statistics team told us that some of the main users of Key Economic Statistics go straight to the data tables to find the information they require and not the bulletin by default. We encourage Welsh Government to find out how its users engage with the various statistical outputs to ensure they remain relevant to users.

Scottish Government

  • The Labour Market Trends bulletin is easy to follow and we welcome the improvements that have been made to the presentation of chart headings and footnotes. The bulletin signposts to the new quarterly youth APS publication, which was previously included in the monthly LFS bulletin, as well as a number of ONS pages relating to the LFS. To improve clarity of the statistics, Scottish Government should look to expand on the methodology information within the bulletin itself.
  • The statistics team told us that the process for producing the monthly bulletin has largely been automated to ensure the statistics can be published at the same time as the ONS release. As a result, the narrative in the bulletin focuses on the latest figures and the change on the previous quarter or year. We would encourage the statistics team to consider how to bring out more insight from the statistics to improve their public value.
  • We welcome the work Scottish Government and ONS are doing to ensure uncertainty is properly reflected in the bulletin, as part of our recommendation in the assessment of employment and jobs statistics, to help users understand the precision of estimates. For example, the statistics team should avoid presenting figures as absolute in the headline
    infographic such as “x% of people aged 16 to 64 were in employment” and instead refer to the latest figures as estimates.
  • We were pleased to hear from the statistics team about its ongoing user engagement and its plans for developing alternative products for accessing the data to complement the ScotGov open data platform and to meet a range of users’ accessibility needs. The statistics team also told us that its economic statistics development plan is being updated to cover a wider range of economic statistics than in previous years. We would encourage Scottish Government to increase the visibility of its developments by publishing updates and outcomes of user engagement to highlight the good work they are doing in this area and to keep users informed of their plans and progress.

Northern Ireland Statistics and Research Agency

  • NISRA’s Northern Ireland Labour Market report is engaging and comprehensive. The narrative is proportionate to the statistics and the ‘Further information’ section of the report is thorough in addressing the strengths, limitations and comparability of the data.
  • The presentation of uncertainty in the bulletin and supporting materials is effective, for example including statistical significance and confidence intervals of estimates, and we are pleased to hear the report is being used as a case study for presenting uncertainty by the Government Statistical Service’s Good Practice Team. To improve this further, NISRA should ensure comparisons between Northern Ireland and the UK also take into the account the level of uncertainty for the estimates.
  • The team carried out a user consultation of labour market statistics in 2019 and has published its planned developments in response. We would encourage NISRA to seek feedback on its progress against the developments and continue to collate feedback on its various statistical outputs.

We appreciate each of the teams’ willingness to engage with us in this review as well as the wider assessment process with ONS. We wish to thank them for taking on board our recommendations. Our labour market and welfare domain team will continue to engage with your teams over the coming months to discuss progress.

I am copying this letter to Melanie Brown (Welsh Government), Gayle Mackie (Scottish Government) and Cathryn Blair (NISRA), the lead statisticians.

Yours sincerely
Mark Pont
Assessment Programme Lead

Related Links:

Assessment Report: UK employment and jobs statistics (March 2020)

Assessment of the UK employment and jobs statistics (March 2020)

Compliance Check of Subnational Population Projections

Dear all

STATISTICS ON SUBNATIONAL POPULATION PROJECTIONS

I am writing to you following our recent review against the Code of Practice for Statistics of the four sets of subnational population projections published by the Office for National Statistics (ONS), National Records of Scotland (NRS), Welsh Government and Northern Ireland Statistics and Research Agency (NISRA):

I am pleased to confirm that all four sets of subnational population projections should continue to be designated as National Statistics.

Subnational population projections are widely used for local planning purposes and the outputs are subject to high user interest. This review focussed mainly on the accessibility of the data, statistics and supporting information, and supplements our previous compliance check of the national population projections in April 2019. We welcome the way that the teams have responded to our earlier letter, and in particular how they are planning to apply the findings there to the Subnational projections.

In reviewing the subnational projections, we found several examples of good quality assurance, clear analysis and explanations, and demonstrable trustworthiness, which we detail separately for each country later in this letter. However, we have identified some key areas for improvement across the four producer teams, some similar themes have emerged, and therefore there are some common recommendations. In order to improve the public value of these statistics, the teams should:

  • Continue to apply the findings from the previous review to ensure the positive changes to presentation and supporting documentation in the national publications are applied for the subnational publication planned for later in 2020.
  • Consider how existing channels can be used to enhance collaboration between all the producer teams. Cross-producer working arrangements and collaboration for national projections has been effective. However, discussion and consideration of issues relating to subnational projections are not normally included. While this is primarily due to subnational projections being produced separately by different producers with some differing methods, potential opportunities for collaboration and sharing of best practice at subnational level are being overlooked.
  • Be transparent about proposed changes to the timings of the publications, as well as future developments, considering the 2021 Censuses. The teams should continue to consider how best to engage with their users in relation to planned developments and changes in the wider statistical landscape, such as any impacts from the ONS migration and population transformation programme.
  • Refresh and update the comparison document which sets out the differences in methods used to produce the four sets of subnational projections, to make it more accessible and easier for users to identity differences in approach and the reasons for any differences

ONS’s Subnational population projections for England are presented clearly and objectively. The graphs and interactive graphics do a good job of aiding understanding, and there are clear explanations on the accuracy and use of the statistics. We welcome the team’s commitment to mirror the positive changes to the national population projections. For example, following our previous recommendation, future developments are explained fully, and the relevant quality information is included in the main bulletin. The bulletin published in November 2019 demonstrates a commitment to users and flagged method changes well. As in the national publication, in the next subnational release we recommend that information be provided on the future of the outputs, especially in relation to proposals for the timings of the next publication and seeking user feedback on the proposal.

NRS’s Population Projections for Scottish Areas bulletin is engaging and informative. NRS sets out the key messages clearly and objectively and make good use of charts and graphs. The methods information is well set out in the methods guide, as it is in the national population projections, NRS could improve this information by bringing some of the key information on quality into the main bulletin. The Quality Assurance of Administrative Data (QAAD) documentation is available to support this but could better describe the strengths and limitations of the data sources to give users a clearer picture of the processes in place. We were pleased to hear from the statistical team about its future communication plans and a review of the quality information. As with the national population projections, NRS should ensure future plans are clearly communicated with users, including the impact of any GSS-wide programmes such as the ONS migration and population transformation programme.

Welsh Government’s Local authority population projections statistics and analysis are presented clearly and objectively. There are good descriptions of data sources and data quality, with clear links to the accompanying quality report. The sections on comparisons with earlier data, mid-year estimates, and ONS data are useful and appropriately detailed. Some supporting documentation could be linked more clearly to help users navigate to it, for example, statements on data governance and pre-release access, as well as guidance on comparing with NRS and NISRA data. After speaking with the team, we are pleased to hear that it is considering aspects of accessibility by planning a move to HTML publication as well as PDF. The publication has recently been delayed due to quality issues. This has been clearly explained in an accompanying blog and we look forward to hearing more on this and future plans in line with the 2021 Census in the next publication.

NISRA’s Population projections for areas within Northern Ireland are comprehensive, accessible and the analysis and data are demonstrably valuable to users. NISRA’s supporting material on methodology and data quality are thorough and assure users of the trustworthiness of the publication. The team regularly engage with users through surveys and is planning an in-house user engagement event in 2020 which we welcome and encourage. NISRA could improve by mirroring changes successfully made in the national population projections, such as improving the presentation and readability of the publication.

We appreciate each of the teams’ willingness to engage with us and take on board our recommendations. We recommend that the teams continue to engage users in the proposed changes to the publications and timings and clearly communicate the impact of changes to similar publications or datasets such as those involved in the ONS migration and population transformation programme. Our population and society domain team will continue to engage with your teams over the coming months to discuss progress ahead of the next publication of the subnational population projections.

I am copying this letter to Andrew Nash (ONS), Denise Patrick (NRS), John Morris (Welsh Government) and Jonathan Harvey (NISRA), the lead statisticians.

Yours sincerely

Mark Pont
Assessment Programme Lead

 

Related Links:

Compliance Check of National Population Projections (April 2019)

Compliance check of Northern Ireland Safe Community Survey

Dear Siobhan

STATISTICS FROM THE NORTHERN IRELAND SAFE COMMUNITY SURVEY

This letter updates our review of compliance with the Code of Practice for Statistics for the Northern Ireland Safe Community Survey. In this part of the work, we focused on data collection and the quality of statistics. We also reviewed the extent to which the team has met the recommendations from our previous compliance check. These compliance checks form part of a series of reviews, the last of which will focus on the reporting of the statistics. Once complete, we will then publish an overall report of our assessment of the statistics from the Northern Ireland Safe Community Survey, which we aim to do in spring 2020. Overall, the package of work will consider trustworthiness, quality and value, all of which are fundamental to supporting public confidence in statistics and, together, form the basis of the Code.

The first compliance check highlighted that the team should demonstrate trustworthiness, ensuring resources are sufficient for the ongoing improvement of quality and to realise the potential value of the statistics. We were pleased to see that the team has recently recruited a new member to provide more resource to develop the statistics further; we look forward to seeing the statistics continue to improve as a result. We were also encouraged to see that the team has published a future work programme, showing transparency around unmet demand. Once the new team member is in post, we look forward to you firming up timescales, which would be helpful so that users can better understand when improvements are going to be finished.

We previously identified the need to understand child victimisation. We welcome the team’s commitment to publish a scoping paper looking into this area. This is an important issue and it was positive to see questions being included in the Young Persons’ Behaviour and Attitudes Survey as part of an effort to understand the nature of child victimisation more fully.

We were assured of the quality of the statistics and commend the team for working closely with our regulators and making changes to the quality and user guide throughout the course of our compliance check. Given the changes in the sample size we were encouraged to see the work that has been done to ensure the integrity of the long-term crime trend. The inclusion of confidence intervals so that users can understand the quality of the estimates more fully when results are published.

We were pleased to see the willingness of the team to engage with users and to collaborate with producers of similar statistics. We welcome that the team is currently holding an internal working group with policy colleagues and has quarterly meetings with producers from the Office for National Statistics and Scottish Government. We recommend that the team should publish findings from its user survey to ensure transparency and to enable further continued improvement to the statistics.

Our Crime and Security domain lead will continue to liaise with staff at the Department of Justice throughout these developments, and in planning the last phase of our work. I look forward to seeing the trustworthiness, quality and value of the statistics continue to improve.

I am copying this letter to Tracy Power, Director of Analysis NISRA; and to Joan Ritchie, Department of Justice.

Yours sincerely

Mark Pont
Assessment Programme Lead

 

Related Links:

Siobhan Carey to Mary Gregory (October 2018)

Mary Gregory to Siobhan Carey (August 2018)

National Statistics confirmation: Sickness Absence in Northern Ireland Civil Service statistics

Dear Siobhan

ASSESSMENT OF SICKNESS ABSENCE IN NORTHERN IRELAND CIVIL SERVICE STATISTICS

We have reviewed the actions that your team has taken to address the requirements in the Assessment Report 342 on Sickness Absence in Northern Ireland Civil Service Statistics.

On behalf of the Board of the UK Statistics Authority, I am pleased to confirm the designation of National Statistics for the NISRA Sickness Absence in Northern Ireland Civil Service Statistics. Given the actions taken, you may retrospectively apply this to the most recent publication dated 20 June 2019.

During the course of the assessment the team responsible for producing the statistics showed great commitment to the improvement of these statistics. We commend the team for bringing forward the publication date of the statistics and for publishing the full report to this quicker timetable, thereby increasing the value of the statistics. We welcome the progress made in ensuring collaborative engagement with key stakeholders and look forward to seeing the actions taken because of this process.

We have included more detail about our judgement in an annex to this letter. If necessary I, or my team, would be happy to talk you or your colleagues through any aspects of this letter or Code compliance more generally.

National Statistics status means that official statistics meet the highest standards of trustworthiness, quality and value and is something to be celebrated. We invite you to include a statement alongside the statistics which reflects the National Statistics status.

I am copying this letter to Tracy Power, Director of Analysis (NISRA); Danielle Clarke, HR Consultancy Services (NISRA) and Ruth Fulton, Head of NISRA Statistical Support and Business Planning.

Your sincerely

Ed Humpherson
Director General for Regulation

 

Related Links:

Assessment Report 342: Statistics on Sickness Absence in the Northern Ireland Civil Service

Statistics on Sickness Absence in the Northern Ireland Civil Service

Dear Siobhan

ASSESSMENT OF SICKNESS ABSENCE IN NORTHERN IRELAND CIVIL SERVICE STATISTICS

Thank you for inviting us to assess NISRA’s Sickness Absence in Northern Ireland Civil Service statistics against the Code of Practice for Statistics. We have completed the assessment and have today published our Assessment Report. I appreciate your team’s positive engagement throughout the assessment process, and for its proactive approach and commitment to addressing our requirements during the course of the assessment.

The sickness absence statistics provide a quality controlled annual version of management information on Northern Ireland civil service sickness absence. The value of these statistics to users is clear, and they recognise the high data quality standards applied to their production. Our report identifies many examples of good practice. In particular:

  • The team producing the statistics is experienced and exercises sound judgment, especially around decisions relating to definitions of part-time and full-time employee sickness absence;
  • The processes for data gathering and collection are robust and well-established;
  • The team has positive links with users of the statistics and is flexible to their needs; and
  • We found the team signposted relevant statistics, providing a coherent picture of sickness absence statistics in UK.

We identified a number of improvements, which in the course of the assessment your team started working on. These include publishing the headline figures earlier, before the release of the analysis and insight report. Meeting the requirements will ensure that further value is added to the statistics.

We judge that the Sickness Absence in Northern Ireland Civil Service statistics can be designated as National Statistics once we have confirmed that the five requirements set out in the report have been met. Addressing the requirements will demonstrate that your statistics meet the highest standards of trustworthiness, quality and public value and comply with the Code of Practice for Statistics. We have agreed that you will report back to us by 31 July 2019 outlining how you have addressed the requirements.

I am copying this letter to Tracy Power, Director of Analysis NISRA; Tony Mathewson, NISRA and Ruth Fulton, Chief Statisticians Office.

Yours sincerely,

Ed Humpherson
Director General for Regulation

 

Related Links:

Assessment Report: Statistics on Sickness Absence in the Northern Ireland Civil Service