Mark Pont to Siobhan Carey: Statistics from the Northern Ireland Safe Community Survey

Dear Siobhan

STATISTICS FROM THE NORTHERN IRELAND SAFE COMMUNITY SURVEY (NISCS)

We have completed our review of the compliance of statistics from the Northern Ireland Safe Community Survey, produced by the Department of Justice (DoJ), with the Code of Practice for Statistics. This was the final part of our three-part review, which began in May 2018. This letter sets out our key findings from the review. I am pleased to confirm the statistics from the survey should continue to be designated as National Statistics: in this letter we have highlighted several areas for improvement, which will ensure the statistics continue to meet the highest standards of the Code.

As we completed our review, the COVID-19 pandemic meant the Safe Community Survey, which was a face-to-face survey, had to pause while arrangements were made to collect data for the 2020/21 statistics by phone instead. We support this approach, which will ensure the safety of those collecting data for, or contributing data to, the statistics. We look forward to seeing how your plans develop for the 2020/21 statistics and encourage the team to keep users of the statistics informed about the changes.

The rest of this letter briefly summarises our approach to the recent review and our findings, which will apply to the 2019/20 statistics. We illustrate areas where progress has been made since the first part of the review, as well as areas where we consider improvements should still be made to enhance the value of the statistics.

Our review of these statistics has been in three parts; this part focusses on the presentation and reporting of the statistics. As part of our research, we spoke to a range of users of the statistics to understand the impact of recent changes to the statistical bulletins and the extent to which users feel that their needs are being met.

In each part of our review, we considered the trustworthiness, quality and value – all of which are fundamental to supporting public confidence in statistics and, together, form the basis of the Code – of the survey and the statistics. Taken together, our findings across the review are as follows.

 

Trustworthiness

Ensuring sufficient resource

The team has taken on two new statisticians during the course of our review, allowing improvements to the current statistics and to address the improvements suggested in the earlier phases of our work. A new member of staff has now joined the statistics team and the team has published a Future Programme of Work, which provides a high-level outline of developments that the team aims to address in relation to the survey and statistics. These steps demonstrate DoJ’s prioritisation of, and commitment to, the statistics, while offering greater transparency about upcoming changes to the statistics for users. Users we spoke to said that engagement on the Future Work Programme had been good and they were happy with the information around the proposed statistical priorities. We note that dates for different pieces work on the Future Programme are still to be confirmed. In order to enhance trustworthiness, the team should include more-explicit timescales in the Programme and we encourage DoJ and NISRA to continue to ensure enough resources in the team to fulfil this work. Including this information will also allow users of the statistics to more easily engage with DoJ on the order and prioritisation of developments.

 

Quality

Improving data sources

Throughout this review, the statistics team has demonstrated ongoing commitment to improving the quality of the statistics from the Safe Community Survey. In general, the information about quality provided alongside the survey and statistics in the Quality Report and User Guide is extensive, and provides helpful information that allows users to understand the strengths and limitations of the data and statistics.

The survey sample size has increased from 2,000 to 3,500 from April 2018. This was achieved through additional financial investment in the survey, in response to feedback that statistics from the survey were no longer adequate to track progress against outcomes in the Northern Ireland Executive’s Programme for Government Framework. Given the change in the sample size, it was good to see that the team had done some work to ensure the integrity of the long-term crime trend estimates, and included confidence intervals around published survey estimates, so that users can more easily understand the quality of the statistics.

Regional Data

Despite the increase in survey sample size since 2018, some users of the statistics we spoke to still feel that estimates below country and population level need to be more robust. For example, when looking at disaggregated data for different counties in NI, users feel they are unable to make evidenced-based decisions because the confidence intervals around the estimates are so large. This issue arises because the sample sizes for geographic areas and demographic groups below country and population level respectively are relatively small. The team is aware of the user demand for robust statistics at a more disaggregated level, especially for analysis at Local Government District level, and is currently considering how this might be met. In addition to considering how the user need at low geographic levels can be met, the statistics team should consider how it can most effectively explain uncertainty around the survey estimates, so that users understand why the confidence intervals around disaggregated data are variable and can be large. The team should also be clear about how the statistics can and cannot be used.

Impact of COVID-19

It is good that the statistics team was able to move from face-to-face survey data collection to phone data collection. The team has been working well with policy colleagues to discuss changes to the questions and informing users of the changes happening to the survey on its website. We understand that there are some areas that will not be able to be covered in a phone survey. However, it is good know that important topics have been retained, such as:

  • experience of crime
  • perceptions of crime and anti-social behaviour
  • confidence in the police and police accountability arrangements

It is also good that the team has contacted both ONS and Scottish Government to discuss their experiences of changing to a phone survey design, this type of cross-nation engagement will further the improvement of the statistics, particularly during the current challenges. The team now has quarterly meetings with statisticians from the ONS and the Scottish Government, which will enable shared learning and provide an opportunity to discuss current issues and future development.

The change of data collection method will have an impact on the type of data that can be collected and the questions that can be asked. As this work develops, we encourage the team to be clear to users in advance about the impact this may have on future statistics, for example on trends in domestic violence, which could be difficult to collect by phone.

Value

Continued relevance to users

We have been impressed by the statistics team’s collaborative and user-focused approach to developing these statistics. This approach continues to help the statistics team through the current difficulties; the communication between the team, the Scottish Government and ONS will ensure the continued value of the statistics despite a more reduced output in 2020/21.

In 2018, the statistics team completed a consultation process and had extensive dialogue with users and potential users in government about their needs from, and proposed changes to, the survey and statistics. The team took account of these needs, by accommodating demand from policy teams to collect data that reflect important issues not previously covered by the survey. Several of the users we spoke to had been able to input into the questions asked in the survey. To maintain an understanding of the use and potential use of the statistics and data, the team now holds a biannual working group meeting with key policy and statistical stakeholders. In addition, the team has recently completed a branch-wide customer survey which sought comments on individual branch publications including NISCS reports. We look forward to seeing the results of this survey. Having this dialogue will help ensure that the maximum value is achieved from the survey and the statistics.

One of the key messages from the DoJ consultation in 2018 was that the presentation and reporting of the statistics needed to be improved. In response, the statistics team has introduced a new layout for its statistical bulletins, with changes to the content and presentation of the data and supporting narrative. Many of the users we spoke to now find the statistics to be useful and the narrative easy to follow: in particular, the use of annotations on charts has proved effective, as this helps users to better understand trends in the data.

Greater insight for users

As part of the improvements to the statistical bulletins, the inclusion of tables of police recorded crime data alongside the survey data in the Experience of crime publication gives users a fuller picture of crime in Northern Ireland. Our user engagement highlighted that, while the addition of these data is welcomed, there is still more that could be done to maximise the insight users get from them: the statistics team should include commentary that highlights which source of data is best for answering the different questions users might have about different crime types, for example. The crime statistics produced by ONS do this well and provide a helpful example. We anticipate the positive relationship the team already has with statisticians in ONS will facilitate greater insights.

Among the planned developments to the statistics, an issue worth highlighting is the clear demand to understand child victimisation. It has been positive to see questions being included in the Young Persons’ Behaviour and Attitudes Survey as part of an effort to meet this need, it also good that the Northern Ireland Safe Community Survey: Assessment of the Feasibility to Include Children Aged 10 to 15 and Communal Establishments was published on 17 April 2020. To maintain transparency, we encourage the team to continue to communicate its plans effectively to users in this area. This is an important issue and it is vital that the team is transparent with users about whether and when the data and statistics could be improved, remaining clear about the possible effects of moving to a phone survey.

I would like to thank your team for working closely with us throughout the review process and for its commitment to improving these statistics. Our Crime and Security team will continue to liaise with you and your team and, if you wish to discuss any aspects of this letter, we are happy to do so. I look forward to seeing the trustworthiness, quality and value of the statistics continue to improve. I am copying this letter to Ruth Fulton, Head of NISRA Statistical Support Branch; and to Joan Ritchie, the crime statistician at the Department of Justice.

Yours sincerely

 

Mark Pont

Assessment Programme Lead

 

Related Links

Compliance check of Northern Ireland Safe Community Survey

Siobhan Carey reply to NISCS compliance check

Northern Ireland Safe Communities Survey

Mark Pont to Siobhan Carey: Northern Ireland Multiple Deprivation Measure

Dear Siobhan 

 NORTHERN IRELAND MULTIPLE DEPRIVATION MEASURE  

I am writing to you following our review of the indices of deprivation statistics produced by the Ministry of Housing, Communities and Local Government (MHCLG), the Scottish Government and the Welsh Government against the Code of Practice for StatisticsThe statistics were reviewed against the three pillars of the Code: trustworthiness, quality and value.  

As you are aware, our review did not cover Northern Ireland as the Northern Ireland Multiple Deprivation Measure (NIMDM) is not produced to the same time scale as the other indices. However, there may be findings from our review which are applicable to the future development of NIMDM. 

If you have any questions about our findings, our Labour Market and Welfare team is happy to engage with you and your team. 

I am copying this letter to Dr Tracy PowerDirector of Analysis for NISRA and to Ruth Fulton, Head of Statistical Support Branch at NISRA. 

 Yours sincerely 

 

Mark Pont 

Assessment Programme Lead  

 

Related Links

Mark Pont to Sandra Tudor: English Indices of Deprivation 2019 statistics

Mark Pont to Roger Halliday: Scottish Index of Multiple Deprivation 2020 statistics

Mark Pont to Glyn Jones: Welsh Index of Multiple Deprivation 2019 statistics

Response from Siobhan Carey to Ed Humpherson: Devolved public finance statistics-Northern Ireland

OSR Review of the Public Value of Devolved Public Finance statistics

Dear Ed

Thank you for your correspondence regarding the constructive review that the OSR recently completed regarding the “Public Value of Devolved Public Finance statistics”. I apologise that a range of other competing priorities have prevented me from responding until now.

As you may be aware, the level of interest in Northern Ireland public finances has grown quite considerably over the course of the past year. In addition, the return of the NI Executive along with its associated spending priorities (including those arising from the global coronavirus pandemic) will no doubt heighten interest in, and scrutiny of, public finances and, in particular, the discharge of the NI Executive’s budget.

While the precise remit, objectives and workings of the new independent Fiscal Council committed through the ‘New Decade New Approach’ agreement have still to be formulated, it will provide a fresh and renewed impetus for such scrutiny and the delivery of greater insights and compelling narratives from the data as the review recommends.

The work that we do in the peer review process of the Country and Region Public Sector Finance Statistics publication is undertaken in conjunction with economist colleagues in the Department of Finance. With their support, I am happy to work collaboratively with the ONS and the other Devolved Administrations to develop more engaging messages and better ways of presenting these data. Clearly it will be important that these are commensurate with the needs of users in Northern Ireland and further afield.

I also support the need for better quality country and region public sector investment statistics including estimates for the value of assets and liabilities, which will be beneficial in our supply & use tables work in terms of ensuring more robust data on the investment side. It would also be useful to have better information on capital formation statistics. Again, I am more than content to work with colleagues across the Devolved Administrations and at the ONS in developing these statistics over the medium term.

In closing, I very much welcome the findings from your review and am committed to working collaboratively with the various stakeholders within the Department of Finance, the NI Executive, at the ONS and in the other Devolved Administrations, to address the recommendations that have been made.

Yours sincerely,

SIOBHAN CAREY, CBE
Registrar General for Northern Ireland & Chief Executive NISRA

 

Related links:

The Public Value of Devolved Public Finance Statistics

Letter from Ed Humpherson to Siobhan Carey

Letter from Dr Eugene Mooney regarding statistical information relating to COVID-19 in Northern Ireland

Dear Ed,

RE:    Publication of Daily COVID-19 related statistics via a Public facing Dashboard

I am writing to you to set out our proposed plans for the orderly release of COVID19 statistics by the Department of Health Northern Ireland into the public domain.

It would be my intention to release daily information in the form of a public facing dashboard on COVID19 beginning Friday 1st of May 2020 at 2.00pm each day. I recognise that this would be a departure from the normal release practice of the standard publication time of 9.30am but would suggest that by moving to this release time we will be able to provide the latest position on a range of statistics that are of interest to our users. It will, for example, allow us to present a range of information reflecting the position in the preceding 24 hours as of 9.00am each day. It would be my intention to apply the normal pre-release practice.

Once the dashboard is ‘up and running’ it would be my intention to add further content both to provide a more fuller picture of how the virus continues to effect citizens in Northern Ireland and to reflect user feedback. In doing so I will of course pre-announce any changes to the dashboard content so that users are aware of our plans in advance.

I trust that this meets with your approval.

Yours sincerely,

Dr Eugene Mooney
Senior Statistician, NISRA

Related links:

Letter from Ed Humpherson

Exceptional pre-release access granted for NISRA weekly deaths

Dear Ed,

You will be aware of the recent publicity surrounding deaths due to COVID-19 and whether the published daily figures for the UK, and Northern Ireland, cover just deaths in hospital or also include deaths in care homes. You will also be aware that ONS published deaths split by location for the week ending 3rd April last Tuesday and NISRA plan to publish deaths split by location for the week ending 10th April this morning at 9.30am.

Not only is there high public interest in this information but there is also high potential for misunderstanding (due to the complexity of the definitions for both the weekly deaths published by NISRA and daily figures provided by the PHA). To avoid the risk of misinterpretation, NISRA has taken the decision to brief journalists in advance of the release (via interactive video conferencing) with the data being released to these journalists at 9.20am.

We have applied the public interest test outlined in section (5)(2)(b) of the Pre-Release Access to Official Statistics Order (NI) (the public benefit likely to result from such access (including the timing of such access) significantly outweighs the detriment to public trust in official statistics likely to result from such access) and concluded that in the current circumstances the criterion is met. In particular, we consider that the Value of the statistics will be significantly enhanced at the expense of only a small risk of reduced Trustworthiness.

A list of the organisations represented by the journalists who attended the briefing (see Annex A) has been published as part of the Pre-release Access list, along with the rationale for granting such access.

Yours sincerely,
SIOBHAN CAREY, CBE

ANNEX A: ORGANISATIONS GRANTED EXCEPTIONAL PRE-RELEASE ACCESS

  • BBC
  • UTV
  • Belfast Telegraph
  • Daily Mirror
  • Irish Times
  • Slugger O’Toole
  • Financial Times
  • Reuters
  • Downtown Cool FM
  • Q Radio
  • JPI media
  • Impartial Reporter
  • Belfast Media Group
  • Derry News
  • Causeway Coast and Glens News

Request to temporarily suspend Northern Ireland travel and tourism statistics

Dear Ed,

As you will be aware, one of the sources of data used in the production of NISRA’s Northern Ireland Tourism Statistics is unavailable. The source in question is “The
Household Travel Survey (HTS)” conducted by Central Statistics Office (CSO), which provides an estimate of overnight trips to Northern Ireland by visitors from the
Republic of Ireland (ROI). In the most recent annual and quarterly publications, this estimate accounted for approximately 12% of all overnight trips and 11% of the
overall tourism spend.

CSO has suspended the HTS due to underlying data quality issues (see Annex 1 for further detail), which they are currently in the process of addressing. It was expected that it be will summer 2020 before they would be in a position to determine if the matter has been satisfactorily resolved, thus facilitating the release of the information to NISRA. However, the COVID-19 global pandemic has meant that CSO surveys have been suspended which ultimately means that these indicative timings will be pushed back further. The impact of this is that NISRA is currently unable to produce and publish its National Statistics publications (Northern Ireland annual tourism statistics).

In view of this and as an interim measure, I propose to suspend the publication of the National Statistics until the ROI data become available and move to publish the tourism information that we have in appropriately tailored and caveated releases (information relating to quarters 2 and 3 of 2019 will constitute the first release).

These will focus on the place of origin of visitors (clearly differentiating each market), highlight the information that is missing and include appropriate data coverage and quality guidance for users. The information in the new format releases will account for the vast majority of overnight trips in NI and will include domestic tourism within NI, visitors from GB and visitors from overseas areas.

I consider that this approach is in the public interest and provides the best compromise until normal business can resume. While it has been discussed and
agreed with key stakeholders including tourism policy leads, all of whom are content, I would welcome your views on this proposed way forward.

In addition to the above, the COVID-19 global pandemic will undoubtedly continue to cause major disruption to the tourism sector for the foreseeable future. In
accordance with government guidance, I have taken the decision to temporarily suspend NISRA’s face-to-face survey field operations resulting in the cessation of
interviews conducted through the NI Passenger Survey. Clearly this will have a further impact on NISRA’s NI Tourism Statistics, which will also need to be managed.

Yours sincerely

SIOBHAN CAREY, CBE
Registrar General for Northern Ireland & Chief Executive NISRA

 

Related Links:

Ed Humpherson to Siobhan Carey (March 2020)

Temporary suspension of Northern Ireland travel and tourism statistics

Dear Siobhan,

Thank you for your letter of 2 April 2020 regarding Northern Ireland tourism statistics. Given the ongoing data quality issues with the Household Travel Survey (HTS) conducted by the Central Statistics Office in the Republic of Ireland, I support your decision to temporarily suspend the National Statistics publications which rely on the HTS as one of the main data sources. The Office for Statistics Regulation recognises that there is a strong user need for figures on overnight trips to Northern Ireland. Therefore, I am pleased that you will continue to publish your remaining tourism data which is unaffected by this issue as official statistics in order to provide users with as much information as possible.

We will continue to engage with your team, via our Travel, Transport and Tourism domain, regarding the pre-existing data quality issue with the HTS and the longer-term impact of Covid-19 on Northern Ireland tourism statistics.

Yours sincerely
Ed Humpherson
Director General for Regulation

Related Links:

Siobhan Carey to Ed Humpherson (March 2020)

NISRA request to temporary change to timing of key Northern Irish economic statistics

Dear Ed

You will be aware of developments at the ONS in respect of their plans to publish Market Sensitive Releases at 7.00am.

Historically NISRA has co-ordinated the publications listed below with corresponding releases from the ONS and, if content, it would be our intention for that to continue.

Accordingly, I would be grateful if you could indicate your support for NISRA moving its release times for these publications to 7.00am in line with contingency arrangements being implemented by the ONS. I appreciate that this represents a departure from the requirements of the Code, but consider it prudent as the releases complement one another.

We will of course keep this interim arrangement under review, in conjunction with colleagues at the ONS and in the other devolved authorities, and involve you as appropriate in any further developments.

NISRA Publications that we propose to move to a 7.00am release

Labour Market Report (Monthly)
Quarterly Employment Survey

Yours sincerely,

SIOBHAN CAREY, CBE
Registrar General for Northern Ireland & Chief Executive NISRA

 

Related Links:

Ed Humpherson response to Siobhan Carey (March 2020)

Scottish Government to Ed Humpherson (March 2020)

Welsh Government to Ed Humpherson (March 2020)

Ed Humpherson response to Siobhan Carey: NISRA request to temporary change to timing of key Northern Irish economic statistics

Dear Siobhan,

Thank you for your letter of 26 March 2020. I am happy to confirm the exemption from the Code of Practice for Statistics to permit an earlier release time for NISRA’s market sensitive publications.

Your decision is sensible and proportionate in the circumstances.

Yours sincerely
Ed Humpherson
Director General for Regulation

 

Related Links:

Siobhan Carey to Ed Humpherson (March 2020)

Ed Humpherson to Scottish Government (March 2020)

Ed Humpherson to Welsh Government (March 2020)

Compliance check of Northern Ireland Safe Community Survey

Dear Siobhan

STATISTICS FROM THE NORTHERN IRELAND SAFE COMMUNITY SURVEY

This letter updates our review of compliance with the Code of Practice for Statistics for the Northern Ireland Safe Community Survey. In this part of the work, we focused on data collection and the quality of statistics. We also reviewed the extent to which the team has met the recommendations from our previous compliance check. These compliance checks form part of a series of reviews, the last of which will focus on the reporting of the statistics. Once complete, we will then publish an overall report of our assessment of the statistics from the Northern Ireland Safe Community Survey, which we aim to do in spring 2020. Overall, the package of work will consider trustworthiness, quality and value, all of which are fundamental to supporting public confidence in statistics and, together, form the basis of the Code.

The first compliance check highlighted that the team should demonstrate trustworthiness, ensuring resources are sufficient for the ongoing improvement of quality and to realise the potential value of the statistics. We were pleased to see that the team has recently recruited a new member to provide more resource to develop the statistics further; we look forward to seeing the statistics continue to improve as a result. We were also encouraged to see that the team has published a future work programme, showing transparency around unmet demand. Once the new team member is in post, we look forward to you firming up timescales, which would be helpful so that users can better understand when improvements are going to be finished.

We previously identified the need to understand child victimisation. We welcome the team’s commitment to publish a scoping paper looking into this area. This is an important issue and it was positive to see questions being included in the Young Persons’ Behaviour and Attitudes Survey as part of an effort to understand the nature of child victimisation more fully.

We were assured of the quality of the statistics and commend the team for working closely with our regulators and making changes to the quality and user guide throughout the course of our compliance check. Given the changes in the sample size we were encouraged to see the work that has been done to ensure the integrity of the long-term crime trend. The inclusion of confidence intervals so that users can understand the quality of the estimates more fully when results are published.

We were pleased to see the willingness of the team to engage with users and to collaborate with producers of similar statistics. We welcome that the team is currently holding an internal working group with policy colleagues and has quarterly meetings with producers from the Office for National Statistics and Scottish Government. We recommend that the team should publish findings from its user survey to ensure transparency and to enable further continued improvement to the statistics.

Our Crime and Security domain lead will continue to liaise with staff at the Department of Justice throughout these developments, and in planning the last phase of our work. I look forward to seeing the trustworthiness, quality and value of the statistics continue to improve.

I am copying this letter to Tracy Power, Director of Analysis NISRA; and to Joan Ritchie, Department of Justice.

Yours sincerely

Mark Pont
Assessment Programme Lead

 

Related Links:

Siobhan Carey to Mary Gregory (October 2018)

Mary Gregory to Siobhan Carey (August 2018)