Response to DAERA Action Plan for Statistics from the Northern Ireland June Agricultural Census

Dear Máire

STATISTICS FROM THE NORTHERN IRELAND JUNE AGRICULTURAL CENSUS

Many thanks for sharing your action plan for the statistics from the Northern Ireland June Agricultural Census, in response our compliance check. We welcome your commitment to enhancing the quality and value of these statistics and look forward to seeing the results of the plan.

Your suggested approach is well thought-out and addresses all our recommendations. We are pleased that you have already implemented some of our recommendations, and that, where possible, you are consulting others about the changes. For example, you consulted statisticians in the devolved administrations and the Good Practice Team about discontinuing the preliminary and final results bulletins, and you intend to consult users and stakeholders about re-designing the statistics bulletin.

As you seek to make improvements, we encourage you to have high ambitions for data validation and your statistical outputs. For instance, it might be helpful to review your data validation checks as part of the process of preparing the upcoming report (expected to be published early 2020). Similarly, you should ensure that you publish data tables in an Excel or open data format alongside the statistics. This will not only improve accessibility of the data, it will also encourage its re-use, and is a simple yet effective way of enhancing the value of the data for existing and potential users.

We would be happy to discuss these points with you.

Yours sincerely

Mark Pont
Assessment Programme Lead

 

Related Links:

Máire Brolly to Mark Pont (June 2019)

Mark Pont to Conor McCormack (March 2019)

Response to Compliance Check of Statistics from the Northern Ireland June Agricultural Census

Dear Mark,

Compliance check of statistics from the Northern Ireland June Agricultural Census

Thank you for your letter of 28 March 2019 regarding the compliance check of statistics from the Northern Ireland June Agricultural Census and for confirming the publications’ continued designation of these statistics as National Statistics.

DAERA Statisticians have found the recent engagement with the OSR team to be extremely valuable in highlighting the strengths and areas of improvement for the main June Census publication, and the associated results releases. The DAERA Statistics Charter, our rich historical data, and our commitment to the ongoing documentation of quality assurance processes were demonstrated as strengths, as outlined in your letter.

We would like to thank the OSR team for taking the time to review the publications in detail, as well as providing advice on request.

The compliance check letter recommended a number of improvements and as a result, DAERA statisticians have prepared an action plan (see attached), which sets out how we intend to implement these recommendations. In doing so, we will ensure that the releases associated with the June Agricultural Census meet the highest standards of trustworthiness, quality and public value, as well as maximising the accessibility, relevance, clarity and insight of the statistics. Again, I’d like to express my thanks for this impartial and constructive review of the one of the Department’s key data collections.

Please do not hesitate to contact me if you require any further detail with regards to the response provided.

Yours sincerely,

Máire Brolly
Statistics and Analytical Services Branch

 

Related Links:

Mark Pont to Máire Brolly (July 2019)

Mark Pont to Conor McCormack (March 2019)

Compliance check of statistics from the Northern Ireland June Agricultural Census

Dear Conor

STATISTICS FROM THE NORTHERN IRELAND JUNE AGRICULTURAL CENSUS

As you are aware, we recently completed a compliance check of statistics from the Northern Ireland June Agricultural Census. I am pleased to confirm the continued designation of these statistics as National Statistics, subject to your team addressing the areas highlighted below. We would like you to write to us in the coming months, setting out how you intend to enhance the value of the statistics following this review, so that the changes can be implemented by the next publication cycle. Our Agriculture and Environment Domain Lead, Job de Roij, will continue to engage with your team on progress.

We initiated this review to better understand the state of agriculture statistics in Northern Ireland and due to the time elapsed since the previous assessment. To gain a perspective across the Devolved Administrations, we also carried out reviews of statistics from the Scottish June Agricultural Census, produced by the Scottish Government, and statistics from the Welsh Agriculture and Horticulture Survey, produced by the Welsh Government.

Statistics from the Northern Ireland June Agricultural Census are published as a suite of bulletins: the preliminary results are published in August, the final results are published in November, and the main report is published in January. This compliance check focused largely on the main report, but our findings are also relevant to the preliminary and final results bulletins. Similarly, we encourage you to reflect on the findings from this compliance check when producing statistics from the December Agricultural Survey, a survey with similarities to the June Agricultural Census.

We found several examples of how the statistics demonstrate Trustworthiness and Quality. We welcome DAERA’s Statistics Charter, which sets out the ways in which National Statistics are produced by DAERA; it is transparent about revisions and alignment to the Code of Practice for Statistics. The main report clearly explains the statistics team’s approach to data governance, covering topics such as statistical disclosure control and access to survey micro-data.

The team maximises use of available administrative data sources. Currently, the team uses administrative data on cattle, poultry, pigs, and, for the December Agricultural Survey, sheep. We are encouraged by the team’s recent progress on documenting the quality assurance arrangements of these administrative data; for instance, the team is working on a quality report for the cattle data (from the Animal and Public Health Information System (APHIS)) using our Quality Assurance of Administrative Data toolkit. We suggest sharing insight and best practice with the Welsh Government and Scottish Government, who are also working on administrative data quality reports for their June Agricultural Censuses.

The main report provides a helpful overview of changes to the survey methods and contains detailed information on the census coverage. We are pleased that the team has managed to maintain a high response rate (of over 70 per cent), when other surveys continue to experience a decline in response rate, and we welcome the team’s direct and proactive engagement with farmers to achieve this.

However, we have concerns about the value of the statistics, specifically, the clarity and insight, accessibility and relevance to users of these statistics. The bulletins should tell a story about what is happening in the Northern Ireland farming sector for a range of audiences, but it is difficult to find this in the narrative of the main report. We would encourage you to move away from lengthy, descriptive commentary and consider how you can more effectively provide insights for users. It might be helpful to consider the following:

  • Help users answer key questions by emphasising the key findings at the start of the bulletin rather than background information and methodology.
  • Aid users’ interpretation of the statistics by adding clear commentary that provides context and explains the reasons for changes in figures and trends.
  • Help users understand the policy context by highlighting what policies the data and statistics inform, at the Northern Ireland level, UK level and international (EU) level.
  • Communicate the statistics to the widest possible audience, including non-expert readers, through minimising use of technical language.
  • Support users’ understanding of the statistics by improving data tables and visualisations, so that they convey key messages quickly and effectively, in line with the Government Statistical Service (GSS) Good Practice Team’s guidance.

The statistics team told us about the task and finish group which has been set up to look at improving data visualisations across NISRA. We welcome this initiative and look forward to seeing the outcomes of the project. In the meantime, we recommend that the team contacts the GSS Good Practice Team for advice on enhancing the structure, design and accessibility of the bulletin.

The main report is a traditional statistical bulletin, with information on quality and methods, whereas the preliminary and final results bulletins resemble press releases, with limited contextual information. We urge your team to consider the value of these separate releases: is there a clear user need for the preliminary and final results bulletins, and how do they inform decisions? It is difficult to find these related statistics on the DAERA website. To improve accessibility of the suite of statistical bulletins, we encourage you to cross-reference the bulletins and add links on the bulletin landing pages of the DAERA website where possible.

We think there is scope to publish the main report earlier. We encourage the team to review the processes involved in preparing the bulletin and improve efficiency by minimising the need for manual copying and validation. To encourage re-use of the data, the statistics team should consider making the data tables available in an Excel or open data format; currently, these are embedded in the PDF version of the main report, which limits their accessibility.

On quality, to help users understand the methods used, we encourage your team to include information on the calculation of estimates from the survey data and imputation, and to discuss their limitations. It would also be helpful to include a description of the survey design and how survey data are quality assured. We encourage your team to add confidence limits to survey estimates to highlight the uncertainty around the estimates. The data quality report for the APHIS data will help users understand quality assurance arrangements, and the team should produce similar reports for the other administrative data sources.

We encourage your team to continue to collaborate with, and seek input from, statisticians in the Scottish Government, Welsh Government and Defra, to ensure a consistent and coherent approach to producing statistics from the June Agricultural Census across the UK.

Please let me know if there is any aspect of this letter that you wish to discuss.

I am copying this letter to: Siobhan Carey, Chief Executive and Registrar General, Northern Ireland Statistics Research Agency (NISRA); Ruth Fulton, Head of NISRA Statistical Support Branch, NISRA; Jay Calvert, statistician, DAERA; and Des Muldoon, statistician, DAERA.

Yours sincerely

Mark Pont
Assessment Programme Lead