Quality assurance (QA) of administrative data is more than simply checking that the figures add up. It is an ongoing, iterative process to assess the data’s fitness to serve their purpose.
QA should cover the entire statistical production process. It involves monitoring data quality over time and reporting on variations in quality.
Post-collection QA methods, such as data validation, are an important part of the quality assurance process, but can be of limited value if the underlying data are of poor quality. They only tell you part of the story.
It is important to check the underlying data from administrative systems before the data are extracted for supply into the statistical production process.
As with survey data, producers need to:
- investigate the administrative data to identify errors, uncertainty and potential bias in the data
- make efforts to understand why these errors occur and to manage or, if possible, eliminate them
- communicate to users how these could affect the statistics and their use
They represent three types of actions for assuring the suitability of the administrative data and in documenting the findings: Investigate: Manage: Communicate
Quality Management Actions
- Data suppliers’ own QA arrangements
- Results of external audit of the admin data
- Areas of uncertainty and bias
- Distortive effects of targets and performance management regimes
- identify any coverage issues and potential sources of bias in the data collection and supply process
- the types of checks carried out by data collectors and suppliers, as well as the operational circumstances in which the data are produced
Manage – such as:
- Cooperative relationship with suppliers, IT and operational and policy officials
- Guidance information on data requirements
- QA checks and corroboration against other sources
- Producers should also manage their relationships with suppliers by establishing clear processes for data provision and for managing change
- Producers should also maintain regular quality assurance checks of the data and use other data sources where possible to corroborate their findings
Communicate – such as:
- Description of data collection process
- Regular dialogue with suppliers and providers
- Document quality guidelines for each set of statistics
- Description of errors and biases and their effects on the statistics
- Communicate with users
- Producers should communicare effectively with their data suppliers and others to ensure users are provided with clear explanations of the strengths and limitations of the data
- Producers should also work closely with other statistical producers using the administrative data to ensure a common understanding of any quality issues
Iterative and ongoing – continue to comply with the Code
These practices are continuous and iterative, reflecting the ongoing use of the data and the dynamic nature of operational environments.
This review of administrative data should not be a one-off event, but instead it is a process that requires repeated evaluation to understand the implications of changes and allow for the ongoing monitoring of the data quality.
If flaws are found in administrative data, statistical producers should:
- evaluate the likely impact on the statistics
- establish whether the issue can be resolved, or whether there is any other action they can take to mitigate the risks
- determine whether the level of impact is such that users should be notified
It is recognised that often issues discovered through quality assurance are complex and will require time and staffing and financial resources to address. Statistical producers are required to maintain ongoing compliance with the Code of Practice.
If, in the course of these investigations, a statistical producer discovers a systemic issue in the administrative data that has a substantial adverse impact on the statistics, we encourage the statistical Head of Profession to contact the Authority to discuss appropriate action.