Chapter navigation
Annex A: QA Matrix
A0: No assurance
Operational context & administrative data
• Context and data collection by supplier not investigated, managed or documented
Communication with data supply
- No communication
QA principles, standards and checks
- No description of suppliers’ QA procedures
Producer’s QA investigations & documentation
- No description of own QA checks
A1: Basic assurance
Statistical producer has reviewed and published a summary of the administrative data QA arrangements
Operational context & administrative data
Consider the following types of activities:
- Producer has provided users with an outline of the administrative data collection process,
- Illustrated the administrative data collection process and main stages,
- Outlined the operational context,
- Identified actions taken to minimise risks to quality,
- Identified and summarised the implications for accuracy and quality of data, including the impact of any changes in the context or data collection
Communication with data supply
Consider the following types of activities:
- Producer has outlined the data provision arrangements including:
- annual statement of needs,
- timing and format of data supply,
- coordination of data sign-off from supplier,
- Fed back identified errors to data suppliers and recorded their response,
- Sought the views of statistics users about the data and resolved any quality issues reported
QA principles, standards and checks
Consider the following types of activities:
- Producer has knowledge of suppliers’ QA checks and published a brief description,
- Identified whether audits are conducted on the admin data (such as internal or operational audits, external audit such as by regulator),
- Described the implications for the statistics
Producer’s QA investigations & documentation
Consider the following types of activities:
- Producer has established regular QA checks on the received admin data,
- Published a description of its own QA checks on the admin data,
- Outlined general approach and findings,
- Identified the strengths and limitations of the admin data,
- Explained the likely degree of risk to the quality of the admin data
A2: Enhanced assurance
Statistical producer has evaluated the administrative data QA arrangements and published a fuller description of the assurance
Operational context & administrative data
Consider the following types of activities:
- Producer has provided users with a fuller description of the operational context and administrative data collection arrangements, eg:
- a process map detailing data collection processes,
- explanations for classifications,
- Identified and summarised potential sources of bias and error in administrative system,
- Identified and described safeguards taken to minimise risks to data quality,
- Provided a detailed description of the implications for accuracy and quality of data, including the impact of any changes in the context or collection arrangements
Communication with data supply
Consider the following types of activities:
- Producer has agreed and documented:
- data requirements for statistical purposes,
- legal basis for data supply,
- data transfer process,
- arrangements for data protection,
- sign-off arrangements by data suppliers,
- Established an effective mode of communication with contacts (eg with data collector and supplier bodies, IT systems, operational/policy officials) to discuss the ongoing statistical needs in the data collection system and quality of supplied data
QA principles, standards and checks
Consider the following types of activities:
- Producer has provided a fuller description of the main QA principles, quality indicators and checks used by the data suppliers, management or governance groups in data quality management,
- Described the role of audit of the admin data within the collection and operational settings,
- Described the implications for the statistics for the quality issues identified by data supply bodies and regulators
Producer’s QA investigations & documentation
Consider the following types of activities:
- Producer has provided a fuller description of its own QA checks on the admin data,
- Detailed the general approach and
- Described the role of relevant information findings for specific quality indicators,
- Identified the strengths and limitations of the admin data,
- Explained the likely degree of risk to the quality of the admin data
A3: Comprehensive assurance
Statistical producer has investigated the administrative data QA arrangements, identified the results of independent audit, and published detailed documentation about the assurance and audit
Operational context & administrative data
Consider the following types of activities:
- Producer has provided users with a detailed description of the administrative system and operational context:
- explained why the data are collected, who by and how,
- identified differences across areas in the collection and recording of the data,
- identified issues for individual data items, such as whether objective or based on subjective recording, missing and/or imputed, poorly recorded,
- Identified issues in design and definition of performance measurements and targets,
- Identified and described potential sources of bias and error in the administrative system,
- Identified and explained any safeguards used to minimise the risks to data quality,
- Provided detailed description of the implications for a quality of the data, including the impact of any changes in the context or data collection
Communication with data supply
Consider the following types of activities:
- Producer has established/maintained collaborative relationships,
- Has a written agreement specifying:
- roles and responsibilities,
- legal basis for data supply,
- data supply and transfer process,
- security and confidentiality protection,
- schedule for data provision,
- content specification,
- Used a change management process,
- Regularly communicated with the data collector and supplier bodies, IT systems, operational/policy officials eg newsletters, conferences, attending data supplier/IT system group meetings,
- Regularly engaged statistics users, resolved any reported quality issues, and held user group conferences
QA principles, standards and checks
Consider the following types of activities:
- Producer has described the data suppliers’ principles, standards (quality indicators) and quality checks,
- Reviewed quality reports for the received data (such as input quality indicators for data accuracy, coverage and completeness),
- Identified and documented the findings of investigations and audits conducted on the admin data and associated targets (such as internal and operational audits, and external audits by regulators and professional bodies),
- Described the implications for the statistics and determined whether the data continue to be satisfactory for official statistics purposes
Producer’s QA investigations & documentation
Consider the following types of activities:
- Producer has provided a detailed description of its own QA checks on the admin data (including validation, sense and consistency checks),
- Given quantitative (and where appropriate qualitative) metrics for specific quality indicators (such as input, process and output quality metrics),
- Undertaken comparisons with other relevant data sources (such as survey or other admin data),
- Identified possible distortive effects of performance measurements and targets,
- Identified the strengths and limitations of the admin data and any constraints on use for producing statistics,
- Explained the likely degree of risk to the quality of the admin data