Appendix
Results of Intercoder Reliability Analysis
Table 5: The agreement between two independent coders
| Intercoder Reliability | Provide an evidence base for public policy decision-making | Provide an evidence base for public service delivery | Provide an evidence base for decisions which are likely to significantly benefit the UK economy, society or quality of life of people in the UK | To replicate, validate or challenge official statistics | To replicate, validate or challenge existing research | To improve the quality, coverage or presentation of existing statistical information | To significantly extend understanding of social or economic trends or events by improving knowledge or challenging widely accepted analyses |
|---|---|---|---|---|---|---|---|
| Application | 2 6 9 16 | 2 6 9 16 | 2 6 9 16 | 2 6 9 16 | 2 6 9 16 | 2 6 9 16 | 2 6 9 16 |
| Agreement | ✓ ✓ ✓ ✓ | ✓ ✓ ✓ ✓ | - ✓ ✓ - | - - ✓ ✓ | ✓ ✓ - ✓ | ✓ ✓ - ✓ | ✓ - ✓ ✓ |
Each check mark represents agreement on an application. Four applications were coded by both coders, therefore four check marks in a row represents full agreement on whether or not that public benefit has been referenced in the applications. Dashes indicate that there was not agreement, therefore one coder identified a public benefit but the other coder did not.
Quantitative Analysis: Stage Two
Are there patterns in how often public benefits are referred to comparing types of applicant?
Table 6: The number of public benefits referred to by types of applicant
| Provide an evidence base for public policy decision-making | Provide an evidence base for public service delivery | Provide an evidence base for decisions which are likely to significantly benefit the UK economy, society or quality of life of people in the UK | To replicate, validate or challenge official statistics | To replicate, validate or challenge existing research | To improve the quality, coverage or presentation of existing statistical information | To significantly extend understanding of social or economic trends or events by improving knowledge or challenging widely accepted analyses | |
|---|---|---|---|---|---|---|---|
| Academia (8) RAP (6) NSDEC (2) | 6 6 0 | 5 3 2 | 6 6 0 | 2 1 1 | 4 2 2 | 1 0 1 | 3 2 1 |
| Government (11) RAP (2) NSDEC (9) | 7 2 5 | 7 2 5 | 5 2 3 | 2 0 2 | 2 1 1 | 10 1 9 | 4 0 4 |
| Organisation (11) RAP (4) NSDEC (7) | 11 4 7 | 6 0 6 | 5 2 3 | 1 0 1 | 1 0 1 | 5 0 5 | 7 1 6 |
Are there patterns in how often public benefits are referred to by theme of application?
Table 7: The number of public benefits referred to by theme of application
| Provide an evidence base for public policy decision-making | Provide an evidence base for public service delivery | Provide an evidence base for decisions which are likely to significantly benefit the UK economy, society, or quality of life of people in the UK | To replicate, validate or challenge official statistics | To replicate, validate or challenge existing research | To improve the quality, coverage, or presentation of existing statistical information | To significantly extend understanding of social or economic trends or events by improving knowledge or challenging widely accepted analyses | |
|---|---|---|---|---|---|---|---|
| Business (7) RAP(4) NSDEC (3) | 6 4 2 | 4 2 2 | 4 3 1 | 1 0 1 | 1 0 1 | 4 1 3 | 2 1 1 |
| Children (6) RAP (2) NSDEC (4) | 5 2 3 | 3 1 2 | 2 2 0 | 3 1 2 | 2 1 1 | 4 0 4 | 2 1 1 |
| Environment (2) RAP (0) NSDEC (2) | 2 0 2 | 2 0 2 | 1 0 1 | 0 0 0 | 0 0 0 | 1 0 1 | 2 0 2 |
| Health (7) RAP (2) NSDEC (5) | 3 2 1 | 5 0 5 | 5 2 3 | 1 0 1 | 2 0 2 | 4 0 4 | 4 0 4 |
| Population (8) RAP (4) NSDEC (4) | 8 4 4 | 5 2 3 | 4 3 1 | 0 0 0 | 2 2 0 | 3 0 3 | 4 1 3 |