National Reference Test Results Digest 2019
22 August 2019
Ofqual has introduced the National Reference Test (NRT) to provide additional information to support the awarding of GCSEs. NFER developed and administers the NRT under contract to Ofqual.
In February/March 2019 a random sample of around 13,000 year 11 students from over 300 schools took the NRT. The first live NRT, taken in 2017, was benchmarked against the first awards of the reformed GCSEs in English language and maths. The 2018 NRT provided the first opportunity to compare the performance of students across two years. Before 2018, Ofqual had decided that test results would not be used in awarding GCSEs until 2019 at the earliest. 2019 is therefore the first year when Ofqual will consider the results of the NRT alongside other evidence to make decisions on grade standards for GCSE English and maths. Results are reported as expected percentages of students achieving those grades (and above) based on changes in performance on the NRT.
Given that in summer 2019 there are multiple years’ data, several statistical comparisons could be made (e.g. comparing 2017 and 2018, 2018 and 2019, or 2017 and 2019) across the three grade boundaries (7, 5 and 4). Where multiple comparisons are made, this must be taken into account when judging statistical significance, so an adjustment is made to prevent the likelihood of finding a statistically significant difference purely by chance from increasing due to the number of comparisons. For the NRT a Bonferroni adjustment is made to account for there being nine comparisons for each subject. This is a relatively conservative approach and, having reflected on this, Ofqual refined the comparisons they consider to be most pertinent in 2019 and have adjusted for three comparisons in the results reported in their annual statement. This accounts for some differences in the level of statistical significance of the results reported by NFER and Ofqual.
More information about the NRT can be found in the background report National Reference Test Information.
Key Findings
The NRT Results Digest 2019 includes details of the sample, the analyses conducted on the test data and how the tests performed overall. The main findings are:
- Overall, for both English and maths, the NRT tests functioned well, just as they had in 2017 and 2018.
- The expected percentage of students in English at grade 4 and above in 2019 is significantly lower than in 2017, the baseline year, at the 5% level of significance, but not at the 1% level of significance. The differences for grade 5 and above and grade 7 and above were lower than in 2017 but they were not significantly different. Nor were there any significant differences from 2017 to 2018 or from 2018 to 2019 for any of the three grades.
- In maths, the data indicates that the expected percentage of students at grade 7 and above is significantly greater than in 2017 at the 5% level of significance, but not at the 1% level of significance. The differences for grade 4 and above and grade 5 and above from 2017 to 2019 were not significantly different. Nor were there any significant differences from 2017 to 2018 or from 2018 to 2019 for any of the three grades.