Statistical Methods Used for the Analysis of National Monitoring Surveys
14 May 2010
Proceedings from the technical Seminar on statistical methods used for the analysis of national monitoring surveys
Since October 2008, national assessment policy in England has been in a state of change. At that time, the Secretary of State for Children, Schools and Families announced the end of national testing at Key Stage 3, a system that had been in place since the mid-1990s. This announcement was followed by the deliberations of an Expert Group to consider revised arrangements, which reported in February 2009, a report to which the Secretary of State gave a formal response soon after.
A central element of the new assessment landscape is the intention to establish a national monitoring survey. Originally, the introduction of such a survey had the intention of monitoring standards only at the end of Key Stage 3, where there were no longer national tests. Later, a national monitoring survey for science at the end of Key Stage 2 also became policy, in response to the Expert Group recommendation of discontinuing the science tests at this key stage. The group further recommended that national monitoring should be linked to existing international surveys.
Previous expert seminars held by the NFER during 2009 considered some of the implications of these changes. In January 2009, researchers and policy-makers discussed national assessment arrangements for Key Stage 3 at a seminar hosted jointly by NFER, Cambridge Assessment and The Nuffield Foundation. In June, 2009, NFER and the Chartered Institute of Educational Assessors held a seminar on methods for ensuring reliability of teacher assessments in the new context.
The complexities of planning a national monitoring survey were introduced at the January seminar, and it was clear at this point that there were many detailed technical issues in need of examination. This latest seminar was set up by NFER to respond to that need by stimulating discussion about the different statistical techniques that might be useful for the proposed national monitoring surveys.
In particular the seminar sought to question: issues and solutions involved in sampling; possibilities for links to the TIMSS international survey; and statistical techniques available for analysis.
Contributions were made by Simon Rutt, Dougal Hutchison, Graham Ruddock, Ben Styles and Tom Benton (NFER), Harvey Goldstein (University of Bristol) and Sandra Johnson (Assessment-Europe). The seminar was steered by Sarah Maughan and Chris Whetton (NFER).
This report summarises the presentations and discussions.