Designing a new assessment system: messages for policymakers
21 July 2010
The current assessment system is under review, providing a genuine opportunity to reconsider what assessments are used for, what information the government, policy makers and schools need, and to select the best way forward to respond to these needs.
Based on its long-term track record in assessment research and test development, NFER is advising policy makers that when designing a new assessment system they should define what assessment will be used for, select the model/s of assessment that best meet the defined uses, build teachers’ expertise to use the assessments most effectively, base any accountability on a broad range of measures, and allow time for a new system to bed in before making further changes.
1 ) Define the purposes
Being clear about the purposes for which assessment results will be used is essential. One model of assessment cannot effectively meet all possible purposes, and a new system could use a combination of models to meet the different objectives.
2) Select the assessment model/s that best meet the purposes
Different models of assessment have different strengths and weaknesses. When introducing a new system the models should be selected to match the requirements, for example teacher assessment should provide the most detailed information about an individual child on a day to day basis, whereas externally set tests used with a representative sample of pupils may provide the most useful information for measuring standards over time. A revised system could use a combination of several models of assessment.
External tests: Well-developed, external tests used with a whole cohort of pupils form a valuable and cost-effective strand of any assessment system, because they ensure that the same standard is being applied across different schools, provide reliable results about each pupil and ensure public confidence in the system.
Teacher assessment: As with external tests, teacher assessment plays a useful role in any assessment system because it provides specific feedback on the progress of individual pupils to feed into teaching and learning, informs conversations with parents and pupils about individual strengths and weaknesses and complements test results.
National sample tests: Using tests with a sample of pupils, rather than a whole cohort, is an efficient and cost effective way of monitoring national standards over time in a low stakes way. As all pupils do not answer all the same questions, it is possible to cover a much broader range of the curriculum. However, a national sample test cannot be used for collecting reliable information about the performance of pupils or schools.
International surveys: the UK aspires to have a world class education system and we need to know where we stand in relation to our economic competitors. Participation in international surveys such as PIRLS, TIMSS and PISA can provide the necessary comparison.
3) Build teachers’ expertise
For any system to be effective, teachers must understand the purposes of the key components. It will be important to build the expertise of teachers so that they have a rounded understanding of assessment, and can make best use of different forms of evidence.
4) Base accountability on a range of measures
There is a consensus that accountability information about schools should be available for the system and its current and potential users. However, the use of any form of assessment results for high stakes accountability purposes is likely to distort how the assessment is used. If test results are used to judge schools, this could lead to an over-emphasis on success in the tests at the expense of broader learning. If teacher assessment is used, this puts the teachers in the position of making judgements against which they themselves may be judged. Accountability systems need to be based on a broad range of measures, to ensure that assessment does not narrow the focus of teaching and learning.
5) Allow time for a new system to bed down
Comparisons over time are difficult. Where the assessment instruments used to measure change are themselves changed it becomes impossible to draw conclusions about whether standards have improved. Once decisions have been made about the purposes of a refined assessment system and the models that will be used to provide the necessary information, time must be allowed for the new system to bed down and for reliable data to be established and collected.
NFER’s Sarah Maughan said: “The current review of the testing system provides a real opportunity to make improvements. The existing system served us well for many years, but assessment results are being used for ever more purposes. It is time to consider whether the tests that we use provide the best means of answering the questions that we have about the learning of children in our schools.”