DRS marker reliability research

Our work used empirical analysis to enhance a tool designed by DRS to ensure quality marking within high stakes examinations. Our analysis has enhanced the potential for the tool to efficiently identify poor markers and remove them from the marking process.

The research was concerned with improving the processes associated with percentage double marking; that is where a marker’s work is periodically checked be peers. Analysis quantified consistency between markers as well as identifying those factors that were associated with erratic marking. From this starting point an optimal process for percentage double marking was designed dealing with issues such as how often checks should be performed and what level of discrepancy between markers should be considered acceptable.

Further details on e-marker.

Presentation given in November 2010 to the 11th Annual Conference
of the Association for Educational Assessment, Oslo.

Statistical analysis home