PISA: why context is everything

Around this time, every three years, there is one education story which dominates the hearts, minds and headlines of the sector like no other – PISA, the OECD’s Programme for International Student Assessment, the results of which were released on Tuesday (3 December).

In England, Wales, Northern Ireland and Scotland, this survey of 15-year-olds’ educational achievement was carried out by NFER on behalf of the respective governments. Over 500 schools participated in PISA 2012 in the UK, with over 12,500 pupils taking part, and we are very grateful for the support from teachers, headteachers and pupils who participated – without their support and cooperation we could not have run the survey.

Rankings have been the subject of political and media debate following release, but our focus when looking at countries that perform better, similarly, or less well, will be those countries that are statistically significantly different rather than those with a score that puts them higher or lower. Looking at countries in this way means that we can be sure that if the test was run again, these countries would be highly likely to still be in the group ahead of us, or the group behind us. In addition, the number of participating countries has increased since the early rounds of PISA and some countries have dropped in or out of the survey in each round, so comparing ranks from previous rounds of the survey can mean that we are not comparing like with like.

The information from pupil assessments are only part of the story – it is the contextual information from the survey that adds real depth to the PISA data. The information pupils provide in questionnaires tells us about their background, their experience of school, their relationships with their teachers and, in the case of PISA 2012, their attitudes towards mathematics. Headteachers also tell us about their school, how it is managed and issues they are facing in terms of staffing and resources.

This is where we see some very positive stories for England, whose pupils reported:

• a high amount of control over their ability to succeed in mathematics and a high level of conscientiousness towards learning mathematics
• a greater level of conscientiousness and perseverance for mathematics tasks than the OECD average.
• confidence in their ability to perform mathematics tasks, and low anxiety about the subject
• a higher level of support from their mathematics teachers than that found for the OECD, on average.

Pupils in England are also better able to overcome disadvantage and achieve scores higher than predicted by their background when compared with some other OECD countries.

There is, therefore, a lot that we can learn from PISA, particularly in conjunction with results from other national and international research. In the coming months, we will be particularly keen to make links between results in PISA, results from the IEA Trends in Mathematics and Science Survey (TIMSS) and the OECD Survey of Adult Skills, in which England and Northern Ireland have recently participated.