Another shake-up of school league tables: how should we measure and hold schools accountable for the progress of their pupils?
The new school progress measures continue to ignore the very large socioeconomic and demographic differences between schools which also drive results.
About the research
Every year the Government publish ‘school league tables’ holding secondary schools accountable for the academic progress and GCSE results of their pupils. Schools deemed underperforming face sanctions, increased scrutiny, potential takeover by neighbouring schools or even closure. However, what are these school progress measures actually measuring? Our work compares, contrasts and critiques the three main Government school progress measures published over the last decade: Contextual Value-Added (CVA) published from 2006- 2010; Expected Progress (EP) published from 2011-2015; and Progress 8 (P8) published from 2016 onwards.
CVA acknowledged that poor pupils make less progress than their richer peers and adjusted for this to allow fairer and more meaningful comparisons between schools. EP was an ideological shift away from CVA whereby the Government declared all pupils must make the same progress, irrespective of their prior attainment and socioeconomic circumstances. P8 represents a partial return to CVA in that it again recognises that pupils with higher prior attainment make more progress, but it continues to ignore the very large socioeconomic and demographic differences between schools which also drive results.
These progress measures have become increasingly highstakes as the Government’s definition of underperforming shifted from concentrating on GCSE results to focusing on the progress schools make with their pupils. In this year’s tables, P8 scores alone will be used in these decisions. Schools will be judged underperforming if, on average, their pupils achieve half a grade less in their GCSE examinations than other pupils nationally with the same KS2 achievements.
- Socioeconomic and demographic differences between schools are crucial in any measurement of school performance, but these are not included in P8 measures.
- Whether a school is deemed underperforming should not be based solely on its P8 score; rather a much wider range of information should be taken into account.
- The high-stakes nature of P8 places pressure on schools to game the league tables and may create perverse incentives and create unintended consequences just as its predecessors CVA and EP did.
- School progress measures are best used as tools for school selfevaluation and as a first step towards identifying the policies and practices which can make schools successful.
- If school progress measures are to be used as part of school inspection systems, they would be better used as monitoring and screening devices to identify schools performing unexpectedly poorly for the purpose of careful and sensitive further investigation.
- Design and interpretation differences between CVA, EP and P8 lead to fundamentally different school rankings with many schools moving up or down the national tables by hundreds or even thousands of places with the introduction of each new measure.
- The move from CVA to EP greatly inflated the school league table positions of schools with higher prior-attaining pupils, especially grammar schools, while the move from EP to P8 pulls these schools substantially back down the league tables.
- That some schools misused CVA to set differential GCSE target grades for pupils with different socioeconomic and ethnic status reflects the perverse incentives that arise when too much emphasis is placed on test scores when holding schools to account.
- EP suffered from fundamental design flaws including being biased in favour of high prior attaining pupils, perversely incentivising schools to focus on pupils making borderline progress, setting otherwise identical pupils different GCSE target grades, ignoring the substantial uncertainty in predicting school performances.
- An important explanation for the school differences in P8 will be differences in the social and demographic composition of schools’ intakes; P8, like EP, ignores all pupil background characteristics other than their prior attainment.
The full article upon which this policy brief is based: Leckie, G., & Goldstein, H. The evolution of school league tables in England 1992-2016: ‘contextual value-added’, ‘expected progress’ and ‘progress 8’. British Educational Research Journal. Forthcoming.
A working paper version is available online: Leckie, G., & Goldstein, H. (2016). The evolution of school league tables in England 1992-2016: ‘contextual value-added’, ‘expected progress’ and ‘progress 8’. Bristol Working Paper in Education Series. Working Paper, 2/16.
This work was funded by an ESRC Future Research Leaders grant (ES/ K000950/1). Further details can be found on the grant website.
Policy Briefing 41: January 2017
Contact the researchers
Dr George Leckie, Graduate School
of Education, University of Bristol:
Professor Harvey Goldstein,
Graduate School of Education,
University of Bristol:
Dr George Leckie, Reader in Social Statistics, University of Bristol
Professor Harvey Goldstein, Professor in Social Statistics, University of Bristol
- The success of ‘naming and shaming’: league tables have a positive impact on performance of schools and hospitals
- Bursaries are an important component of widening participation in Higher Education and their funding should be maintained
- Widening access to university: Step change needed to support talented learners from disadvantaged backgrounds