League tables ‘punish and reward wrong schools’, new analysis shows
24 January 2019
Secondary school league tables, published today [24 January], ‘punish and reward the wrong schools’ by failing to take pupil background into account, according to new research from the University of Bristol.
Once factors such as pupil ethnicity, deprivation, and special educational needs are taken into account, a fifth of schools saw their national league table position change by over 500 places.
Critically, 40 percent of schools currently judged to be 'underperforming' would no longer fall into this category.
These are the striking findings from a new study by Dr George Leckie and Professor Harvey Goldstein, from the University of Bristol’s School of Education, who analysed the 2016 data from all 3,098 state-maintained secondary schools in England.
They looked at 'Progress 8', the headline measure used by the Department of Education to assess progress made by pupils during their time at secondary school. It was introduced in 2016 and compares GCSE results to Key Stage 2 test results, which the Government argues takes prior attainment into account when judging progress.
Critics have previously argued that the measure is too simplistic and punishes schools with a higher proportion of disadvantaged pupils. This may lead to schools 'gaming' the system by not admitting pupils, such as those with special educational needs, as they are less likely to make progress during their secondary education.
To present a more informative picture of school performance, the researchers incorporated Progress 8 results with information on pupil background - age, gender, ethnicity, whether pupils speak English as an additional language, special educational needs, free school meal eligibility, and deprivation.
Dr George Leckie, a Reader in Social Statistics, said: "By factoring in vital information about a pupil’s background, we have seen a dramatic change in the league tables. This leads to very different interpretations and conclusions about education in England.
"It seems clear from our results that the higher the proportion of disadvantaged pupils in a school, the more it will effectively be punished for the national underperformance of these pupil groups. On the flip side, other schools are rewarded merely for teaching educationally advantaged intakes."
Under the current Progress 8 measure, researchers say there's a risk that truly effective schools in disadvantaged areas are going undetected.
Eligibility for free school meals and a pupil’s ethnicity were shown to be the most important characteristics to consider. Pupils eligible for free school meals (27 percent of all pupils) make nearly half a grade less progress per subject than their peers, while Chinese pupils on average make more progress than other ethnic groups.
For example, the high average pupil progress seen in London – sometimes known as the 'London effect' - more than halves when data is adjusted for pupil background. This is principally due to the high proportion of particular ethnic groups who tend to make good progress at secondary school.
In contrast, the low average pupil progress seen in the North East increases substantially after the data is adjusted to account for the disproportionately high numbers of deprived pupils taught in this region.
Other dramatic changes are seen for grammar schools and faith schools, whose high average pupil progress reduces substantially once the educationally advantaged nature of their pupils is taken into account.
In contrast, the low average pupil progress seen in sponsored academies increases once the disadvantaged nature of their pupils is recognised.
The researchers are calling for the Government to publish and explain a pupil background adjusted Progress 8 measure side-by-side with the current measure to present a more informative picture of schools' performances.
Dr Leckie added: "There are many well-known statistical issues with all attempts to measure school performance. While Progress 8 currently uses prior attainment at Key Stage 2 as an important predictor of progress at secondary school, we’ve shown that adding seven pupil characteristics improves these predictions and changes interpretations.
"The Department for Education's decision to ignore pupil background when comparing schools is in stark contrast to both the academic literature and feedback from teachers. At the moment, the simplistic nature of Progress 8 as a measure places too much emphasis on schools rather than the Government or society as a whole.
"We hope our research will encourage the Government to provide users with greater insights as to why schools achieve the scores they do."
The research was funded by the Economic and Social Research Council (ESRC).
‘Should we adjust for pupil background in school value-added models? A study of Progress 8 and school accountability in England’ by Dr George Leckie and Professor Harvey Goldstein.
The research confirms that different groups make different progress during their secondary education:
- Girls make 0.26 grades more progress per subject between Key Stage 2 and GCSEs than boys
- Chinese pupils make 0.70 grades more progress per subject than expected
- White British pupils make 0.08 grades less progress per subject than expected
- Pupils with special educational needs account for 11 per cent of all pupils and make considerably less progress than pupils with no special educational needs, with attainment gaps widening during secondary school
- Pupils eligible for free school meals account for 27 per cent of all pupils and make 0.43 grades less progress per subject than their peers
The research looked at 501,851 pupils in 3,098 schools, located in 151 local authorities across nine regions in England. Once Progress 8 was adjusted for pupil background characteristics:
- 574 schools (19 per cent of all state secondary schools in the country) moved up or down the national league table by 500 or more ranks
- 110 schools (four per cent) moved over 1000 ranks
- 988 schools (38 per cent) changed their Progress 8 banding
- The number of schools judged to be in the ‘well below average banding’ would drop from 303 schools (9.8 per cent) to 196 schools (6.3 per cent)
- The number of schools ranked ‘well above average’ would decrease from 193 schools (6.2 per cent) to 148 schools (4.8 per cent)