The 2020 GCSE and A-level 'exam grades fiasco': A secondary data analysis of students' grades and Ofqual's algorithm
ESRC Secondary Data Analysis Initiative grant
- Grant Code: ES/W000555/1
- Duration: 01/10/2021 - 30/09/2023
- Amount Awarded: £299,000
People
- George Leckie, Principal Investigator
- Lucy Prior, Research Associate
Summary
The awarding of the 2020 GCSE and A-Level exam grades in England was widely viewed as a ‘fiasco’. When COVID-19 forced the cancellation of exams, DfE and Ofqual asked centres (schools and colleges) to submit Centre Assessment Grades (CAGs) and rankings. Namely, the grades and rank orders within their centres that teachers thought students would have achieved had they sat their exams. Ofqual, tasked with preventing grade inflation and ensuring grading consistency , viewed students’ CAGs as overly optimistic and so replaced them with calculated grades predicted via their Direct Centre-level Performance (DCP) algorithm. The result was that 40% of CAGs were downgraded by one or more grades. There was immediate public outcry that students were ‘robbed’ of the grades they deserved. The media quickly reported that the calculated grades were systematically biased against various students and schools. Others argued that they were not reliable enough, with predictive accuracy especially low in smaller centres. The furore resulted in a government U-turn, Prime Minister Boris Johnson declaring Ofqual’s DCP approach a ‘mutant algorithm’, and Education Secretary Gavin Williamson instructing Ofqual to revert to the original CAGs. In January 2021, the government announced that the 2021 exams will also be cancelled with CAGs used in their place.
Students are accepted into universities and employment based on their GCSE and A-level grades. Their grades directly impact their immediate future. It is therefore vitally important for society to understand the extent to which students’ grades were unfairly awarded in 2020 and 2021 with biases potentially varying across individual centres and by student and school characteristics. It is also crucial to learn from the fiasco to help inform DfE and Ofqual responses when CAGs might again be needed in place of exam grades (e.g., due to future pandemics, teacher strikes, exam boycotts, leaked exam papers, centre malpractice, technology failures with onscreen assessments). More generally, our findings will be relevant to those calling for a reintroduction of coursework and other non-exam assessments at GCSE and A-level and especially those calling for a removal of exams altogether, since this would imply a permanent reliance on school and college assessments.
Our overarching aim is to therefore conduct an independent and rigorous secondary data analysis of the 2020 and 2021 GCSE and A-level exam grades to explore not just what went wrong statistically, but to identify what could be improved statistically when predicting grades in future years.
Articles in preparation for peer reviewed journals
- School effects on University admissions
- Sociodemographic inequalities in UCAS predicted grades
Working papers
- Prior, L. and Leckie, G. (2023). School effects on whether and where students apply to university. Bristol Working Paper in Education Series.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England. Bristol Working Papers in Education Series. Working Paper: 06/2022. Article.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England. arXiv:2211.06321 [cs.CY]. Article.
Submitted conference session
- Royal Economic Society 2023 Annual Conference: Special Session "Administrative data for education research: building bridges using the GRADE data sharing project", April 2023.
Invited speakers
- Dennis Sherwood 'The Great Grading Scandal', 18th October 2022. Bristol Conversations in Education.
Research discussions
- Meeting with James Zuccollo of Education Policy Institute (EPI) on school effects on longer term outcomes, February 2023.
- Discussion around research ideas with Dennis Sherwood, October 2022
- Discussion with Andrew Boyle Director of Quantitative Research at AlphaPlus and Analysis and Ben Smith Senior Statistician at AlphaPlus about 2020 CAGs, June 2022
- Discussion with Alex Jones Ofsted Director for Insights and Research, May 2022
- Discussion around research ideas with Professor Robin Shields, School of Education, University of Bristol, October 2021
GRADE events attended
- GRADE User Group meeting, Spring 2023
- GRADE dataset roundtable: data-mine and discussion on the art of the possible, November 2021
- GRADE Applicant Webinar, September 2021
Other events attended related to the grant
- ADR UK Showcase for Ambassadors, London, 21 November 2022
- Longitudinal Education Outcomes (LEO) data: what next? London, 17 October 2022.
- Education Symposium: Administrative Data for Education Research: Building a sustainable future. Sheffield, 16 June 2022.
- Meeting with UKRI ESRC, MoJ, DfE and academics about ‘Building admin data communities’, Zoom, April 2022
Other activities related to the grant
- ADR UK Ambassador for increasing and improving the use of administrative data in academia for research to inform policy
- Submitted a PhD proposal for the ESRC linked administrative data studentship call, October 2022.