How should we measure school performance and hold schools accountable? A study of competing statistical methods and how they compare to Progress 8
ESRC standard grant
- Grant Code: ES/R010285/1
- Duration: 01/10/2018 - 30/09/2021
- Amount Awarded: £467,000
People
- George Leckie, Principal Investigator
- Harvey Goldstein, Co-Investigator
- Lucy Prior, Research Associate
Summary
In 2016, the Department for Education radically overhauled their secondary school accountability system and introduced 'Progress 8', arguing it to be the simplest and fairest school performance measure to date. Progress 8 aims to quantify and communicate the average academic value each school adds to their pupils' learning. Specifically, Progress 8 measures how much higher each school's pupils score in their age 16 GCSE examinations than expected given their age 11 KS2 test cores when they started secondary schooling. Progress 8 scores are used to hold schools to account, with the lowest scoring schools judged 'underperforming' and 'coasting'; classifications that trigger intense scrutiny and intervention from the school inspectorate, Ofsted. Given the high-stakes involved, research is urgently needed to first evaluate the strengths and weaknesses of the statistical method underlying Progress 8, and second, to explore the potential benefits of alternative methods for measuring school performance. Our proposed research will address these needs and in doing so will advance scientific understanding about school performance measurement.
See the ESRC page ES/R010285/1 for more details.
Videos
The following video is a research presentation of the working paper below given to mixed audience of acdemics and non-academics at UCL Institute of Education 0n 14/11/2018:
Journal articles
Working papers
The following working paper is currently under review:
- Leckie, G., & Goldstein, H. (2018). Should we adjust for pupil background in school value-added models? A study of Progress 8 and school accountability in England. arXiv:1811.09240 [stat.AP]. Working paper. Video. Slides. Since published in the British Educational Research Journal.
Representative articles
The following articles are representative of the style of work we are aiming to do:
- Leckie, G. (2018). Avoiding bias when estimating the consistency and stability of value-added school effects using multilevel models. Journal of Educational and Behavioral Statistics. Forthcoming. DOI: 10.3102/1076998618755351. Article. Slides.
- Leckie, G., & Goldstein, H. (2017). The evolution of school league tables in England 1992-2016: ‘contextual value-added’, ‘expected progress’ and ‘progress 8’. British Educational Research Journal, 43, 193-212. DOI: 10.1002/berj.3264. Article. Video. Slides.
- Leckie, G., & Goldstein, H. (2011). Understanding uncertainty in school league tables. Fiscal Studies, 32, 207-224. DOI: 10.1111/j.1475-5890.2011.00133.x. Article. Visualization.
- Leckie, G., & Goldstein, H. (2009). The limitations of using school league tables to inform school choice. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172, 835-851. DOI: 10.1111/j.1467-985X.2009.00597.x. Article.
Keynotes conference presentations:
Other presentations:
- September 2019. BERA. Manchester.
- April 2019. ‘Progress 8’ National Education Union annual conference, Liverpool.
- April 2019. 'Calculating intraclass correlation coefficients in multilevel models for count responses'. 12th International Amsterdam Multilevel Conference, Utrecht.
- November 2018. ‘Accountability and school differential effects’. Department of Learning and Leadership, IOE UCL, London. Flyer. Article. Slides. Video.
- September 2018. Bristol Cathedral School, Bristol. Article. Slides.
- September 2018. 'Avoiding bias when estimating the consistency and stability of value-added school effects'. RSS Annual Conference, University of Cardiff. Article. Slides.
- May 2018. 'Avoiding bias when estimating the consistency and stability of value-added school effects'. Q-step Seminar Series, University of Exeter. Article. Slides.
- April 2018. 'Avoiding bias when estimating the consistency and stability of value-added school effects'. 2018 AERA Annual Meeting, New York. Article. Slides.
- July 2017. ‘Modelling heterogeneous variance-covariance components in two-level models'. JEBS: Methods for Eductional and Behavioral Applications session at Joint Statistical Meetings (JSM), Baltimore. Article. Slides.
- June 2017. Data Intensive Research Workshop, Cardiff University. Slides.
Teaching presentations:
- 'The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England'. M-level: Statistics in Education. University of Bristol. December 2018.
- 'Introductory overview of quantitative methods and basic analysis approaches'. EdD: Conducting Educational Research. University of Bristol. June 2018.
Multilevel modelling short courses:
- October 2018, Introduction to Multilevel Modelling, Bamberg.
- August 2018, Multilevel Modelling: Three-level, Cross-Classified and Multiple Membership Models, EARLI Special Interest Group 18 Educational Effectiveness, Groningen, Netherlands (Course Instructor). Flyer.
Radio coverage
- Janauary 2019: Heart radio
- Janauary 2019: Love sport radio
Newspaper and other print coverage
- January 2019: Bristol Post, Daily Express, The Guardian, The Independent, Leigh Journal, The Star, The Times, Times Educational Supplement, Warrington Guardian, Wigan Today, Yorkshire Post
- June 2018. 'Testing children gives a poor measure of ability'. Letter in the Guarduan in response to Simon Jenkins’s piece on how ‘the cult of tests is ruining our schools’. Article.
Other online media coverage
- January 2019: Ekklesia, Greensheets, Humanists UK, Local Gov, My Science, Reclaiming Schools, Schools Improvement, Schools Week, World News Live 4 You
Knowledge exchange meetings:
- Department for Education: November 2017.
- Fischer Family Trust: December 2018.
- Ofsted: September 2018, February 2019.
Related advisory work:
- 2018-present. Office for Student (OfS) Expert Advisory Group on benchmarking methodologies.
- March 2018. Discussions with Bulgarian minister of education and heads of directorates as expert member of World Bank team.
- November 2017. Discussion with DfE regarding proposed research on evaluating competing statistical methods for measuring school performance.
- November 2017. Discussion with HEFCE regarding proposed research on evaluating competing statistical methods for measuring school performance.
- 2017-present. Higher Education Funding Council England (HEFCE) Learning Gain Expert Group.
Related conference organisation:
- September 2018. Co-organiser (with Becky Allen) of NPD User Group meeting, Bristol.