Centre for Multilevel Modelling

menu

How should we measure school performance and hold schools accountable? A study of competing statistical methods and how they compare to Progress 8

 

ESRC standard grant

  • Grant Code: ES/R010285/1
  • Duration: 01/10/2018 - 30/09/2021
  • Amount Awarded: £467,000

People

Summary

In 2016, the Department for Education radically overhauled their secondary school accountability system and introduced 'Progress 8', arguing it to be the simplest and fairest school performance measure to date. Progress 8 aims to quantify and communicate the average academic value each school adds to their pupils' learning. Specifically, Progress 8 measures how much higher each school's pupils score in their age 16 GCSE examinations than expected given their age 11 KS2 test cores when they started secondary schooling. Progress 8 scores are used to hold schools to account, with the lowest scoring schools judged 'underperforming' and 'coasting'; classifications that trigger intense scrutiny and intervention from the school inspectorate, Ofsted. Given the high-stakes involved, research is urgently needed to first evaluate the strengths and weaknesses of the statistical method underlying Progress 8, and second, to explore the potential benefits of alternative methods for measuring school performance. Our proposed research will address these needs and in doing so will advance scientific understanding about school performance measurement.
 
See the ESRC page ES/R010285/1 for more details.

Videos

The following video is of the Northern Powerhouse Partnership giving oral evidence to MPs on the Education Select Committee about our recent Adjusted Progress 8 reserach on 05/06/2019. We previously gave evidence to this committee on this same topic at a private meeting with them in March 2019.

The following video is a research presentation of the working paper below given to mixed audience of academics and non-academics at UCL Institute of Education 0n 14/11/2018:

Journal articles

  • Leckie, G. and Goldstein, H. (2019). The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England. British Educational Research Journal. Forthcoming. Article. Video. Slides.

Representative articles

The following articles are representative of the style of work we are aiming to do:
  • Leckie, G. (2018). Avoiding bias when estimating the consistency and stability of value-added school effects using multilevel models. Journal of Educational and Behavioral Statistics. Forthcoming. DOI: 10.3102/1076998618755351. Article. Slides.
  • Leckie, G., & Goldstein, H. (2017). The evolution of school league tables in England 1992-2016: ‘contextual value-added’, ‘expected progress’ and ‘progress 8’. British Educational Research Journal, 43, 193-212. DOI: 10.1002/berj.3264. Article. Video. Slides.
  • Leckie, G., & Goldstein, H. (2011). Understanding uncertainty in school league tables. Fiscal Studies, 32, 207-224. DOI: 10.1111/j.1475-5890.2011.00133.x. Article. Visualization.
  • Leckie, G., & Goldstein, H. (2009). The limitations of using school league tables to inform school choice. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172, 835-851. DOI: 10.1111/j.1467-985X.2009.00597.x. Article.

Keynote conference presentations:

  • August 2018. '25 years of school leagues tables, accountability and choice: Lessons from England'. Educational Effectiveness EARLI SIG 18 & 23 conference. Groningen. Flyer. Slides.

Other presentations:

  • January 2020. Wolverhampton.
  • September 2019. 'School accountability in crisis - holding the judges to account'. British Educational Research Association (BERA) . Manchester.
  • April 2019. ‘Progress 8’ National Education Union annual conference, Liverpool.
  • April 2019. 'Calculating intraclass correlation coefficients in multilevel models for count responses'. 12th International Amsterdam Multilevel Conference, Utrecht.
  • November 2018. ‘Accountability and school differential effects’. Department of Learning and Leadership, IOE UCL, London. Flyer. Article. Slides. Video.
  • September 2018. Bristol Cathedral School, Bristol. Article. Slides.
  • September 2018. 'Avoiding bias when estimating the consistency and stability of value-added school effects'. RSS Annual Conference, University of Cardiff. Article. Slides.
  • May 2018. 'Avoiding bias when estimating the consistency and stability of value-added school effects'. Q-step Seminar Series, University of Exeter. Article. Slides.
  • April 2018. 'Avoiding bias when estimating the consistency and stability of value-added school effects'. 2018 AERA Annual Meeting, New York. Article. Slides.
  • July 2017. ‘Modelling heterogeneous variance-covariance components in two-level models'. JEBS: Methods for Eductional and Behavioral Applications session at Joint Statistical Meetings (JSM), Baltimore. Article. Slides.
  • June 2017. Data Intensive Research Workshop, Cardiff University. Slides.

Book chapters and Encyclopedia entries

  • Leckie, G. (2019). Cross-classified multilevel models, Encyclopedia of Quality of Life and Well-Being Research. Second edition.
  • Leckie, G. (2019). Multilevel models for continuous outcomes, SAGE Research Methods Foundations series.
  • Leckie, G. (2019). Multiple membership multilevel models, Encyclopedia of Quality of Life and Well-Being Research. Second edition.

Teaching presentations:

  • 'The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England'. M-level: Statistics in Education. University of Bristol. December 2018.
  • 'Introductory overview of quantitative methods and basic analysis approaches'. EdD: Conducting Educational Research. University of Bristol. June 2018. 

Multilevel modelling short courses:

  • October 2018, Introduction to Multilevel Modelling, Bamberg.
  • August 2018, Multilevel Modelling: Three-level, Cross-Classified and Multiple Membership Models, EARLI Special Interest Group 18 Educational Effectiveness, Groningen, Netherlands (Course Instructor). Flyer.

Policy briefings

  • Leckie, G., Goldstein, H. (2019). The Government’s Progress 8 school performance measure needs to account for pupil background. PolicyBristol. Policy Briefing 66: 01/2019.

Radio coverage

  • January 2019: Heart radio
  • January 2019: Love sport radio

Newspaper and other print coverage

  • March 2019: Times Educational Supplement.
  • January 2019: Bristol Post, Daily Express, The Guardian, The Independent, Leigh Journal, The Star, The Times, Times Educational Supplement, Warrington Guardian, Wigan Today, Yorkshire Post
  • June 2018. 'Testing children gives a poor measure of ability'. Letter in the Guarduan in response to Simon Jenkins’s piece on how ‘the cult of tests is ruining our schools’. Article.

Other online media coverage

  • March 2019: Schools Week
  • January 2019:  Ekklesia, Greensheets, Humanists UK, Local Gov, My Science, Reclaiming Schools, Schools Improvement, Schools Week, World News Live 4 You

Knowledge exchange meetings:

  • Augsust 2019: Fischer Family Trust. Discussed their related research work.
  • April 2019: Department for Education, London. Discussed Adjusted Progress 8 work.
  • February 2019: Ofsted, Bristol.  Discussed Adjusted Progress 8 work.
  • December 2018: Fischer Family Trust.  Discussed Adjusted Progress 8 work.
  • September 2018: Ofsted, Bristol. Discussed Adjusted Progress 8 work.
  • November 2017: Department for Education, London.  Discussed grant proposal.
  • November 2017. HEFCE, Bristol. Discussed grant proposal.

Other meetings 

  • August 2019. Discussion with Education Campaigns Manager at Humanists UK about Progress 8
  • May 2019. Discussion with Director and Chief Economist of Northern Powerhouse Partnership
  • April 2019. Presentation of Progress 8 research to Emma Hardy MP (Education Select Committee member) at Portcullis house
  • March 2019. Presentation of Progress 8 research to MPs at private meeting with UK Parliament Education Select Committee
  • February 2019. Discussion with Head of Co-Op Academies about Progress 8

Related advisory work:

  • June 2019. Discussion with Israel Academy of Sciences about school value added models.
  • 2018-present: Discussions with Trinidad and Tobago Department for Education about school value added models.
  • 2018-present. Discussions with Sweden Department for Education about school value added models.
  • 2018-present. Office for Student (OfS) Expert Advisory Group on benchmarking methodologies.
  • March 2018. Discussions with Bulgarian Minister of Education and heads of directorates as expert member of World Bank team.
  • 2017-present. Higher Education Funding Council England (HEFCE) Learning Gain Expert Group.

Related conference organisation:

Previous Grant: ESRC Future Research Leaders Grant on school performance statistics.

Feedback

Edit this page