Centre for Multilevel Modelling

menu

Multilevel modelling of the Government's new school performance measures, 'floor standards' target and 'narrowing the gap' priority

ESRC Future Research Leaders Grant

  • Grant code: ES/K000950/1
  • Duration: 1st January 2013 – 31 December 2015
  • Amount awarded: £207,000

People

Objectives

Each year, the Government publishes school performance tables that report the achievement and progress of pupils in English secondary schools. In the 2011 tables, the Government:

  • introduced a range of new school performance measures to attempt to better measure how schools are performing for all their pupils, not just the average pupil;
  • introduced a new government target that defines schools that fail to meet ‘floor standards’ in these new measures as ‘underperforming’;
  • made it a government priority that all schools narrow the performance gaps between the most disadvantaged pupils in their schools and their peers.

These changes have important implications - ‘underperforming’ schools will risk being placed in ‘special measures’, take-over, or even closure - and it is therefore imperative that they are critically reviewed and that innovative techniques are developed and applied to explore potential improvements to the accuracy, usefulness and communication of school performance tables.

This grant will do this by pursuing a programme of methodological and substantive research using multilevel modelling, and based on secondary analyses of the Government’s highly detailed pupil and school level performance data which underlie their tables.

Visiting fellow

  • Faculty of Education, Queensland University of Technology, Brisbane. January 2015 - February 2015.
  • Department of Quantitative Social Science, Institute of Education, London. January 2013 - August 2013 (2 days a week) May 2014 - June 2014 (2 days a week)

Peer reviewed journal articles

Book chapters

  • Leckie, G. and Goldstein, H. Monitoring school performance using value-added and value-table models: Lessons from the UK. In Perna, C., Pratesi, M., & Ruiz-Gazen, A. (Eds.) Studies in Theoretical and Applied Statistics, Springer, Chapter X, X-X. 

Articles under review in peer reviewed journals

  • French, R., Leckie, G., Sariaslan, A., & Larsson, H. Exploring the role of the family in multilevel models of school effectiveness and student achievement using Swedish registry data. Submitted.
  • Sindall, K., Sturgis, P. Steele, F, Leckie, G., & French, R. A reassessment of socio-economic gradients in child cognitive development using Growth Mixture Models. Revise and resubmit.

Working papers

Articles in preparation for peer reviewed journals

  • Leckie, G. A multilevel longitudinal analysis of rich-poor achievement gaps in English schools. In preparation.

Conference proceedings

Other publications

Invited presentations

  • '25 years of school leagues tables, accountability and choice: Lessons from England'. Keynote at Educational Effectiveness EARLI SIG meeting. Groningen. August 2018.
  • 'The evolution of school league tables in England 1992-2016: ‘contextual value-added’, ‘expected progress’ and ‘progress 8’' University of Warwick. February 2018.
  • 'Heterogeneous variance-covariance components in two-level models'. ‘JEBS: Methods for Educational and Behavioral Applications’ session at Joint Statistical Meetings (JSM), Baltimore. August 2017.
  • 'England’s experience with school value-added models 2004-2017'. World Bank and Bulgarian Ministry of Education - Republic of Bulgaria, Sofia. March 2017.
  • 'Monitoring school performance using value-added models: Lessons from the UK' ‘Statistical tools for monitoring the educational system and assessing students’ performances’ session of the 48th Scientific Meeting of The Italian Statistical Society, University of Salerno. June 2016. 
  • 'School League Tables: The Good, the Bad and the Ugly' East Midlands local group of the Royal Statistical Society, University of Leicester. March 2016. Flyer.
  • 'A multilevel longitudinal analysis of rich-poor achievement gaps in English schools'. Sheffield Methods Institute, University of Sheffield. October 2015.
  • 'A multilevel longitudinal analysis of rich-poor achievement gaps in English schools'. Royal Statistical Society Annual Conference. September 2015.
  • 'Monitoring school performance: A statistical critique of England's ‘expected progress’ approach with comparison to multilevel ‘value-added’ models', Berkeley Evaluation & Assessment Research Center, University of California, Berkeley. February 2015.
  • 'Monitoring school performance: A statistical critique of England's ‘expected progress’ approach with comparison to multilevel ‘value-added’ models', Faculty of Education, Queensland University of Technology, Brisbane. February 2015.
  • 'Symposium on strengths and shortcomings of value-added models in education', Institute for Positive Psychology and Education (IPPE), Australian Catholic University, Sydney. February 2015. YouTube video.
  • 'Monitoring school performance: A multilevel value-added modelling alternative to England’s ‘expected progress’ measure'. Quantitative Methods Hub seminar series, Department of Education, University of Oxford. December 2013.
  • 'Monitoring school performance: A multilevel value-added modelling alternative to England’s ‘expected progress’ measure'. RSS Glasgow Local Group seminar. November 2014.  Report.
  • 'Statistical flaws in the Government’s ‘expected progress’ measure of school performance'. 2nd JURE Special Interest Group Educational Effectiveness pre-conference, Southampton. August 2014. Keynote. Flyer.
  • 'England’s multilevel model based value-added school league tables: Measuring and communicating statistical uncertainty to parents'. Advanced Multilevel Modelling workshop, Eberhard Karls Universität Tübingen, Department of Education, Tuebingen. September 2013.
  • 'England’s multilevel model based value-added school league tables: Measuring and communicating statistical uncertainty to parents'. Statistical methods in educational evaluation session, 59th World Statistics Congress, Hong Kong. August 2013. Conference proceeding.
  • 'Understanding School League Tables', Centenary Festival of Education 2013, Graduate School of Education, University of Bristol. June 2013. YouTube video.
  • 'Multilevel modelling of the Government’s new school performance tables'. ESRC visit, University of Bristol. March 2013.

Other conference, workshop and seminar presentations

  • 'Avoiding bias when estimating the consistency and stability of value-added school effects'. AERA annual meeting, New York. April 2018.
  • 'Avoiding bias when estimating the consistency and stability of value-added school effects'. 11th International Multilevel Conference, Utrecht. April 2017.
  • 'The evolution of school league tables in England 1992-2016: ‘contextual value-added’, 'expected progress’ and ‘progress 8’' Bristol Conversations in Education, Graduate School of Education, University of Bristol. November 2016.
  • 'Mind the gap: A multilevel longitudinal analysis of rich-poor achievement differences in London schools'. 12th Conference of the European Sociological Association, Prague. August 2015.
  • 'Mind the gap: A multilevel longitudinal analysis of rich-poor achievement differences in London schools'. 10th International Multilevel Conference, Utrecht. April 2015.

Teaching presentations

  • 'Introductory overview of quantitative methods and basic analysis approaches'. EdD: Conducting Educational Research. University of Bristol. June 2018.
  • 'Introductory overview of quantitative methods and basic analysis approaches'. EdD: Conducting Educational Research. University of Bristol. June 2016.
  • 'Understanding school level data, 'value-added' measures and school self-evaluation'. Quality and Improvement unit in the Leadership and Policy specialism, University of Bristol. October 2015.
  • 'Design and analysis issues in school league tables'. EdD Research Methods in Learning, Leadership and Policy. Graduate School of Education, University of Bristol. June 2015.
  • 'Design and analysis issues in school league tables'. EdD Research Methods in Learning, Leadership and Policy. Graduate School of Education, University of Bristol. June 2014.
  • 'Multilevel modelling'. MRes Quantitative Analysis III (one lecture and one practical), UCL Institute of Education, London. May 2014.
  • 'Multilevel modelling'. MRes Quantitative Analysis III (one lecture and one practical), UCL Institute of Education, London. May 2013.

Multilevel modelling short courses given

  • 'Multilevel Modelling'. Short Course Doctoral Schools Ghent University. June 2016. ~30 participants.
  • 'Multilevel Modelling'. Faculty of Education, Queensland University of Technology, Brisbane. February 2015. Course Instructor. ~25 participants.
  • 'Multilevel Modelling'. Institute for Positive Psychology and Education (IPPE), Australian Catholic University, Sydney. February 2015. Course Instructor. ~10 participants.
  • 'Advanced Multilevel Modelling'. Eberhard Karls Universität Tübingen, Department of Education, Tuebingen. September 2013. Course Instructor. ~60 participants.

Citations in government and other non-academic publications

  • NSW Department of Education and Communities (2014) Value added models for NSW government schools. Centre for Education Statistics and Evaluation, Strategic Information and Reporting, Office of Education, NSW Department of Education and Communities.

Media coverage

Training and development undergone

An important part of the grant is for me to accelerate my statistical modelling skills development and expertise. I have attended the following courses:

  • Mixed Models for Longitudinal Categorical Outcomes. Donald Hedeker. ASA webinar. 16 July 2015.
  • Multilevel Structural Equation Modeling in R. Paras Mehta. 10th International Multilevel Conference, Utrecht. 7 April 2015.
  • The Application of Item Response Models using R. Dave Hessen. VI European Congress of Methodology, Utrecht. 22 July 2014.
  • Structural Equation Modeling with lavaan. Yves Rosseel. VI European Congress of Methodology, Utrecht. 22 July 2014.
  • Geographically Weighted Modelling. Martin Charlton, Chris Brunsdon and Paul Harris. University of Bristol. 23-24 April 2014.
  • A 2 Day course on Longitudinal and Incomplete Data. Geert Verbeke and Geert Molenberghs. RSS, London. 26-27 March 2014.
  • Inference for Clustered Data. Colin Cameron. IFS, London. 11 October 2013.
  • Simulation Methods. Colin Cameron. IFS, London. 10 October 2013.
  • Joint Modelling of Repeated Measurement and Event Time Data in Clinical Research. RSS Conference, Newcastle. 2 September 2013.

I am also developing my more general research and knowledge exchange-related skills:

  • ESRC FRL Workshop, Swindon, 17 September 2013

In addition to the above conferences and events, I have also attended (but not presented) at the following conferences and events:

  • 'Advances in Multilevel Modeling for Educational Research: Addressing Practical Issues Found in Real-World Applications', University of Maryland, US. 14-15 November 2014. Attended via live streaming.
  • 'The Rise of Testing and the Demise of Teacher Assessment in Australia', Centre for Assessment and Evaluation (CAER), Graduate School of Education (GSOE), University of Bristol. 23 October 2014.

Related conference organisation

Related advisory work

  • Member of Higher Education Funding Council England (HEFCE) Learning Gain Expert Group, 2017-.
  • Advised World Bank and Bulgarian Ministry of Education and Science on their National school value-added project. 2017-2018.
  • Advised RAND Europe on Department for Education project 'Approaches to address the missing KS2 data in LSYPE2: User guide'. February-July 2016.
  • Advised Slovak Republic National Institute for Certified Educational Measurements on their Schools Value-Added Report. July 2015.
  • Advised Reclaiming Schools website and the Charter for Primary Education on the predictive validity of baseline tests for KS2 scores. June 2015.
  • Member of the Sutton Trust Evaluation Framework Advisory group. 2015 -.
  • Advised Centre for Appearance Research at the University of the West of England on multilevel modelling of two school based randomised controlled trials, June 2015.
  • Advised RAND Europe on Department for Education project 'Factors associated with underachievement: statistical model development'. September 2014 and June 2015.
  • Advised the The Times and Sunday Times Data Team on their key stage 4 to key stage 5 value added project. 30 May 2014.
  • Advised Colombian Evaluation Institute on their ICFES Value Added in Higher Education project. March 2014.
  • Participated in group Department for Education group discussion with Professor Kane on teacher effectiveness research. 26 February 2014.
  • Advised Department for Education on development of family indicators in the national pupil database. 10 September 2013.
  • Advised Pearson UK on multilevel modelling of rater effects on examination marking. 2013.
  • Advised National Centre for Educational Achievement (NCEA) / ACT testing company on a value-added modelling report. July 2013.

Previous publications

Prior to the grant, I have published the following related articles:

Feedback

Edit this page