Comparing Chilean school performance: improving fairness in the classification of schools
In recent years Chile has introduced new school accountability mechanisms. This research examines primary and secondary school performance in context using value-added approaches.
Image: El Mercurio
About the research
In recent years Chile has introduced new school accountability mechanisms. The ‘Sistema Nacional de Aseguramiento de la Calidad de la Educación’ (National System of Education Quality Assurance) encompasses the creation of two new bodies: the Superintendent of Education and the Quality Agency of Education (QAE).
This year, the QAE – a local version of the Office for Standards in Education (OFSTED) – is mandated to classify schools into four groups: High, Medium, Low-Medium and Insufficient performance based on schools’ SIMCE (Sistema de Medición de la Calidad de la Educación) test results and other data. Schools that do not meet the required standards will also face high-stakes consequences (ranging from organisational interventions, to school closure).
This research examines primary and secondary school performance in context using value-added approaches.
The first study focused on students within more than 7,000 classrooms in 2,283 schools in 313 municipalities which took the SIMCE national tests in languages and maths while enrolled in year 8 and 10 during 2004 and 2006. The second study used a range of statistical models to explore girls’ and boys’ (from age 8 to age 12) attainment and progress in maths in more than 3,000 primary schools in 310 municipalities over four successive years (from 2005 and 2009).
Both studies demonstrated that how effective a school appears to be, changed significantly when different approaches to measure school effectiveness are used (Raw, Value Added (VA) or Contextualised Value Added (CVA)). Moreover, the extent to which schools can be more or less effective is affected by the organisation of schools into municipalities.
- The new Chilean school accountability system should be made fairer by complementing attainment measures in place with value-added measures that reflect students’ academic progress.
- School league tables should also control for pupil background characteristics and contextual features of Chilean schools.
- Different value-added scores should be provided for different groups of students (i.e boys and girls, low and high achievers) as well as for different subjects (language and maths) when assessing school performance.
- When a high-stakes accountability policy has been adopted, such as the one recently implemented in Chile, a four-level model (municipality, school, class and student) could be adopted by QAE in order to make fair comparisons between schools; the regional policy level plays a salient role in what happens in the classroom.
- Schools that work in similar contexts could be encouraged by local actors (such as Lideres Educativos) to collaborate to enable them to better develop those features associated with more effective schools (those with positive contextual value-added scores).
- Highlighting the strengths and limitations of each school when teaching different students (e.g. girls/boys) and subjects (e.g. language/ maths) could facilitate partnering and network schemes between schools to enable wider learning possibilities and collaboration.
We found evidence of a very unequal school system, expressed in sizable differences in students’ raw attainment, which, without controlling for additional data, might wrongly be attributable to differences between schools’ effectiveness. In reality these differences are largely due to prior attainment and socioeconomic inequalities.
The apparent differences in school effectiveness based upon schools’ SIMCE raw results and other data, reduced dramatically after controlling for aspects arguably out of the school control – such as prior achievement, parents’ education and number of books at home (reflecting socio-economic status).
Chilean secondary schools tend to be inconsistently effective when teaching different groups of students. Some schools are more effective at teaching boys than girls, low instead of high achievers, or when teaching language instead of maths, and vice versa.
Value-added approaches can provide a powerful tool for school self-evaluation and school accountability. However, just as with any numerical measure, validity issues, uncertainty and limitations need to be understood by stakeholders before using the results.
Muñoz-Chereau, B. (2013) Adding or not Adding Contextual Value in Language: An Instrumental Case Study in Two Chilean Secondary Schools. ICSEI Special Issue. Pensamiento Educativo. Revista de Investigación Educacional Latinoamericana, 50(2), pp. 28-50.
Muñoz-Chereau, B., & Thomas, S. M. (2015) Educational effectiveness in Chilean secondary education: comparing different ‘value added’ approaches to evaluate schools. Assessment in Education: Principles, Policy & Practice, Special Issue on Latin America, pp.1-27.
Muñoz-Chereau, B. (submitted) Gender Gap and School Differential Effects in Mathematics in Chilean Primary Schools. School Effectiveness and School Improvement.
Policy Briefing 49: October 2017
Contact the researchers
Dr Bernadita Muñoz-Chereau, ESRC GCRF Postdoctoral Fellow, School of Education, University of Bristol
Professor Sally Thomas, School of Education, University of Bristol
Dr Bernardita Muñoz-Chereau and Professor Sally Thomas