Bristol Conversations in Education - Establishing Dependability of Judgement in a Graduate Teaching Performance Assessment
Michele Haynes, Australian Catholic University
Room 4.10, School of Education, 35 Berkeley Square, Bristol, BS8 1JA
This event is part of the School of Education's 'Bristol Conversations in Education' seminar series. These seminars are free and open to the public.
Speaker: Prof. Michele Haynes, Australian Catholic University
Teaching performance assessments have recently been adopted in Australia to ensure graduate readiness for high quality teaching in schools. The Graduate Teacher Performance Assessment (GTPA) is one of two Australian TPAs endorsed in 2018 and is currently being implemented in teacher education programs offered in 14 Higher Education Institutes (HEIs) across Australia. The introduction of a performance assessment requires a rigorous approach to establishing validity and consistency of judgement against a performance standard and, for the GTPA, a multi-stage judgemental process was used as a framework to establish judgement consistency and to calibrate cross-institution judgement decisions against benchmark performances. During the trial implementation of the GTPA in 2017 two independent studies were undertaken to: validate the instrument, assess judgement consistency, and establish the standard. In this presentation the approach to implementation of the GTPA and the methodologies used in the two studies will be described. Results for consistency of judgement obtained from the first study will be discussed, where subsets of GTPA samples from the participating HEIs were rated on an online platform by multiple judges and the scores were analysed using a multi-facet Rasch measurement (MFRM) model. Overall, the study has established dependability of the GTPA and has demonstrated the need to undertake cross-institution calibration activities each year to maintain this at a high level, particularly as new raters are recruited and experienced raters develop expertise in recognising quality according to the criteria.