What does the EEF study tell us?
In 2011 – 2014, the Education Endowment Foundation funded an independent, randomised controlled trial to assess the impact of the mastery approach on pupil attainment.
The study was carried out in two parts:
Read the full Mathematics Mastery Primary report
Read the full Mathematics Mastery Secondary report
In total, 127 schools and 10,114 pupils were involved. Half were assigned as ‘control’ schools, and did not follow the Mathematics Mastery programme. The other half were assigned as ‘intervention’ schools and followed the Mathematics Mastery programme for one year.
All schools involved in the trial were neutral:
None of them had been involved with Mathematics Mastery previously. The base of schools was nationally representative, with variance in size and academic performance.
Assessment:
At the end of the year, the primary pupils were assessed by undertaking the Number Knowledge Test. The secondary pupils were assessed by undertaking the Progress in Maths test.
Evaluation:
The evaluation was performed by UCL’s Institute of Education (IoE), and involved a meta-analysis of the primary and secondary findings.
What were the results?
What are the limitations of the study?
The study only assessed the impact of the Mathematics Mastery programme after one year.
Our programme is designed as a long-term, cumulative approach. This means mathematical understanding, knowledge and skills are systematically deepened and built-upon year after year. Furthermore, the first year of the programme involves significant adjustment for all staff involved. Due to the nature of the programme, the impact after one year only is not reflective of the medium and long-term impact.
We hope to conduct further research on the long-term impact of the Mathematics Mastery programme, (using data from SATs and GCSEs) based on pupils who’ve been taught our programme from the start of primary or secondary school. This will not be possible for several years, as we are not yet active in every year group.
The study was also conducted in the very early days of the Mathematics Mastery programme coming into fruition. The programme has now been running for several years longer, and has continued to evolve and improve.
The study focused on pupil attainment only, and did not take into account pupil engagement or the professional development of teachers. Whilst the academic success of pupils is crucial, it’s vital to look at attainment as just one element of the path to transforming mathematics education. Our programme is designed to engage pupils in mathematics, so they enjoy the subject and are enthusiastic about their learning. We focus on the professional development of teachers, enabling them to develop and enhance their mathematical understanding and teaching. When measuring the impact of our programme, we consider both of these elements alongside pupil attainment.
The findings were substantially lower than the average effects seen in the existing literature on ‘mastery approaches’.
Previous studies were conducted in the United States in the 1970s and 80s, so may overstate the possible impact in English schools today. In addition, our Mathematics Mastery programme is designed to align with the National Curriculum and therefore differs in structure from the examples of ‘mastery learning’ previously studied.
The findings from the primary and secondary trials were combined using an approach called ‘meta-analysis’. Meta-analysis is a robust research method but care is needed when interpreting meta-analysed findings. In this study, the overall positive impact of the programme (as identified in the study results) was likely to be more predictive of the programme’s impact on primary pupils rather than on secondary pupils.
However, this should again be looked at in context. Secondary pupils – who were taught on the Mathematics Mastery programme for the first time in this study – have already had six years of teaching with a non-Mathematics Mastery approach. Therefore, adjustment time must be allowed while teachers and pupils adjust to the new approach.
Primary pupils’ attainment was assessed via the Number Knowledge Test. This test focuses on pupils’ understanding and fluency with number and calculations. Research has shown that the knowledge assessed at each age level of this test is essential for successful learning of arithmetic in school, and is foundational for higher mathematics learning.
Assessment via this test may have given the pupils in ‘intervention’ schools an advantage. This is because the Mathematics Mastery Year 1 programmes of study spends more time on number, and less on geometry and data, in comparison to the programmes of study followed by most ‘control’ schools.
Secondary pupils’ attainment was assessed via the Progress in Maths test. Progress in Maths (PiM) is a standardised assessment of pupils’ mathematical skills and knowledge. This includes number, shape, data handling and algebra.
Assessment via this test may have given pupils in ‘control’ schools an advantage. This is because the Mathematics Mastery Year 7 programmes of study focus on fewer topics in greater depth, and so 40% of the questions on the PiM test assessed curriculum topics which had not yet been taught to the pupils in the ‘intervention’ schools.
