The report evaluated the impact of signing up to the Mathematics Mastery secondary programme on GCSE mathematics grade, in two outcome years: 2018 and 2019.
The evaluation used what is known as a quasi-experimental design. This involves comparing the outcomes of pupils who went to a school that took part in Mathematics Mastery to those of pupils from a matched comparison group of statistically similar schools. This approach mimics what would be done in a formal experiment such as a randomised control trial.
FFT selected schools that were similar with respect to:
Only mainstream state-funded schools in England were considered for the comparison group. We also excluded selective schools, as none of the treated schools were in this category. The comparison group was constructed using a technique known as covariate balancing propensity score matching (CBPS). This procedure reweights the non-participating schools such that, in aggregate, the set of non-participating schools and the set of participating schools balance (i.e. match) on the pre-treatment characteristics listed above.
Having calculated weights for each school, FFT estimated the effect of the programme using weighted pupil-level data. We used multilevel models, nesting pupils within schools and controlling for a number of confounding variables related to the outcome: gender,
whether their first language was English, whether they were eligible for free school meals
The FFT evaluation found:
Find out more about Mathematics Mastery's secondary programme
