In the fall of 2019, Zearn concluded a two-year study of the impact of Zearn Math usage on state test scores in Arizona. Researchers matched student-level digital lesson completion on Zearn Math with anonymized AzMERIT test scores in order to answer two primary questions:
- Did scores improve more for students with high Zearn Math usage?
- Did improvement vary based on starting point (i.e., student scores on the 2018 AzMERIT)?
- What was the incremental effect of lesson completion at the student level?
Our researchers addressed the first question by comparing the percentage of treatment vs. control students who improved by at least one level on the AzMERIT test, which categorizes scores from Level 1 (lowest proficiency) to Level 4 (highest proficiency). The findings are illustrated in the graphs below: students with high Zearn Math usage (as depicted in the blue bars) were approximately twice as likely to improve their score during the 2018-19 school year, relative to students with no/low Zearn Math usage (as depicted in the gray bars).
Average percent of students who increased at least 1 level on 2018-19 AzMERIT
In addition to measuring overall score improvement, our researchers considered the rate of students reaching the proficiency bar (Level 3 on the AzMERIT) after 5th grade. The analysis applied the observed 2018-19 score changes over two years, from 3rd through 5th grade, and indicated that students with high Zearn Math usage are 64% more likely to reach proficiency than students with low Zearn Math usage. The results are even stronger when the researchers analyzed a subset of students starting at Level 1 after 3rd grade. For those students, who were struggling in math the most at that point, consistent Zearn Math usage is tied to a 124% increase in the likelihood of students reaching proficiency by 5th grade.
To better understand the effect of each incremental lesson on achievement gains, we also ran a student-level regression measuring how well student lesson completion predicted 2019 score, controlling for grade level, starting score, and classroom-level progress. The effect of student lesson completion was notable in its significance and magnitude - on a weekly basis, increasing student usage by three Zearn Math lessons per week over the course of the year drove learning gains equivalent to adding 140 additional instructional days.
In the analysis conducted, students across all levels demonstrated stronger growth on the AzMERIT when learning consistently with Zearn Math, and growth was highest for students who had previously struggled. In other words, Zearn Math usage supported achievement gains across the board and also a narrowing of the achievement gap for students in the Arizona district. The incremental impact of each lesson at the student level was quite strong, and underlines the impact each additional lesson can have on student learning gains. We are encouraged by these results as we pursue our mission to ensure all kids love learning math, and look forward to continuing to study the impact Zearn Math is driving in schools and districts across the country.