District Snapshot

Consistent Zearn usage resulted in a 22 percent increase in math proficiency for multilingual learners

A new, district-wide study analyzing the impact of programmatic Zearn usage for students in a large, urban district in the Midwest offers evidence that Zearn Math accelerates math learning for all students.

50K + students
70% economically disadvantaged 
70% Black and Latino 
30% Multilingual learners

Researchers examined how elementary- and middle-school students performed on the spring 2022 state math assessment. The research, which uses quasi-experimental matching techniques to isolate the impact of Zearn Math on student math achievement, compared students who consistently completed three or more Zearn lessons each week with similarly matched students who did not consistently use Zearn Math<cms-citation-text>[1]<cms-citation-text>. Unlike a standard correlational analysis, this method allows differences in outcomes to be more confidently attributed to Zearn and not to other variables. Researchers found:

  • Elementary- and middle-school students who consistently used Zearn had 4.2 times the growth in their state assessment scores than students who did not use Zearn. 
  • Consistent Zearn usage resulted in a double-digit increase in proficiency for Black and Latino Students, economically disadvantaged students, and multilingual learners, compared to declines in proficiency for matched students who did not use Zearn<cms-citation-text>[2]<cms-citation-text>.
  • Across all starting levels of proficiency, students with consistent Zearn usage were more likely to improve their proficiency level. 
  • Students at the lowest level of math proficiency were three times more likely to improvetheir proficiency level when they used Zearn compared to students who did not use Zearn.

<text-style-figure>FIGURE 1<text-style-figure>

Students who consistently used Zearn had 4.2 times the growth on the 2022 state assessment than students who did not use Zearn.

Average State Math Assessment Score by Zearn Math usage graph

<text-style-figure>FIGURE 2<text-style-figure>

Consistent Zearn usage resulted in a double-digit increase in proficiency for each student subgroup, while similarly matched students experienced declines.

Percentage point change in students reaching proficiency by Zearn Math usage graph

Additionally, students in this district saw a 16 percentage point  increase in the percent of students meeting proficiency on the state assessment among students who consistently used Zearn<cms-citation-text>[3]<cms-citation-text>. By contrast, among students who did not use Zearn, there was an eight percentage point decrease in students reaching proficiency.

This increase in students meeting proficiency was even greater for students in historically marginalized subgroups. Consistent Zearn usage resulted in a 22 percentage point increase in proficiency for multilingual learners.

<text-style-figure>FIGURE 3<text-style-figure>

Across all starting levels of proficiency, students with consistent Zearn usage were more likely to improve their proficiency level.

Student Mobility by Starting Score Level comparing by Zearn Math usage graph

The gains in math learning from consistent Zearn usage held true for students at every starting level of proficiency. Across all three starting proficiency levels, students with consistent Zearn usage were not only more likely than their similarly matched peers to move up a level on the state assessment, but also less likely to move down. Specifically, 63% of students who scored above proficiency in 2021 and did not use Zearn moved down a level on the spring 2022 state assessment, compared to only 38% of students who consistently used Zearn.

<text-style-figure>FIGURE 4<text-style-figure>

Students at the lowest level of math proficiency were three times more likely to improve their proficiency level when they used Zearn compared to students who did not use Zearn.

Mobility for Students at Lowest Proficiency by Zearn Math usage graph

Zooming into students who started the school year below proficiency, students who scored at the lowest level of math achievement in 2021 who consistently used Zearn during the 2021–2022 school year were three times more likely to score at a higher proficiency level in spring 2022 compared to their similarly matched peers who did not use Zearn. Thirty-eight percent of lowest-scoring students moved up at least one full level in 2022.

  1. This efficacy analysis used a two-step Coarsened Exact Matching (CEM) method with optimal matching to create a control group that was as similar as possible to the treatment group of consistent Zearn users. The treatment group was composed of students who completed three or more Zearn digital lessons each week during the 2021–22 school year. The control group was selected from other students in the district who completed an average of less than one lesson per week. Using CEM, treatment students were put into matching strata with control students that were in the same grade and within five scale score points in math and ten points in ELA on the spring 2021 state assessment. Then, within strata, treatment students were matched to control students with whom they shared at least four of seven other student characteristics.
  2. Zearn uses only de-identified data to identify insights and improve students’ learning experiences. All student-level data used in this study was provided by the district under the terms of a data-sharing agreement. For more information on how we protect student privacy, visit about.zearn.org/privacy. 
  3. For each subgroup in treatment and control, percent proficient in the fall was subtracted from percent proficient in the spring. If the percent proficient within a subgroup was the same in the Fall and Spring, the change listed in the chart would be 0.
Notes
  1. This efficacy analysis used a two-step Coarsened Exact Matching (CEM) method with optimal matching to create a control group that was as similar as possible to the treatment group of consistent Zearn users. The treatment group was composed of students who completed three or more Zearn digital lessons each week during the 2021–22 school year. The control group was selected from other students in the district who completed an average of less than one lesson per week. Using CEM, treatment students were put into matching strata with control students that were in the same grade and within five scale score points in math and ten points in ELA on the spring 2021 state assessment. Then, within strata, treatment students were matched to control students with whom they shared at least four of seven other student characteristics.
  2. Zearn uses only de-identified data to identify insights and improve students’ learning experiences. All student-level data used in this study was provided by the district under the terms of a data-sharing agreement. For more information on how we protect student privacy, visit about.zearn.org/privacy. 
  3. For each subgroup in treatment and control, percent proficient in the fall was subtracted from percent proficient in the spring. If the percent proficient within a subgroup was the same in the Fall and Spring, the change listed in the chart would be 0.

Additional Resources

Download Technical Appendix