District Snapshot

Students who consistently used Zearn grew 1.3 grade levels in one year of learning.

A new study of the impact of consistent Zearn usage on student math achievement in a large urban district in the South provides evidence that Zearn Math helps accelerate math learning for all students.

80K+ K-8 Students (18K+ Grade 4-6 Students)
51% Economically disadvantaged students
33% Black and/or Latino students
15% Students in special education

Researchers looked at student academic growth for students in grades 4–6 between the spring 2021 and spring 2022 state assessment. The research, which uses quasi-experimental matching techniques to isolate the impact of Zearn Math on student achievement, compared a sample of 1,111 students who completed three or more Zearn lessons each week with a control sample of 1,111 matched students who did not consistently use Zearn<cms-citation-text>[1]<cms-citation-text>. Unlike a standard correlational analysis, this method allows differences in outcomes to be more confidently attributed to Zearn Math and not to other variables.

Researchers found:

  • Students who consistently used Zearn Math scored an average of 15 scale score points higher on the spring 2022 state assessments, equivalent to more than 1.3 grade levels of math learning in one year.
  • Zearn’s impact was even greater for students who scored below standard in spring 2021; students who previously scored below standard grew 1.7 grade levels in one school year, while matched students who did not use Zearn experienced declines.
  • Black and/or Latino students, economically disadvantaged students, and students in special education who consistently used Zearn exceeded the state benchmark for one-year growth, while matched students who did not use Zearn fell short of expected growth<cms-citation-text>[2]<cms-citation-text>.
  • Across all starting achievement levels, students with consistent Zearn Math usage were more likely to improve their achievement level in 2022 than students who did not use Zearn Math. Students who scored at the lowest levels of math achievement were more than four times more likely to improve their achievement level when they consistently used Zearn Math.

<text-style-figure>FIGURE 1<text-style-figure>

Students who consistently used Zearn Math scored an average of 15 scale score points higher on the spring 2022 state assessment.

<text-style-figure>FIGURE 2<text-style-figure>

Consistent Zearn Math usage resulted in significant score growth across all starting levels of proficiency.

<text-style-figure>FIGURE 3<text-style-figure>

Students who consistently used Zearn Math grew 1.3 grade levels in one school year. Students who previously scored below standard gained 1.7 years of math learning, while matched students who did not use Zearn experienced declines.

Across starting proficiency levels, students who consistently used Zearn Math during the 2021–2022 school year exceeded state benchmarks for expected one-year growth on the state assessment, while students who did not use Zearn fell short of expected growth<cms-citation-text>[3]<cms-citation-text>. Students who scored below standard in 2021 and consistently used Zearn Math experienced an additional 2.1 years of growth in math learning compared to matched peers, who fell behind 0.4 years during the same school year.

<text-style-figure>FIGURE 4<text-style-figure>

Black and/or Latino students, economically disadvantaged students, and students in special education experienced significant score growth with consistent Zearn Math usage.

Achievement gains also held true across student subgroups. Black and/or Latino students, economically disadvantaged students, and students in special education who consistently used Zearn during the 2021–2022 school year outscored matched peers on the spring 2022 state assessment.

<text-style-figure>FIGURE 5<text-style-figure>

Black and/or Latino students, economically disadvantaged students, and students in special education who consistently used Zearn Math exceeded the state benchmark for one-year growth, while matched students who did not use Zearn fell short of expected growth.

Across student subgroups, consistent Zearn usage resulted in more than a year of math learning gains. Economically disadvantaged students who consistently used Zearn Math grew 1.4 grade levels in one school year, while matched students fell behind 0.4 years between spring 2021 and spring 2022.

<text-style-figure>FIGURE 6<text-style-figure>

Across all starting achievement levels, students with consistent Zearn Math usage were more likely to improve their achievement level in 2022.

Looking beyond score growth and years of learning, students who consistently used Zearn Math were more likely to move up at least one achievement level on the state assessment. Students with consistent Zearn usage were also less likely to move down a level: 58 percent of students who scored “Satisfactory,” or at standard, on the spring 2021 state assessment and did not use Zearn moved down one or more levels on the spring 2022 assessment, compared to just 14 percent of students who used Zearn.

<text-style-figure>FIGURE 7<text-style-figure>

Students who scored at the lowest levels of math achievement in spring 2021 were more than four times more likely to improve their achievement level when they consistently used Zearn Math.

Zooming into students who started the year below standard, students who scored at the lowest two achievement levels in 2021 and consistently used Zearn Math during the 2021–2022 school year were more than four times more likely to score at a higher achievement level in spring 2022 compared to students at the same starting levels who did not use Zearn. Forty-two percent of students who scored “Inadequate” on the spring 2021 state assessment moved up at least one level in spring 2022, compared to just nine percent of students at the same starting level who did not use Zearn. Similarly, 52 percent of students who scored “Below Satisfactory” in 2021 moved up a level or more with consistent Zearn usage, compared to just six percent of matched peers who did not use Zearn.

ESSA Tier 2 Efficacy Research

This study was designed to meet required rigor of the What Works Clearinghouse (WWC) standards for quasi-experimental research and to meet the criteria for an ESSA Tier 2 (Moderate) rating in the ESSA guidelines for evidence-based interventions. The study uses quasi-experimental matching methods to create baseline equivalency between treatment and control groups along major confounding factors. Reference the full technical appendix to learn more.

  1. This efficacy analysis used a two-step Coarsened Exact Matching (CEM) method with optimal matching to create a control group that was as similar as possible to the treatment group of consistent Zearn users. The treatment group was composed of students who completed three or more Zearn digital lessons each week during the 2021-22 school year. The control group was selected from other students in the district who completed an average of less than one lesson per week. Using CEM, treatment students were put into matching strata with control students that were in the same grade and within five scale score points on both the math and ELA spring 2021 state assessment. Then, within strata, treatment students were matched to control students with whom they shared at least four of seven other demographic and academic characteristics: school, gender, race, free/reduced lunch eligibility, multilingual learner status, special education status, and gifted status.
  2. Zearn uses only de-identified data from our platform to identify insights and improve students’ learning experiences. All student-level data used in this study was provided by the partner district under the terms of a data-sharing agreement. For more information on how we protect student privacy, visit about.zearn.org/privacy.
  3. Yearly growth expectations were calculated based on the change in scale score points needed for students to maintain the same achievement level on the state assessment on subsequent tests. Expected growth varied by students’ grade and starting Achievement Level. On average, students were expected to grow 11.2 scale score points between spring 2021 and spring 2022. For more details, see the Technical Appendix.
Notes
  1. This efficacy analysis used a two-step Coarsened Exact Matching (CEM) method with optimal matching to create a control group that was as similar as possible to the treatment group of consistent Zearn users. The treatment group was composed of students who completed three or more Zearn digital lessons each week during the 2021-22 school year. The control group was selected from other students in the district who completed an average of less than one lesson per week. Using CEM, treatment students were put into matching strata with control students that were in the same grade and within five scale score points on both the math and ELA spring 2021 state assessment. Then, within strata, treatment students were matched to control students with whom they shared at least four of seven other demographic and academic characteristics: school, gender, race, free/reduced lunch eligibility, multilingual learner status, special education status, and gifted status.
  2. Zearn uses only de-identified data from our platform to identify insights and improve students’ learning experiences. All student-level data used in this study was provided by the partner district under the terms of a data-sharing agreement. For more information on how we protect student privacy, visit about.zearn.org/privacy.
  3. Yearly growth expectations were calculated based on the change in scale score points needed for students to maintain the same achievement level on the state assessment on subsequent tests. Expected growth varied by students’ grade and starting Achievement Level. On average, students were expected to grow 11.2 scale score points between spring 2021 and spring 2022. For more details, see the Technical Appendix.

Additional Resources

Download study
Download technical appendix