District Snapshot

Students in Tulsa Public Schools who consistently used Zearn grew 1.5 grade levels during the 2021–2022 school year.

A new, district-wide study analyzing the impact of Tulsa Public Schools’ partnership with Zearn Math offers research-backed evidence that Zearn Math accelerates math learning.

14,000 K–6 students
79% Economically disadvantaged students
38% Multilingual learners
60% Black and Latino students

Researchers looked at how K-6 students in Tulsa Public Schools performed on the 2022 MAP Growth assessment. The research, which uses quasi-experimental matching techniques to isolate the impact of Zearn Math on student achievement, compares students who consistently completed three or more Zearn lessons each week with similarly matched students who did not consistently use Zearn<cms-citation-text>[1]<cms-citation-text>. Unlike a standard correlational analysis, this method allows differences in outcomes to be more confidently attributed to Zearn Math and not to other variables. Researchers found:

  • Students who used Zearn grew 1.5 grade levels in one school year, while their peers who did not use Zearn fell short of expected one-year growth benchmarks.
  • Across all starting MAP Growth achievement levels, students who used Zearn were more likely to improve their proficiency level<cms-citation-text>[2]<cms-citation-text>.
  • Students who started the school year with below average math achievement were 2.2 and 2.6 times more likely to improve their achievement level with consistent Zearn usage compared to students who did not use Zearn.
  • Consistent Zearn usage resulted in a double-digit increase in proficiency for Black and Latino students, economically disadvantaged students, and multilingual learners<cms-citation-text>[3]<cms-citation-text>

<text-style-figure>FIGURE 1<text-style-figure>

Students who used Zearn grew 1.5 grade levels in one school year, while other students fell short of expected one-year growth<cms-citation-text>[4]<cms-citation-text>.

<text-style-figure>FIGURE 2<text-style-figure>

Across all starting achievement levels, students with consistent Zearn usage were more likely to improve their achievement level in 2022.

Student Mobility by Starting Score Level by Zearn Math Usage

Students with consistent Zearn usage were also less likely to move down a level: 39 percent of students who met Tulsa Public Schools’ benchmark for proficiency on the fall 2021 MAP Growth assessment who did not use Zearn scored below average on the spring 2022 assessment, compared to only 11 percent of students who used Zearn.

<text-style-figure>FIGURE 3<text-style-figure>

Students who started with below average math achievement were 2.2 and 2.6 times more likely to improve their achievement level when they consistently used Zearn. 

Student Mobility for Students at Lowest Level of Math Proficiency by Zearn Math usage graph

<text-style-figure>FIGURE 4<text-style-figure>

Consistent Zearn usage resulted in a double-digit increase in proficiency for each student subgroup, while similarly matched students experienced declines.

Percent Point Change in Students Reaching Proficiency on the MAP Assessment from Fall 2021 to Spring 2022 graph

Between fall 2021 and spring 2022 consistent Zearn usage resulted in a 20 percent increase in students meeting Tulsa Public Schools’ benchmark for proficiency on the MAP Growth assessment, while there was a four percent decline in students meeting proficiency among students who did not use Zearn. This increase in students meeting proficiency held true for each student subgroup. Black and Latino students experienced a 27 percent increase in proficiency after using Zearn during the 2021–2022 school year. In comparison, Black and Latino students who did not use Zearn experienced a 13 percent decline in proficiency.

Endnotes

  1. This efficacy analysis used a two-step Coarsened Exact Matching (CEM) method with optimal matching to create a control group that was as similar as possible to the treatment group of consistent Zearn users. The treatment group was composed of students who completed three or more Zearn digital lessons each week during the 2021-22 school year. The control group was selected from other students in the district who completed an average of less than one lesson per week. Using CEM, treatment students were put into matching strata with control students that were in the same grade and within 5 national percentile points on fall Math & ELA scores. Then, within strata, treatment students were matched to control students with whom they shared at least four of seven other student characteristics: school, gender, race/ethnicity, special education status, multilingual learner status, gifted and talented status, and economic disadvantage.
  2. MAP Growth is a mean-centered test with no absolute bar for proficiency. The district in which this analysis was conducted defined proficiency as at or above the 50th percentile nationally and set annual district goals around students meeting this benchmark. Throughout this report, references to percent proficient will be based on the district definition of 50th percentile and above.
  3. Zearn uses only de-identified data to identify insights and improve students’ learning experiences. All student-level data used in this study was provided by Tulsa Public Schools under the terms of a data-sharing agreement. For more information on how we protect student privacy, visit about.zearn.org/privacy.
  4. MAP Growth quintile categories have been collapsed into three achievement categories for analysis. “Above standard” encompasses students who scored in the “High” and “High Average” quintiles. “At standard” encompasses students who scored in the “Average” quintile. “Below standard” encompasses students who scored in the “Low Average” and “Low” quintiles.
Notes

Endnotes

  1. This efficacy analysis used a two-step Coarsened Exact Matching (CEM) method with optimal matching to create a control group that was as similar as possible to the treatment group of consistent Zearn users. The treatment group was composed of students who completed three or more Zearn digital lessons each week during the 2021-22 school year. The control group was selected from other students in the district who completed an average of less than one lesson per week. Using CEM, treatment students were put into matching strata with control students that were in the same grade and within 5 national percentile points on fall Math & ELA scores. Then, within strata, treatment students were matched to control students with whom they shared at least four of seven other student characteristics: school, gender, race/ethnicity, special education status, multilingual learner status, gifted and talented status, and economic disadvantage.
  2. MAP Growth is a mean-centered test with no absolute bar for proficiency. The district in which this analysis was conducted defined proficiency as at or above the 50th percentile nationally and set annual district goals around students meeting this benchmark. Throughout this report, references to percent proficient will be based on the district definition of 50th percentile and above.
  3. Zearn uses only de-identified data to identify insights and improve students’ learning experiences. All student-level data used in this study was provided by Tulsa Public Schools under the terms of a data-sharing agreement. For more information on how we protect student privacy, visit about.zearn.org/privacy.
  4. MAP Growth quintile categories have been collapsed into three achievement categories for analysis. “Above standard” encompasses students who scored in the “High” and “High Average” quintiles. “At standard” encompasses students who scored in the “Average” quintile. “Below standard” encompasses students who scored in the “Low Average” and “Low” quintiles.

Additional Resources

Download study
Download technical appendix