When COVID-19 caused school closures in March 2020, it was clear that the spring would be difficult for teachers and students. Multiple studies have cited the missed learning that took place. Recognizing that missed learning did occur, Zearn researchers set out to investigate the effectiveness of different approaches taken this fall to recover that missed learning, asking “which approach to remediating missed learning is most effective to catch kids up?” Zearn worked with Steve Levitt and his team at University of Chicago's Center for RISC (Radical Innovation for Social Change) to design an analysis that would offer insights on this timely question.
Researchers investigated 5th grade usage data on Zearn’s digital platform for over 10,000 students across two different groups of schools, both of whom missed critical 4th grade content last year covering decimal fractions. The first group is schools that took a heavier approach to remediation for 5th graders this fall, covering a broader amount of 4th grade content in Zearn before moving to 5th grade, grade-level content. The second group is those that took a more targeted approach, following Zearn’s recommended learning recovery sequences with a targeted amount of foundational, 4th grade content, before grade-level learning. Researchers compared their progress to date this school year, as well as the rate of student struggle on that content.
Average on-grade-level lessons completed per-student in fall 2020, for students missing G4 fraction content in Spring 2020
The chart above shows grade-level progress for those two groups of schools. The blue bar is schools that took a heavier approach to remediation for 5th graders this fall, while the purple bar is those who followed Zearn’s targeted learning recovery sequences before grade-level learning. Through the fall, students in classrooms following Zearn’s learning recovery sequences completed nearly two more lessons than those in classrooms with heavier remediation.
Rate of student "struggle" on G5 fraction content, based on classroom approach to remediation (benchmarked to 2018-19 levels), for students missing G4 fraction content in Spring 2020
In addition, researchers compared the “rate of struggle” on early 5th grade content for the two groups of schools, defined by the frequency of Tower Alerts, an indication of struggle, as students progress through content. This is benchmarked to the rate of struggle for the two groups in the 2018-19 school year. In 2019-20, the rate of struggle rose for both groups slightly, which is not abnormal year-to-year variance. But in the 2020-21 school year, those lines notably diverge – those that were over-remediated actually saw even higher rates of struggle, while those with more targeted remediation dropped by a statistically significant amount, beyond what is normally expected in year-to-year variance. And the pattern was consistent across low-, mid-, and high-income schools.
What can we learn from this data? The first finding is unsurprising - those remediating with more prior grade-level content subsequently make less progress on grade-level content. The second finding, however, is important and perhaps surprising. Over-remediating students appears to be harmful, not just in that it prevents students from catching up, but in that it may confuse them more, which we see in higher struggle data.