A new study from the American Institutes for Research shows that Massachusetts students who took the PARCC tests online in the first year scored lower than their peers who took the paper-based versions of those tests, Education Week reports.
The study was published this month in journal Economics of Education Review and considered data from the 2014-15 school year, the first year the state along with many others used tests developed by the Partnership for Assessment of Readiness for College and Career, or PARCC.
In Ohio, a survey of districts found that 85% of districts administering PARCC on paper received an “A” grade, compared to only 17% of online districts. In Illinois, 43% of students who took PARCC on paper scored proficient or above in ELA, compared to 36% of students who took the test online. In Maryland, middle school ELA students in Baltimore County scored worse on the online test after controlling for student background and prior achievement.
While previously noted differences might have resulted from differences among students, rather than a strict mode effect—although the statistical probability that there was “no” mode effect is near zero—the Massachusetts study took a different approach and looks strictly at the difference in mode of administration.
Their “estimates of mode effects in math and ELA represent extremely large changes in measured student learning: up to 5.3 months of learning in math and 11.0 months of learning in ELA in a 9 month school year,” authors Ben Backes and James Cowan write.
They note, however, that some of the loss in performance level from taking the test online is recovered in the second and subsequent years of test taking.