Thursday, November 14, 2019
US flag

Digital device choice affects test scores

Further research now underscores concerns testing experts have when it comes to scores obtained by students who take Common Core-aligned math and reading tests online, Education Week reports, citing assorted analyses from test providers and other organizations.

This is how kids do math. None of this is possible with the equation editor online tool.

Voxitatis reported several months ago that the scores of students who took Common Core-aligned tests in math, administered by the Partnership for Assessment of Readiness for College and Careers, or PARCC, online scored lower at the high school level than those who took the test on paper. We blamed the difference on the “equation editor,” a MathType derivative that requires students to select math symbols from numerous dropdown palettes and enter their math work in paragraph form, resembling neither how they learn about nor study nor complete homework nor understand mathematics in any way.

We quoted S James Gates, a physicist at the University of Maryland, a member of the Maryland State Board of Education, and a recipient of the National Medal of Science from President Barack Obama, as saying the equation editor was an inappropriate tool for doing the work of the PARCC tests in math.

Differences were not reported at all grade levels or for both subjects. Differences between online scores and scores on paper were most pronounced in high school math, but it wasn’t entirely clear that the differences could be attributed solely to the equation editor. That was just a hypothesis several people, including me, had shared over the past year or so.

Ultimately, the online tests were found to be mostly comparable to the paper tests, and we haven’t said much about the subject ever since. But one new report found negative “device effects” among students in Ohio who used tablets to take the tests. PARCC had originally excluded test scores from Ohio, where one out of about seven PARCC test takers lived, before computing the numbers, in effect declaring the entire state an outlier.

With the numbers from Ohio included in the new report, it’s now clear that students who took the PARCC tests on tablets scored an average of 10 points and 14 points lower, respectively, than those who took the tests on laptops or desktop computers.

Unlike the earlier studies, this one didn’t examine score differences between online test takers and paper test takers. That is, it doesn’t replicate the earlier work. It does, however, reinforce the findings obtained previously. The fact that kids who take a test on one device get lower scores than kids who take the same test on a different device is inconvenient, mainly because the type of device the kid uses has nothing whatsoever to do with either the kid’s understanding of mathematics at the appropriate grade level nor the quality of his or her teachers.

As you can read for yourself, the report found that “extensive evidence of device effects was observed on nearly every assessment” when the results from Ohio, the largest state in PARCC at the time, were included.

I stand by my earlier remarks: Students’ familiarity with online test-taking tools like the equation editor is completely beside the point. I don’t care if our students learn how to use some contrived MathType look-alike, which isn’t even a good way of writing mathematical expressions at the high school level, let alone a fair way of holding students, teachers, or schools accountable. It is, rather, an amateurish online tool that bears no resemblance whatsoever to math at the high school level.

Any attempt on the part of schools to divert the public’s attention away from their own failures in the online testing realm and toward a made-up need to familiarize students with an online test-delivery tool reveals a motive to defraud students and communities. Schools should use technology for many good reasons; these tests aren’t one of those reasons—at least until a whole lot more research comes in and tells us what we’re dealing with.

I mean, there are variables that have an impact on student test scores—socioeconomic status, headaches, and so on—even in states that don’t start with the letter ‘O.’ Federal law requires us to test students once in high school and give each student a “score” for performance in math. The use of inadequate technology to assess that performance has both reduced the rigor for the problems on the test in order to accommodate low-end answer-entry tools and forced us to face the fact that we’re measuring more variables than we thought (or told the public about) besides actual math performance. Add device use and familiarity with specific and underdeveloped test-delivery tools to the list of variables. And tell people what’s happening.

Paul Katula is the executive editor of the Voxitatis Research Foundation, which publishes this blog. For more information, see the About page.

Recent posts

Mercury makes a transit; next in 2032

A transit of Mercury occurred today and was visible from the US, provided you had sunny skies. It was one of longest possible transits.

On the Naperville BWW racist incident

A racist incident at a Naperville, IL, sports bar indicates that the threads of racism are strong, perhaps as strong as ever.

IL bill could excuse absences to vote

A proposed law in IL could give students up to two hours during the school day so they could vote in the upcoming election.

Loan forgiveness gains some bipartisan support

One Republican from GA, who used to work under Betsy DeVos at the US Education Dept, offers a plan to forgive some student loan debt.

A band teacher is IL Teacher of the Year

IL named a band teacher the 2020 Teacher of the Year on Oct. 19. He individualizes music instruction and shares his work with 1000s.

‘Little Shop of Horrors’ bookends Halloween

Several high schools have decided to add a little spook to their musical stages in this season of Halloween. Music makes it happen.

New IL law ensures inclusion of LGBTQ+

A law will take effect next school year in IL that will require students to study LGBTQ history as part of the social studies curriculum.

MoCo doubles down on summer learning loss

Research is at least equivocal about summer learning loss, but maybe there's something to a new plan in Montgomery County, Md.

Downers North lights up the gym for Beth

Ongoing fundraising drives for a Downers Grove N. volleyball player killed by an intoxicated driver in Feb. are going strong in this western suburb.

High-payroll Yankees don’t make World Series

The World Series begins Tuesday, but some of the playoff games can teach us valuable things about youth sports, investment, etc.

Chicago teacher strike enters calendar week 2

Chicago teachers strike for the 3rd day Monday; the union wants smaller class sizes and support for paraprofessionals.