Friday, July 3, 2020
US flag

Testing on computer sometimes affects your score

A recent study found only marginal differences in performance for elementary and middle school students who took standardized tests on computer, compared to their performance on paper-and-pencil versions of those same tests, suggesting that it’s the test-delivery platform, which might not allow students to skip around or return to previously-answered questions, such as those on so-called “computer adaptive tests,” that produce lower scores, while just taking tests on computer may have only a slight impact on the scores.

Based on a study of more than 30,000 elementary, middle, and high school students conducted in winter 2015-16, researchers found that elementary and middle school students scored lower on a computer-based test that did not allow them to return to previous items than on two comparable tests—paper- or computer-based—that allowed them to skip, review, and change previous responses.

Elementary school students scored marginally higher on the computer-based exam that allowed them to go back to previous answers than on the paper-based exam, while there was no significant difference for middle school students on those two types of tests.

In contrast, high school students showed no difference in their performance on the three types of tests. Likewise, previous research has found that the option to skip, review, and change previous responses also had no effect on the test results of college students.

For the study, tests were given to students in grades 4–12 that assessed their understanding of energy through three testing systems. Instructors elected to administer either the paper-and-pencil test (PPT) or one of two computer-based tests (CBT) based on the availability of computers in their classrooms.

One CBT (using TAO, an open source online testing system) allowed students to skip items and freely move through the test, while the other CBT (using the AAAS assessment website) did not allow students to return to previous test items. In addition, on the TAO test, answers were selected by directly clicking on the text corresponding to an answer. On the AAAS exam, answers were chosen more indirectly, by clicking on a letter (A, B, C, or D) at the bottom of the screen corresponding with an answer.

Gender was found to have little influence on a student’s performance on PPT or CBT; however, students whose primary language was not English had lower performances on both CBTs compared to the PPT. The cause for the difference depending on primary language was unclear, but could have been linguistic challenges that the online environment presented or limits on opportunities to use computers in non-English-speaking environments.

Overall, the study results, along with previous research, indicate that being able to skip, review, and change previous responses could be beneficial for younger children in elementary and middle school but have no influence on older students in high school and college.

Furthermore, results indicated that marking an answer in a different location on a multiple-choice test could be challenging for younger students, students with poor organizational skills, students who have difficulties with concentration, or students who are physically impaired. In addition, having to match an answer to a corresponding letter at the bottom of the screen likely adds an additional level of complexity and cognitive processing.

The researchers note that additional study of CBT answer-choice selection and test navigation features and how they influence elementary and middle school students’ test performance is warranted.

The study, reported by the American Educational Research Association (AERA), was supported by a grant from the Institute of Education Sciences.

Paul Katula is the executive editor of the Voxitatis Research Foundation, which publishes this blog. For more information, see the About page.

Recent posts

Voxitatis congratulates the COVID Class of 2020

2020 is unique and, for high school graduates, different from anything they've seen. Proms, spring sports, & many graduation ceremonies are cancelled. Time for something new.

Vertical addition (m3.nbt.2) math practice

3rd grade, numbers and operations in base 10, 2, 3-digit vertical addition practice problem

Rubber ducks (m3.oa.1) math practice

3rd grade, operational and algebraic thinking, 1, rubber ducky modeling practice problem

Distance learning begins as Covid-19 thrives

What we learn during & from coronavirus, a challenging & imminent crisis, will provide insights into so many aspects of our lives.

Calif. h.s. choir sings with social distancing

Performances with the assistance of technology can spread inspiration across the globe even as the coronavirus spreads illness and disease.

Families plan to stay healthy during closures

Although schools are doing what they can to keep students learning and healthy during the coronavirus outbreak, that duty now shifts to parents.

Illinois temporarily closes all schools

IL schools will be closed on Tuesday, March 17, through at least March 30. Schools in 18 states are now closed due to coronavirus.

Coronavirus closures & cancellations

Many schools are closed and sports tournaments cancelled across America during what the president called a national emergency: coronavirus.

Coronavirus closes schools in Seattle

The coronavirus pandemic has caused colleges to cancel classes, and now Seattle Public Schools became the nation's first large district to cancel classes due to the virus.

Most detailed images ever of the sun

A new telescope at the National Solar Observatory snapped the most detailed pictures of the sun's surface we have ever seen.

Feds boost Bay funding

Restoration efforts in the Chesapeake Bay watershed received a boost in federal funding in the budget Congress passed last month.