Friday, November 27, 2020
US flag

Pearson misreports scorecards on Va. alternate test

-

Pearson, the largest education and testing company in the world, admitted last month that about 4,000 students in Virginia who took an alternate form of the state’s tests received an incorrect scorecard, the Washington Post reported two days ago.

Virginia

The alternate form of standardized tests is available, under No Child Left Behind, only to the lowest 1 percent of the state’s students—those with severe cognitive impairment. Students who can use the alternate form used to contribute work throughout the school year and submit a “portfolio” for evaluation, which sometimes included state-approved worksheets, videos of the student doing an assignment, and so on. Only a certain number of students within a school system are eligible to take the alternate form under No Child Left Behind. The manual for Virginia’s alternate assessments is available from the state’s Department of Education, here.

What happened in Virginia is, while the tests had been evaluated at the school level, this year the state decided to make a contract with Pearson to evaluate the student portfolios, lifting some of the testing burden off the state’s teachers. Although Pearson says the company scored the portfolios correctly, they made an error in translating the raw scores to the marks of fail, proficient, and advanced, which get reported to parents and to the schools.

Every one of these kids received a passing score, when thousands of them had actually failed on the portfolio assessment. In our attempt to confirm this story, we came across a statement on Pearson’s own website:

We sincerely apologize for the inconvenience this has created for students, parents, schools and school divisions as well as to our colleagues at the Virginia Department of Education. We understand the importance of accurate reporting and know that the school divisions are now working very hard to make internal adjustments to their scores before reporting them to parents and the state. … The correct performance levels for all students participating in the VAAP are now available. We are in communication with the state’s division directors of testing and have offered our support to help them provide information to the impacted families.

Editorial

Thus, it seems to us to be a credible report. So, assuming this error in reporting occurred, there are two camps in response to it:

  • This is another blunder that shows how bad standardized testing is for our schools
  • No actual kid was harmed since teachers know the kids failed long before the scores come back

If you’re in the first camp, you say things like Diane Ravitch:

Why don’t we trust teachers to write their own tests and grade them so they can give timely help to students who need it? … The education-industrial complex errs again. With no accountability, no consequences for failure.

The answer to Ms Ravitch’s first question, although it may have been a rhetorical one, is that teachers have shown themselves to be incapable of developing tests that can be used on anyone but their own students. Let me be clear: teachers can write tests just fine, as long as they know what there is to know about the population of students who will take the test. If the population is variable or unknown, as it is in a statewide assessment, teachers are incapable of developing “fair” questions or, in most cases, of developing fair scoring rubrics. They all want to score the tests based on how they teach their own kids, failing to account for differences in teaching in other parts of the state or the country.

I can’t tell you how often I’ve supervised a rangefinding committee and felt good about the scores assigned, when one of the teachers on the committee will speak up and say, “Well, in my classroom, I would never give this kid a point for this garbage answer.” With all respect to what those teachers do day in and day out—and their inclusion on a statewide rangefinding committee means they are among the state’s top educators—standardized tests aren’t what they use in their classrooms. Standardized tests follow a few different rules that classroom teachers don’t have time to study. They should be too busy educating their own students.

Anyway, the premise of Ms Ravitch’s question assumes that testing is necessary, and it is. But her argument is capricious and a little oversimplified. First, teachers can “give timely help to students who need it” without using any standardized tests at all. At least I hope they can. The people who need the tests aren’t working in a classroom; they’re working in the state education department or in Washington. Teachers, almost certainly, do not need standardized tests in order to give low-cognitive students the help they need.

Second, Ms Ravitch’s claim that there will be “no consequences for failure” is inaccurate—not extremely inaccurate, but a little inaccurate. The state of Virginia is already considering financial penalties, which was part of the original article Ms Ravitch cited from the AP. It said, “Virginia education officials plan to meet with Pearson representatives this week to discuss the problems and whether the state might receive any financial compensation.” As far as “consequences” like putting Pearson out of business go, that’s not going to happen, nor does anyone really want that end.

Corporations are accountable not to Ms Ravitch, not to you and me, but to their shareholders and customers. As part of the public, I guess I count a little, but Pearson’s real loyalty has to be to the state government of Virginia in this case and then, ultimately, to its shareholders who stand to lose value as a result of this fiasco.

Now, if you’re in the other camp, you might write on Scathing Purple Musings and other blogs:

Screw-ups in Florida, Minnesota, New York, and now Virginia. … Isn’t it clear this company cannot handle the contracts it is given? … Isn’t it time states stop giving this company money to do the job that teachers can do and used to do? … Teachers used to review these portfolios in Virginia, but because of “new standards,” Pearson was “given the job” of reviewing the student work and scoring it.

Unlike what Ms Ravitch wrote, these aren’t statements in support of Virginia’s students or teachers; they are largely attacks on Pearson itself. Comments like this make it sound as though people would rather give these contracts to inexperienced, mom-and-pop operations rather than work with Pearson, which has established an admirable track record of working with a diverse array of states and districts, delivering tests.

I say it’s better to work with people who care about the quality of the tests—even if it is, in part, to pad their bottom line—than to throw the baby out with the bathwater.

Let’s address the questions posed here, because they are common in propaganda. Isn’t it clear Pearson can’t handle the contracts it is given? Well, if Pearson can’t handle the contracts, it’s going to be a tall order to find someone who can. I don’t necessarily agree Pearson can’t handle the contracts, but let’s do a reality check here: There are only a handful of major players in this business, and Pearson leads the pack.

When it comes to “doing the job teachers can do,” let me tell you what I have observed. The blogger quoted above offers no proof that teachers “can” do the job of massive testing protocols on a multistate level, but I offer this as proof that they cannot: How many test booklets have states found in school closets, where the “teachers” or “administrators” at the “school” completely disregarded the laws of their state by not turning in the test booklets on time? More than you can count! We had hundreds of books in Maryland this year alone that weren’t returned on time. This blatant disregard for the law makes it clear that teachers wouldn’t be very good at carrying out multistate testing protocols.

So, not only are teachers not going to be able to handle multistate testing protocols, but I would submit, we don’t want to waste their time with something that is so vastly unimportant to our children’s education. I would rather test the teachers to make sure they can deliver instruction properly in the subjects they teach and to the kids who are sitting in their classrooms than give kids even one single test that is biased because it was written without a statewide focus. This would lead to a system where we could trust the teachers about their own students after accepting some standardized form of teacher credibility.

Ultimately, then, I agree with the premise of this argument: the scores coming back from Pearson aren’t really that big a deal, and the mistakes weren’t actually consequential for any student. Teachers know best, and if we could just get rid of these pesky tests, we’d see some real learning in our special-ed classrooms.

Paul Katulahttps://news.schoolsdo.org
Paul Katula is the executive editor of the Voxitatis Research Foundation, which publishes this blog. For more information, see the About page.

Recent Posts

Total shutdown in Baltimore Co.; cyberattack

0
Criminals broke into the network at the 115,000-student district and scrambled files. Schools are closed for remote learning for now.

An Iowa City show choir goes virtual

Biden officially begins transition

Alton Sr. H.S. orchestra plays Bach

Noguchi sculpture added to Rose Garden