In an editorial today, entitled “Maryland offers up bogus reading scores,” the editorial board of the Washington Post opines that “Maryland schools have artificially inflated their performance” on the National Assessment of Educational Progress, commonly called the “Nation’s Report Card.”
Schools in the state did this, the paper writes, by excluding—at a rate much higher than any other state in the nation—students who had learning disabilities or limited English proficiency (in other words, students who have individualized education plans or IEPs) from the reading tests administered to fourth and eighth graders. The editorial fails in three important ways:
- NAEP selects test takers randomly, based on demographics and other parameters
- Schools do not “cull” the list, as the editorial implies, based on disability status
- The IEPs are designed to be applied consistently throughout the year
The editorial correctly asserts that if students who had IEPs had taken the reading test, which is administered in odd-numbered years to fourth and eighth graders from every state, the average scores would have been lower. The exact number is somewhat disputed and, really, only available to NAEP officials, who don’t release these hypothetical numbers to the public. But yes, the score would have been lower if more learning-disabled or limited-English students had taken the test.
It also characterizes a memo from Dr Lillian Lowery, Maryland state superintendent of schools, as essentially “giving a green light” to schools to exclude students with certain indications on their IEPs that can’t be accommodated during the test. Specifically, the “read-aloud” accommodation, which allows a student taking a test to have it read to him by a teacher or computer-simulated voice, is not allowed for NAEP during the reading test, but several students in the state have this accommodation on their IEPs.
As anyone who has a special education student with an IEP knows, this is not a simple document. It is all but negotiated between the parents, the principal, special educators, regular classroom teachers, and possibly a school counselor, speech therapist, or other school officials (depending on the nature of the disability). Appointments are set up, and it is determined which accommodations a student needs in order to perform fairly on tests. For example, some students have an accommodation on their IEPs for calculator use: they get to use calculators, even on tests that would normally prohibit calculators. NAEP has a way around the calculator accommodation, because it just has those students take a special form of the math test that is designed to accommodate students who are using calculators.
But the read-aloud accommodation during the reading test is different (read-aloud is permitted by NAEP on the math tests). The NAEP people haven’t developed any special form of the test for use with the read-aloud accommodation—and I’m not sure they can, since we’re testing kids’ reading ability—so Dr Lowery simply advised schools to exclude randomly selected students who happen to have a read-aloud accommodation in their IEPs. The state’s exclusion rate has been high for some time, but lately, it has gone up even higher compared to national averages. This has made Maryland the center of issue statements from researchers and NAEP officials, which is something the state needs to address. The scores help states understand how their state-specific tests and learning standards compare to those used in other states.
However, the teaser for the editorial, which says, “We’re No 11,” the place the state would be in if read-aloud kids had hypothetically taken the test without the read-aloud accommodation, which they didn’t, because the hypothetical sample bears absolutely no resemblance whatsoever to the way NAEP is actually administered, but if they had, is a bit over the top—sensational writing, great editorializing, but not exactly on the mark. That’s not how NAEP works, and it’s certainly not how educators in Maryland treat kids with IEPs, an approach for which the state has received high praise.
The NAEP has no meaning to our schools, so any conclusions drawn in the editorial about school quality, of which there are a few, do not follow from any accusations, true or false, about NAEP. “Gov Martin O’Malley (D) and the man he has endorsed to succeed him, Lt Gov Anthony G Brown, are fond of boasting” about Maryland schools’ top ranking in the Education Week magazine, which the state has held for five years in a row, the editor writes. I don’t know how “fond” these two politicians are of saying that, but school officials that I know aren’t affected by it too much. It’s mostly based on state-level operations.
The schools of Maryland, some of which are called “mediocre” and “poor” by the editorial, are responsible to their students, not to the editors of Education Week. Kids in these schools, as well as the great ones in the state, have IEPs, and it gets a little confusing, I would imagine, to figure out which tests should use the IEP and which tests shouldn’t. For state and local tests, on which the kid receives a score, the IEP must be enforced. The NAEP, however, doesn’t give scores to kids, so the accommodations may not need to be enforced from a legal point of view. I’m not a lawyer, and the editorial correctly points out that no actual legal opinion can be produced on this matter since the state hasn’t sought one.
But that’s kind of the point. Education Week may give governors and gubernatorial hopefuls something to beat their chests about, but school officials are generally too busy running the schools to seek out legal opinions about things that have no effect whatsoever on the state’s classrooms, schools, or students. The instructions from Dr Lowery essentially allowed schools to keep the test-taking behavior consistent for students with IEPs, which allows the students to remain a little more comfortable with tests. Some test scores count, after all. NAEP doesn’t. Not for the schools.
And another thing, I am against the whole idea of ranking states, except in what I would call “friendly” competition. NAEP is even less meaningful under the Common Core, since kids in every state will be taking the same standardized tests and using the same learning standards. We should focus on collaboration, not competition, since all competition does, as is plainly obvious in the Post’s editorial, is cause people to try to knock down the top performers by bringing in mostly irrelevant, hypothetical arguments. They should rather pay attention to what those top performers do well and try to emulate it. There are bad schools in Maryland, and there’s no point in trying to pretend there aren’t. But other schools in the state are obviously doing a top-notch job, and schools in Minnesota, the top performer on the fourth-grade NAEP mathematics test, are doing a great job as well.
Instead of trying to knock Maryland and Minnesota off their high positions, which isn’t really the point anyway, except, apparently, for governors of those states, schools need to make choices about how they spend their time. Time should be spent on improving and seeking excellence, not on explaining hypothetical test-taking samples and state rankings by newspapers or magazines.
Unfortunately, online, NAEP produces a list of states that are ranked from highest-performing to lowest-performing. The website is careful to point out that states are grouped and that differences between the top state and the one in second place, and so on, may not be statistically significant. The Post should advise people to read the fine print in the “footnotes” to these lists before entering an opinion. I’m sure the state knows which schools are mediocre and poor, and in what ways, and it’s dealing with those schools. A newspaper should focus on independently reporting and entering opinions about the facts, once those facts are understood by the writers.