Paul Leroy of Bel Air, Md., writes a letter in Tuesday’s Baltimore Sun, published online here, asking about confusing messages coming from education professionals about the Maryland School Assessment (MSA), which is required under the No Child Left Behind law for every student in grades three through eight in math and reading. He writes (excerpt):
What the public and students are being told is, “We don’t want students taking the MSA tests because we are preparing them to take harder tests; and therefore, they will not do as well as they usually do on the easier test.” What?
This retired teacher seems confused about the tests. While I have on a few occasions (see here and here) said why I believe students shouldn’t be taken out of class in March to sit for the MSAs in math and reading, my opinions do not address Mr Leroy’s question at all. I will therefore endeavor to respond to this letter below, because I believe keeping people informed about what schools are doing is critical during this time of great transition.
Criterion-referenced tests have a range of difficulty levels
The MSAs, like the upcoming tests from the Partnership for Assessment of Readiness for College and Career (PARCC), are criterion-referenced tests, sometimes described with the plural, “criteria.”
According to an entry on Wikipedia, “A criterion-referenced test is one that provides for translating test scores into a statement about the behavior to be expected of a person with that score or their relationship to a specified subject matter. Most tests and quizzes that are written by school teachers can be considered criterion-referenced tests. The objective is simply to see whether the student has learned the material. Criterion-referenced assessment can be contrasted with norm-referenced assessment …”
For the MSAs, the “specified subject matter” is the state’s learning objectives or standards, which used to be called the “Voluntary State Curriculum,” in reading and math at each grade level. They test whether a given student has “learned the material” that the state asked teachers to teach when it published the baseline learning standards almost a decade ago.
Now Maryand has new standards, the ones found in the Common Core and adopted by the state in 2010.
Let me bring in an example. One learning standard in seventh-grade mathematics, under the old state curriculum, requires students to construct a perpendicular bisector of a line segment. This learning objective would be found in the red oval on the diagram above, since it’s not part of the Common Core for seventh grade.
A good criterion-referenced test would put a question or two on there making sure students could construct a perpendicular bisector of a line segment. Where Mr Leroy seems confused is here: I can write easy questions to test this skill, medium questions, and hard questions, and all would be perfectly aligned to the learning standard about perpendicular bisectors.
The MSA focuses lots of questions around the medium difficulty level, simply because when it comes to “accountability” under federal law, the government doesn’t care if a student has an “A+” mastery of the criteria or an “A” mastery of the criteria. All the state checks—and all the MSAs were designed to measure—is whether students can perform “at grade level” on the list of learning standards that would demonstrate compliance with federal accountability laws.
A big criticism of tests like the MSA is that they don’t reward kids for being excellent creative writers, for example. That’s true, and it is also perfectly predictable: The state government didn’t put “creatively write an essay” or something like that into the list of approved learning standards. The MSA isn’t designed to distinguish between kids at the extreme high end of the ability spectrum. Nor does the state care about distinguishing between kids at the extreme low end of the ability spectrum. There are plenty of other assessment instruments schools can use to get a much better and more precise reading at the extremes than the MSA.
The state only cares about guaranteeing to the federal government that students are performing “at grade level” according to the standards established and approved by the state.
Look at it like this: If I asked seventh graders in Maryland three years ago 100 questions about constructing perpendicular bisectors, with a range of difficulties, kids would score according to their mastery of perpendicular bisectors, because every single seventh-grade math teacher would have covered perpendicular bisectors. If none of a teacher’s students got the perpendicular bisector questions correct, that would provide evidence that the teacher didn’t teach her kids about perpendicular bisectors.
Scores will go down on the MSA
What’s happening now, though, is that the state is not asking seventh-grade math teachers to teach their students about perpendicular bisectors. When the state adopted the Common Core, perpendicular bisectors moved from seventh grade to high school geometry. The state has, officially, in effect, told seventh-grade math teachers not to teach their students anything about perpendicular bisectors.
Logically, the more time teachers spend on material in the blue oval in the diagram above, the less time they spend on the material in the red oval. Since the MSA tests students on the material in the red oval (including the purple overlap region, which varies by grade and subject), students’ scores can be predicted to go down. It goes like this: more time on blue → less time on red → lower scores on the MSA.
Let’s get back to our extreme example about the 100-question test on perpendicular bisectors. If we give that same test to students this year, scores would go down. The test itself would be just as “difficult” as it was three years ago, but the criteria are no longer part of what we’re asking teachers to teach. We cannot hold teachers or their students “accountable” for material that the state has asked them not to teach.
It’s not about the “difficulty,” although the tests from PARCC will be more rigorous—and therefore more difficult—but about the criteria being referenced and measured by the test.