We have reviewed the two sample items in grade 3 mathematics released to the public in early November by the Partnership for Assessment of Readiness for College and Careers (PARCC), a multi-state testing consortium to which both Maryland and Illinois belong. The work is sloppy and appears thrown together without much review. There are still a few months for PARCC to get its act together before we send these poorly designed tests to computer screens in front of our third graders, whose average age will be 8 when they see items like these.
Grade 3: Patricia’s Reading Time
Link (PDF). CCSSM.Content.3.MD.1-2 Solve word problems involving addition and subtraction of time intervals in minutes, e.g., by representing the problem on a number line diagram.
Item Quality: Good (task is clear and student is free to respond)
Format: Type the answer (technology required)
Cognitive Demand: Medium (several related operations)
Scoring: Right-or-wrong (no partial credit possible)
Alignment: Partial (task does not address the entire standard)
Factors that undermine item quality and usefulness:
(1) No partial credit is allowed. It is possible a student could add all three numbers and forget to subtract the sum from 120. This would result in a score of 0, despite the fact that the student was able to complete part of the task. Furthermore, a simple computation error is just as bad as a flaw in the solution strategy. This provides limited information back to teachers about instruction.
(2) The item is easily developed by a teacher for use with students in his or her classroom and requires no special input from PARCC. Nothing about this item requires the enhancements to technology PARCC requires from schools. In fact, since no partial credit is considered for students who show some understanding of how to complete the task, the item on a PARCC test, from which results will be received long after students have left the classroom of the teacher purportedly measured, is even worse.
Grade 3: Art Teacher’s Rectangular Array
Link (PDF). CCSSM.Content.3.OA.8 Solve two-step word problems using the four operations. Represent these problems using equations with a letter standing for the unknown quantity. Assess the reasonableness of answers using mental computation and estimation strategies including rounding. … But maybe, to some degree, Use multiplication and division within 100 to solve word problems in situations involving equal groups, arrays, and measurement quantities, e.g., by using drawings and equations with a symbol for the unknown number to represent the problem.
Item Quality: Poor (fails several principles)
Format: Type the answer, shade a grid, type an equation (technology required)
Cognitive Demand: Medium (several related operations)
Scoring: Analytical (1+1+1 = 3 total points)
Alignment: Partial (task does not address the entire standard: reasonableness)
Factors that undermine item quality and usefulness:
(1) 48 tiles can be arranged in a 12 × 4 array; however, the grid shown has a height of 10 and a width of 10, meaning it is impossible for the student to enter one possible correct answer. The student is warned fairly, I think, but the wording of the problem isn’t really the issue. Rather, suppose a student, before looking at the grid, plans a response of 12 × 4. He’ll try it and find he can’t do it, which wastes time and possibly leads to frustration. It would be better to warn the student more explicitly in the text that the dimensions of the wall are 10 × 10, rather than forcing third graders to count, which can be difficult for students with bad vision. Furthermore, counting to 12 isn’t part of this standard or any third-grade standard. Estimation is, however, and it is perfectly within reason to estimate that there are 12 grid boxes across here.
(2) Arrays of 24 × 2 and 48 × 1 aren’t possible either. Only a 6 × 8 array is correct, and that’s only one-fourth of all possible correct answers from a mathematics perspective. This is an artificial constraint imposed from outside of mathematics (by the dimensions of the wall). Part B therefore doesn’t address the learning standard itself because of an external constraint that has nothing to do with mathematics.
(3) An addition mistake in Step A could result in a prime number being obtained as a sum, which would make it impossible to enter an answer for Step B. The scoring notes for the item suggest that propagation of errors will only be penalized in Step A, which is good, but certain errors, when propagated, make completion of Step B impossible, and the student is thus double penalized for one mistake.
(4) Part C requires students to enter an equation, which is fine at the third-grade level, but a required symbol, the multiplication sign, ×, is not available in the text area where the response is to be entered. It is not intuitive that the students should use a lowercase ‘x’, a capital ‘X’, or some other keystroke to represent the × sign. I suppose it’s a good thing the prompt doesn’t require the student to use the variable X as the number of rows, because that would make it difficult to score.
(5) Part C is an independent problem. Sure, it measures the same objective as the task developed in Parts A and B of the item, but it does not depend on or follow from Parts A and B. The task is disconnected as a result. We would prefer to develop multi-part tasks, rather than two separate items, since it requires more focused thinking on the part of the problem solver and is called for in the learning standard.
More item reviews will follow on these pages as we complete them.