Thursday, November 14, 2019
US flag

Opinion: Stats go too far in medicine & education

Robert M Wachter writes in today’s New York Times that although we need to hold professionals, including teachers and doctors, accountable for their work, numbers mislead us into thinking that we have met our goals when, in fact, we’ve accomplished nothing of lasting value.

“Health care and education, have become increasingly subjected to metrics and measurements,” the professor and interim chairman of the department of medicine at the University of California, San Francisco, and the author of The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age, writes. But, “the focus on numbers has gone too far. We’re hitting the targets, but missing the point.”

Education is experiencing its own version of measurement fatigue. Educators complain that the focus on student test performance comes at the expense of learning. Art, music and physical education have withered, because, really, why bother if they’re not on the test?

He goes on to quote Avedis Donabedian, a now-deceased professor at the University of Michigan’s School of Public Health who developed what is known as Donabedian’s triad for measuring the quality of medical care. The triad holds that quality is measured by looking at

  • Outcomes (how the patients fared)
  • Processes (what was done)
  • Structures (how the work was organized)

Shortly before he died, Dr Donabedian was asked about his view of quality. “The secret of quality is love,” this “hard-nosed scientist” said, Dr Wachter wrote.

The same principle could be applied to education. We hear continuous messages from educators that test scores (the outcomes in Donabedian’s triad) need to be used in conjunction with other estimates of student understanding.

My sister just received individual student reports (ISRs) for the PARCC tests her kids took in third and fourth grade. The Illinois school district sent out the ISRs right along with their report cards, which were, of course—since PARCC results took nearly seven months to be distributed to schools and parents—much more current.

Side by side, these two reports showed both of them to be very good students in math and slightly better than average students in English. The PARCC results reinforced the report cards, but the report cards showed much more “love,” I guess we can call it, since written notes were far more personal than PARCC’s statistics-laden language.

My sister, to be honest, couldn’t understand what the language used on the PARCC ISRs was intended to mean, let alone how she should use the ISRs to help her kids in math or English. And my sister has a college degree from a major university, much more than the majority of parents have who will try to make heads or tails out of these PARCC ISRs this winter and next.

It’s just another example of measurements from standardized tests that have gone too far, of corporate giants like Pearson, which produces the PARCC ISRs, creating something useless for the majority of people it should be helping, people whom the state of Illinois is paying for it to help, actually. Thank goodness the school and my nephews’ teachers have the good sense to spread a little love in the form of a handwritten report card for 10-year-olds!

One commenter on the article writes this:

Two vital measurements have been too long ignored. First, does the provider have a complete and adequate understanding of all the patient’s past and present problems? It is impossible to provide the best care when blinded to the full set of moving parts. Second, and most important and difficult, has the provider negotiated goals for the management of all active problems with the patient? If you aim at nothing, you will hit it.

Unless and until these metrics are tested, we will have providers drowning in a sea of satisfaction surveys, computerized lists and the like (and leaving the job) just as the swelling ranks of the aging population crest on the steps of our offices and hospitals.

I echo this in education with regard to “personalized” instruction now being touted by so many for-profit education chains: Does the teacher have a complete and adequate understanding of all the student’s past and present learning goals and achievements? It is impossible to provide “personalized instruction” when blinded to the “whole” student, a set of moving variables that aren’t accounted for on standardized tests.

Second, and most important and difficult, has the teacher negotiated goals for the management of the “whole” student? If you aim at something that is not, by any definition, personalized instruction, you will hit it, just as lots of schools came quite close to achieving “100 percent proficiency” in math and reading under the old No Child Left Behind law.

Paul Katula is the executive editor of the Voxitatis Research Foundation, which publishes this blog. For more information, see the About page.

Recent posts

Mercury makes a transit; next in 2032

A transit of Mercury occurred today and was visible from the US, provided you had sunny skies. It was one of longest possible transits.

On the Naperville BWW racist incident

A racist incident at a Naperville, IL, sports bar indicates that the threads of racism are strong, perhaps as strong as ever.

IL bill could excuse absences to vote

A proposed law in IL could give students up to two hours during the school day so they could vote in the upcoming election.

Loan forgiveness gains some bipartisan support

One Republican from GA, who used to work under Betsy DeVos at the US Education Dept, offers a plan to forgive some student loan debt.

A band teacher is IL Teacher of the Year

IL named a band teacher the 2020 Teacher of the Year on Oct. 19. He individualizes music instruction and shares his work with 1000s.

‘Little Shop of Horrors’ bookends Halloween

Several high schools have decided to add a little spook to their musical stages in this season of Halloween. Music makes it happen.

New IL law ensures inclusion of LGBTQ+

A law will take effect next school year in IL that will require students to study LGBTQ history as part of the social studies curriculum.

MoCo doubles down on summer learning loss

Research is at least equivocal about summer learning loss, but maybe there's something to a new plan in Montgomery County, Md.

Downers North lights up the gym for Beth

Ongoing fundraising drives for a Downers Grove N. volleyball player killed by an intoxicated driver in Feb. are going strong in this western suburb.

High-payroll Yankees don’t make World Series

The World Series begins Tuesday, but some of the playoff games can teach us valuable things about youth sports, investment, etc.

Chicago teacher strike enters calendar week 2

Chicago teachers strike for the 3rd day Monday; the union wants smaller class sizes and support for paraprofessionals.