Tuesday, January 21, 2020
US flag

Opinion: Stats go too far in medicine & education

Robert M Wachter writes in today’s New York Times that although we need to hold professionals, including teachers and doctors, accountable for their work, numbers mislead us into thinking that we have met our goals when, in fact, we’ve accomplished nothing of lasting value.

“Health care and education, have become increasingly subjected to metrics and measurements,” the professor and interim chairman of the department of medicine at the University of California, San Francisco, and the author of The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age, writes. But, “the focus on numbers has gone too far. We’re hitting the targets, but missing the point.”

Education is experiencing its own version of measurement fatigue. Educators complain that the focus on student test performance comes at the expense of learning. Art, music and physical education have withered, because, really, why bother if they’re not on the test?

He goes on to quote Avedis Donabedian, a now-deceased professor at the University of Michigan’s School of Public Health who developed what is known as Donabedian’s triad for measuring the quality of medical care. The triad holds that quality is measured by looking at

  • Outcomes (how the patients fared)
  • Processes (what was done)
  • Structures (how the work was organized)

Shortly before he died, Dr Donabedian was asked about his view of quality. “The secret of quality is love,” this “hard-nosed scientist” said, Dr Wachter wrote.

The same principle could be applied to education. We hear continuous messages from educators that test scores (the outcomes in Donabedian’s triad) need to be used in conjunction with other estimates of student understanding.

My sister just received individual student reports (ISRs) for the PARCC tests her kids took in third and fourth grade. The Illinois school district sent out the ISRs right along with their report cards, which were, of course—since PARCC results took nearly seven months to be distributed to schools and parents—much more current.

Side by side, these two reports showed both of them to be very good students in math and slightly better than average students in English. The PARCC results reinforced the report cards, but the report cards showed much more “love,” I guess we can call it, since written notes were far more personal than PARCC’s statistics-laden language.

My sister, to be honest, couldn’t understand what the language used on the PARCC ISRs was intended to mean, let alone how she should use the ISRs to help her kids in math or English. And my sister has a college degree from a major university, much more than the majority of parents have who will try to make heads or tails out of these PARCC ISRs this winter and next.

It’s just another example of measurements from standardized tests that have gone too far, of corporate giants like Pearson, which produces the PARCC ISRs, creating something useless for the majority of people it should be helping, people whom the state of Illinois is paying for it to help, actually. Thank goodness the school and my nephews’ teachers have the good sense to spread a little love in the form of a handwritten report card for 10-year-olds!

One commenter on the article writes this:

Two vital measurements have been too long ignored. First, does the provider have a complete and adequate understanding of all the patient’s past and present problems? It is impossible to provide the best care when blinded to the full set of moving parts. Second, and most important and difficult, has the provider negotiated goals for the management of all active problems with the patient? If you aim at nothing, you will hit it.

Unless and until these metrics are tested, we will have providers drowning in a sea of satisfaction surveys, computerized lists and the like (and leaving the job) just as the swelling ranks of the aging population crest on the steps of our offices and hospitals.

I echo this in education with regard to “personalized” instruction now being touted by so many for-profit education chains: Does the teacher have a complete and adequate understanding of all the student’s past and present learning goals and achievements? It is impossible to provide “personalized instruction” when blinded to the “whole” student, a set of moving variables that aren’t accounted for on standardized tests.

Second, and most important and difficult, has the teacher negotiated goals for the management of the “whole” student? If you aim at something that is not, by any definition, personalized instruction, you will hit it, just as lots of schools came quite close to achieving “100 percent proficiency” in math and reading under the old No Child Left Behind law.

Paul Katula is the executive editor of the Voxitatis Research Foundation, which publishes this blog. For more information, see the About page.

Recent posts

Feds boost Bay funding

Restoration efforts in the Chesapeake Bay watershed received a boost in federal funding in the budget Congress passed last month.

Md. & IL bands perform on New Year’s in...

Bands from IL and Md. once again entertained thousands of people who lined the streets of London and Rome on New Year's Day.

Howard Co. sounds an under-staffing alarm

Teachers in a Md. district have filed a grievance over missing planning and lunch periods and, as a result, putting the most vulnerable students at risk.

Top 11 school stories of 2019

We find these 11 stories to have the greatest potential for influencing activity and direction in schools for the near future.

Girls’ volleyball champs in Illinois

We congratulate the Illinois state champions in girls' volleyball: Newark, St Teresa, Sterling, & Benet Academy.

A weekend of ‘band geeks’ across America

The musical Band Geeks was in performance at a MD high school, just as marching bands from across America named a national champion.

2 dead, 3 wounded in Calif. school shooting

Another school shooting has resulted in the death of 2 California high school students. The suspect shot himself and is in custody.

Mercury makes a transit; next in 2032

A transit of Mercury occurred today and was visible from the US, provided you had sunny skies. It was one of longest possible transits.

On the Naperville BWW racist incident

A racist incident at a Naperville, IL, sports bar indicates that the threads of racism are strong, perhaps as strong as ever.

IL bill could excuse absences to vote

A proposed law in IL could give students up to two hours during the school day so they could vote in the upcoming election.

Loan forgiveness gains some bipartisan support

One Republican from GA, who used to work under Betsy DeVos at the US Education Dept, offers a plan to forgive some student loan debt.