Tuesday, April 22, 2008

How NOT to do health research. . .

Today, students handed in their final work (except for the final exam) of this semester--a research paper. I really like their final project--it's a collaborative one, with students working in groups of 3 or 4. They have to select a developing country, then identify a problem in that country, and present an oral report the background on the country and problem, then argue whether or not the developed countries, especially the U.S., have an obligation to help solve the problem. Then each student writes her own research paper on the group topic. For example, one group selected Cambodia and the problem of human trafficking.

What I especially enjoy about this project is helping students to expand their research capabilities. Research is one of those things I do best. In fact, before I returned to teaching five years ago, I was engaged in research--sort of. Of course, there's a back story.

I have written about some of the various jobs I have held through my 40 + years of working, but I don't think I have written much about working at the health insurance company--my last full time job.

I was hired to do health policy research in a newly established institute. This institute was housed entirely within the health insurance company and was really something of a p.r. outreach arm. In other words, the folks at this company weren't really interested in doing true health research studies.

And how, you might wonder, did I learn about this? When I first joined the health insurance company, I had high hopes for doing some interesting health policy research and analysis. I enjoy figuring out why things are the way they are, so I envisioned pursuing some challenging health questions, for their own sake. When I first arrived at this new job, one study had already been completed, and another two were in publishing stages. So I set about to work with these works in progress.

Then I got my rude awakening. The completed study was on the health costs of violence--a very interesting subject. But the researchers were a small group from Princeton, NJ (not affiliated with the university) who had a preconceived notion of what they wanted the research to show--namely that the costs that result from violence are not really health costs, so health insurance shouldn't pay for them. Huh? I argued for awhile about that conclusion--I said what difference does it make which pocket you take the money from. If someone is injured by gunshot, the medical procedures needed to make him whole again have to be done, and someone will pay for the care. Clue # 1.

Then I had one of the doctors at the health insurance company come to me and say--why don't you do a research study that shows the terrible consequences of women having abortions. What do you mean, I asked. Well, they all suffer from guilt, and need psychiatric care--he answered. Huh again? See, when you do a research study you don't normally start out with the conclusion and then bend the data to produce the result you want. I nixed that "research." Clue # 2.

Another member of senior management came to me and said--let's research the health costs of caring for people in the last year of life. OK, I said--and then what? Well, then we (meaning the health insurance company) could decline to pay for really high cost medicine in the last year of life since it won't help the person anyway. Well, I noted, you don't know when a person's last year of life begins. So, I pointed out that such a study can only be done retrospectively and can never be applied prospectively. Oh. Another research study not done. Clue # 3.

Finally, another doctor asked if I could give some grant money to a company that had developed cancer treatments that could target specific cancer types. Sounded good. I asked how the researchers would identify which patients to use the treatments on. Then he said--we'll give them the patient information so they know who to contact. Huh? I suggested (gently) that it might not be a good idea to give to a third party confidential patient information. Oh. Clue # 4.

Well, my health research days were--shall we say--not long at that company. And my little health institute is no more. Truth be told--I am MUCH happier being "retired" and teaching college freshmen how to think clearly--I only hope their research efforts are less thwarted.


Anvilcloud said...

I have a feeling that a lot of research is done to support preconceived notions.

Climenheise said...

Since I also work with research (although in a different field), this all sounds so familiar. I note that the projected pieces of research are all worthwhile, and that the expected outcomes are not themselves invalid. One can state a hypothesis; in fact, doing so helps to keep one's research honest by showing one's presuppositions. It is easier to control for stated prejudices than for hidden hypotheses, which control the outcome at least partly because they are hidden.

The problem in each example you gave is that the hypothesis is not to be tested. Supporting the hypothesis is the only acceptable outcome, which vitiates the possibility of real research. If the question about helping those (men and women) who struggle with unresolved issues after an abortion was genuinely open to unexpected outcomes, it could be valuable. (Part of that openness is the real possibility that an abortion may have been a step towards full mental health in a given situation, for example of ongoing abuse in a family.) If the desire to help people in the last years of their lives did not have a controlling outcome (cut off medical help), it could be useful indeed. And so on for each piece of research.

I almost wish you were still working there, helping a company that doesn't want to know how to find out what's really goi9ng on in their world!

Climenheise said...

We need a way to edit our comments. Then I could take out that silly #9 that slipped into the word "going" in the last sentence!

KGMom said...

Comment on doing research with a preconceived notion in mind.
I think there's a difference between a HYPOTHESIS you want to test, and a preconceived notion. In the instance of the proposed research on abortion, the doctor who asked already knew he wanted any health insurance usage to show the conclusion he had already arrived at. So, it would not have been a research study. If he had said, let's study the health insurance utilization post-abortion, that would have been a legitimate study.
The example on cost of care in the last year of life was most intriguing. It is true--according to Medicare stats--we pay huge costs in the last year of a person's life. But, of course, the person proposing the study hadn't reckoned with the obvious--most of us don't know when the last year of our lives begin.
As for your wish, Daryl, that I was still working at the institute--well, when the company merged with a larger company, the new powers that be weren't interested in RESEARCH.

Anonymous said...

To solve problems, first we must state the problem clearly. This and other rules are found in the new book on amazon.com: "Teaching and Helping Students Think and Do Better". This is an excellent book.

Mary C said...

Sounds like those searches were not only preconceived notions, but sounds a bit unethical to me. And, of course, you can say I'm on the other end of things being a medical librarian. I have had a couple of requests to find material that was in argument or support of what a doctor has done or did not do to convince the powers that be, that a patient's treatment should be covered by their insurance company.

Climenheise said...

I agree that hypothesis and preconceived notion are completely different. I was observing that they are also closely related, and that testing a hypothesis is one way to help stop holding a preconceived notion. Which requires the courage to be wrong in one's notions.

I take the point of the last year of life -- and of the difficulty of determining when it begins! As we get older, I know I don't want someone else deciding when my last year starts. God knows when it will be. Does corporate America sometimes play God?

JeanMac said...

Good move to get out of there!

Tossing Pebbles in the Stream said...

I guess you learned that insurance companies are in trhe business of finding ways to not have to pay.

dguzman said...

Holy moly, some "research" ideas they had at that insurance place! Wow.

joared said...

The truly unfortuate part of many "research" conclusions offered to the public is that the credibility and reliability of the study, or number of people in the sample are often not really addressed. Many people, even some who write these up in the press don't know how to determine what is good research and what isn't. Certainly starts with thinking, so hopefully your students will learn to think at increasingly complex levels.