Trying to Confirm What You Already Believe is a Dangerous Practice

Recently published research in the Archives of Internal Medicine has attracted attention because it concludes that small gifts influence physician choice of medications. I am certain that this study will be used to justify future public and organizational policy decisions.

The Real Issue

The focus of my comments is not related to influence on physician decision-making. Instead, I feel we should examine the misuse of research findings and the practice of extending conclusions beyond the conditions of the research. That reality that such research findings are publicized as credible is evidence of the failure of our educational system. It’s time to stand up and scream “Foul!”

Medical Students vs. Doctors

This research compared medical students at just two medical schools, each of which had different policies about small gifts from pharmaceutical companies. Based upon psychological tests of positive and negative associations, rather than actual prescribing patterns, the researchers concluded that even small gifts influence opinions of physicians.

There are a lot of reasons to believe that there are significant flaws in drawing conclusions about practicing physicians from this study.

Can You Say Laparoscopic Cholecystectomies

First, the subjects of the research were students; not practicing physicians. Relying upon students as representative of physicians would be like assessing how well students can perform laparoscopic cholecystectomies and concluding that physicians are not skilled enough in performing surgery. Is there anyone who does not recognize that medical students differ from practicing physicians?

Passive vs. Engaged

Medical students are not professionally engaged in “self-learning,” evaluating journal articles, or practice guidelines. They are passive recipients of information. Physicians learn how to become life-long learners in their residencies. Medical students have not gone through the socialization processes that are part of post-graduate medical education that prepare the physician to have a more questioning attitude about what they are told. And medical students are in an economically different place than practicing physicians which might cause them to be more influenced by lower cost items than physicians.

Flawed Assumptions Leads to Flawed Conclusions

Second, if that were not enough to debunk this study, a fundamental flaw of experimentation was overlooked. Only two medical schools were studied. While these schools differed in their policy on small gifts, it’s very likely that they varied in other characteristics also. For example, faculty attitudes toward drug expenses might be different (which would align with the differences in policies). The problem is that there may be many unknown dimensions in which the two schools differed. To overcome the unknown differences, one needs to study students from more schools. Even that might not be sufficient, depending upon what the data reveal.

Rhetoric … Not Research

More than likely, the researchers had their minds made up before they conducted the research. That is, they were biased. When researchers are biased, it influences the quality of their research design, their analysis of data and the conclusions they draw. Ultimately, their research is not research, but rhetoric. Research requires skepticism.

Same is True in Good Management

The same is true in management. When managers seek to confirm what they believe to be true, they begin a path down a road that leads to missing important warning signs. Good research and good management research challenges pre-existing belief.

Challenge Your Own Beliefs

Trying to confirm what you already believe is a dangerous practice.

Instead, try to disprove what you believe to be true.