Care Triad

Many commonly used treatments may not work

A Washington Post WonkBlog piece entitled “Surprise! We don’t know if half our medical treatments work” got a lot of attention in social media circles. The title is a bit misleading but the concepts are really important. First, let me say that I worked at the BMJ for 8 years and was involved with the Clinical Evidence publication that is discussed in the blog so I may be a little biased!

The way doctors determine if medical treatments work is to perform research studies called randomized controlled trials (RCTs). These are studies where half the patients get a treatment and half get a placebo (or inactive treatment like a sugar pill) but the patients and the researchers do not know who is getting what. After a period of time (could be years), the researchers look at the results and figure out which group did better.

But you can’t just look at one RCT and decide whether or not a treatment works because a different RCT may have gotten an opposite result. So what the BMJ does in Clinical Evidence is to look at all the RCTs about a particular treatment and then summarize the conclusions. If all the RCTs about a treatment have the same result, the conclusion is pretty easy. If the results conflict with each other, then we can’t be sure the treatment works. Clinical Evidence says these treatments are of “unknown effectiveness”.

But just because we don’t know for sure if a treatment works isn’t necessarily cause for concern. It is possible that the treatment really does work – the evidence is just not strong enough (yet) to say for sure.

And even if a treatment has been found to work in RCTs, it may not work in you. The outcomes that are being measured have usually been determined by the researchers and not by the patients. So it is important to see if the treatment is solving a problem that is important to you. For example, if you have arthritis and a treatment is found to relieve pain but the issue that is most important to you is being able to move easily, then you the treatment may not be ideal for you. Another thing to consider is that in order to be included in an RCT, patients are very carefully selected and usually have only one condition. If you have two conditions or are different from the patients in the study in other ways, it is possible the treatment won’t work as well in you.

Most importantly, there is more to making a medical decision than just the evidence. While it is necessary to understand the evidence, In most situations, the evidence is just one piece of the decision. It must be balanced with our own values and preferences and the experiences of the doctor to come up with the right personal treatment choice. We, as patients need to get our doctors to explain just how good the evidence is and to let us know about other treatment options.

So, it is true that at least half of all treatments in use today are of “unknown effectiveness”. But it is not really a surprise and does not prevent us from making informed healthcare decisions.