Alternative Treatments For Fibromyalgia: A Critical Analysis
One of the common themes regarding alternative medicine is the reversal of normal scientific thinking. In science, we must generally accept that we will fail to validate many of our hypotheses. Each of these failures moves us closer to the truth. In alternative medicine, hypotheses function more as fixed beliefs, and there is no study that can invalidate them. No matter how many times a hypothesis fails, the worst that happens is a call for more research.
Sometimes this is the sinister and cynical intent of an alternative practitioner—refuse to let go of a belief or risk having to learn real medicine. Often, though, there are flaws in our way of thinking about data that interfere with our ability to understand them.
This week, the New York Times had a piece on alternative therapies for fibromyalgia. First a little background.
Fibromyalgia syndrome is a poorly-understood and controversial pain syndrome. In brief, it identifies patients who have significant chronic pain which is not due to any identifiable pathology. It probably includes a heterogeneous group of problems, but our understanding is limited. There may be changes in the way the nervous system deals with pain signals, but even this is not yet clear. It’s a disorder that can be very frustrating to treat, and even more frustrating to have. It is often co-morbid with depression, and the pain can be quite resistant to treatment.
Some practitioners deal with this by rejecting the diagnosis as being vague and useless. Others use the limited evidence we have to develop a treatment plan. And yet others turn to alternative medicine, and that is the topic of the Times piece. The article is a brief presentation by an expert with Q&A in the comment section.
One exciting area of research in the past decade has been in the realm of complementary and alternative medicine, or CAM, treatments for fibromyalgia. These range from well recognized therapies like acupuncture and massage to more novel treatments like d-ribose and qi-gong.
As this research grows, it is increasingly possible to identify CAM therapies that have some evidence of efficacy and minimal risk that can be incorporated right along with the more conventional treatment recommendations.
This is the typical claim of alternative medicine: it’s relatively harmless, and might even help. But what does the evidence say, and what are we to make of it?
One of our goals at Science-Based Medicine is to recognize that in traditional evidence-based medicine it is easy to become overly reliant on the results of randomized-controlled trials (RCTs). While EBM does take into account the concept of plausibility, this is often lost when the data is “hot”. A paper looking at CAM therapy for fibromyalgia was recently released, and can serve as an example of how to think about these problems.
By way of introduction, let’s look at the abstract:
Best evidence was found for balneotherapy/ hydrotherapy in multiple studies. Positive results were also noted for homeopathy and mild infrared hyperthermia in 1 RCT in each field. Mindfulness meditation showed mostly positive results in two trials and acupuncture mixed results in multiple trials with a tendency toward positive results.Tendencies for improvement were furthermore noted in single trials of the Mesendieck system, connective tissue massage and to some degree for osteopathy and magnet therapy. No positive evidence could be identified for Qi Gong, biofeedback, and body awareness therapy.
This study looked at other studies to see how many showed promise in CAM therapies. The authors’ conclusions were based on the “positivity” of RCTs, that is, a modality was seen as possibly effective if the RCTs supported the hypothesis. This was despite the finding that overall, the studies were of mediocre quality. There are a number of flaws in this approach.
The authors state explicitly their reliance on the reputation of RCTs:
Most of the represented publications review both randomized controlled trials (RCTs) and non-RCTs. Though RCTs are considered as the strongest research basis for clinical recommendations in evidence-based medicine, RCTs are particularly difficult to perform in many categories of CAM medicine. An individualized approach to
the patient in diagnosis and therapy is often already part of the healing process itself. This makes standardization and the creation of control groups in order to rule out so called ‘placebo effects’ often very challenging, and blinding of both patients and medical practitioners sometimes impossible. Thus, it is not surprising that many authors focus on different study designs to fully cover the field. However, as RCTs are considered to be less liable to bias, there is also need for publications that focus on only these kinds of clinical trials. In 2002, a methodologically impressing publication appeared that covered RCT research on non-pharmacological approaches in fibromyalgia [24].
The authors re-iterate the special pleading used by CAM advocates to avoid being subject to scientific investigation, but decide to focus on RCTs to avoid the issue. What they fail to do is explicitly state what RCT results mean, beyond being positive and negative.
Plausibility
Dr. Harriet Hall would remind us of Tooth Fairy Science. We can measure all of the important data about the tooth fairy, including average get per tooth, average age of visitee, etc, but if we forget to question the fairy’s existence, we have failed to ask the most important question. It may be true that an RCT showed improvement in fibromyalgia patients using homeopathy, but since homeopathy is water, there is no reason to expect causality, and the results may be better explained by some other phenomenon.
This is explained mathematically by Bayes’ Theorem. If the prior probability of a positive result being due to the intervention is very low (say, because of implausibility), then any positive finding is very likely to be due to chance rather than causality.
Confounding natural variation with causation
Fibromyalgia is a syndrome whose symptoms naturally wax and wane. It can be very easy to confuse a change in disease state that occurs during a study with an actual effect. Rigorous controlling can minimize this but not prevent it. If, by chance alone, subjects in the treatment group had improvement in their disease due to its natural history, this will look statistically like a “win”. This makes the study of such disorders difficult, and opens a big door for CAM, as it is easy to convince others to follow your misattribution of cause. This is similar to concepts such aslead-time bias and regression toward the mean.
Built into this is the common cognitive error of confirmation bias. If you are a believer in the intervention, you may be prone to attribute positive results to the intervention even if there is no causation.
Damned statistics and replicability
The statistical tools we use to interpret RCTs are designed to help us tell systematic variations in the data from chance alone. There are a number of arbitrary assumptions built into this system. For example, if results are described by a normal distribution, we may define “abnormal” as the highest and lowest 2.5% of results. If a single RCT shows statistically promising results (say, >2.5 SDs from the mean), then it’s “positive”—but this still may be due to chance alone. A well-designed study can minimize the chance of this result being due to chance alone but cannot eliminate it. This is one of the reasons a single positive test for a less plausible hypothesis must be replicated before we get too excited.
The bottom line
Fibromyalgia is a complicated syndrome whose very nature makes it susceptible to the abuses of CAM practitioners. When evaluating a therapy for a complex disorder whose natural history is variable, we must very carefully parse out causation from correlation, recognize our own biases, and remember that a positive result of a randomized-controlled trial does not necessarily confirm a hypothesis. If an intervention has no plausible way or working, any positive results are likely a statistical artifact. Science is hard work, but the results are worth it.
___________________________________
References
Baranowsky, J., Klose, P., Musial, F., Haeuser, W., Dobos, G., & Langhorst, J. (2009). Qualitative systemic review of randomized controlled trials on complementary and alternative medicine treatments in fibromyalgia Rheumatology International DOI: 10.1007/s00296-009-0977-5
Return to article »