Does “Anecdotes” mean “Evidence”? Spiritual healing and public health

In my previous post I said anecdotes can be used as evidence, if the conditions are correct. That is a controversial statement, so does need to be explained. And, here I plan to do that by giving you one example of when anecdotes do work and when they don’t. In typical-teacher fashion, I am going to let you think about what the differences are. But I am going to illuminate the difference.

Spiritual healers

Imagine there is an illness that has a 1% survival rate untreated and a 5% survival rate treated (both over 5 years); that is a pretty horrible illness. If you get that illness, the bet to make at the bookies is against you. But assume this illness exists and a series of “alternative healers” (spiritual, Biblical and Raiki healers) have thousands of patients who have overcome the illness while visiting them. Thousands! There must be something in it, therefore God (or something). If I had to conclude from that evidence, I would conclude something in line with Occam’s razor: the power of psychology over the body. There is already evidence for that.

But instead of jumping to conclusions, we should be asking questions. How many people had the illness? How many people went to see an alternative healer? How many of them were still receiving medical treatment*? What are the false-positive diagnosis rates for this illness? Once we have the answer to these questions, we can start to look into whether the alternative medicine anecdotes really are evidence.

I’ll have to assume some data for the illness, because I’m making it up. Lets say 15 million people got the illness in the last 5 years, in the English-speaking world. 5 million of them do not get medical treatments because they can’t afford it (shame on you, USA), or for religious or personal reasons. Assume 80% of those people go to an alternative healer; that’s 4 million people. Untreated, they have a 1% survival rate. 1% of 4 million is 40,000. So, there’s 40,000 anecdotes. And 10 million people left to go.

The remaining 10 million people are receiving treatment, and have a 5% survival rate. Still, the illness means falling on pretty desperate times. Assume 10% of all treated patients are seek alternative healers. That’s 1 million people seeking alternative healers. If alternative healing is of no benefit, expect a 5% survival rate, 50,000 people. There’s a further 50,000 people who survived after seeking alternative healers.

That’s a total of 90,000 anecdotes, and yet the alternative healers don’t have a success rate higher than if they didn’t exist. There’s nothing in it.

*When I was teaching in Austria I remember meeting this one particular hippy Teacher who had some low-lying illness that she couldn’t seem to shift. She went to a doctor and the doctor gave her some antibiotics. After she read the possible side effects she decided they couldn’t be good for her, so sought an alternative healer. They gave her homeopathic medicine. And, voilà, her illness was gone. Evidence for homeopathy. Except, she was taking the antibiotics as well.

Public health investigations

Imagine you work in public health, and your associate NHS Hospital reports an increase in severe stomach upsets. How do you go about pinpointing the cause of this increase? First, you make some informed guesses: if the number has gone up from about 2 per day to about 16 per day, we’re probably not looking at a contaminated water supply. You would expect people to be coming in by their hundreds if that was the case. You have 16 people to start asking questions: where have they bought their food over the last week? What restaurants have they visited over the last week? Any overlap helps you find the cause of the illness. That is anecdotal evidence slowly becoming evidence.

Advertisements

3 thoughts on “Does “Anecdotes” mean “Evidence”? Spiritual healing and public health”

  1. Just a quick point about placebo (and its opposite, the nocebo) effect: this is a subjective report of feeling better (often associated with reporting levels of pain and discomfort). There is no physical change that can be documented. This is important because if you leave wiggle room for an improvement in health outcomes – without clarifying exactly what these improvements really are – then believers in alternative and complimentary therapies will use this ambiguous ‘evidence’ as reason for confidence to boost their unsubstantiated claims.

    And yes, anecdotal reports are evidence… of the very weakest kind. Sometimes – as your NHS example shows – this the only evidence we have to go by, But if there is any probability for some other physical cause, then this consideration trumps the anecdotal kind because it links cause with effect by a known mechanism.

    1. True, if anecdotal evidence suggested your restaurant was the cause of the stomach upset you’d demand they presented physical evidence and you’d be outraged if the media used the anecdotes to publish something.

      I’d not thought about the placebo effect of alternative medicine. But for survival rates, I doubt it’d matter as much.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s