More dodgy analysis: moderate drinking might not be good for you

One of the aims of Oikos is to expose dodgy analysis on environmental and economic issues. Economic data is particularly susceptible to problems because it rarely comes from controlled experiments but rather from observing the real world, where all sorts of unexpected or unseen causes can confound the results.

If we want to see how strong environmental protection affects a country’s economy, for example, we need to work out what ‘strong environmental protection’ means and how to measure it (and ditto for a country’s economic performance). We could then compare a country’s environmental regulation over time with its economic performance over time. Or we could compare different country’s economic performances and level of environmental protection. The problem, of course, is that it’s difficult to isolate the real effect of environmental protection as there are an enormous range of things that can affect economic performance.

Let’s assume that we do a study and find that generally, the stronger the environmental regulations in a country, the stronger the country’s economic performance. It’s a big leap to go from this finding to the conclusion that strong environmental protection assists economic performance. It could be (and more likely is) that a strong economy gives us sufficient wealth to be able to direct more to protecting our environment.

The same problems exist in other social sciences and in population health research. The most effective research is to set up randomised trials – where participants are randomly assigned to a treatment group or a control / placebo group – this can give a very good measure of how effective the treatment is. Unfortunately, these trials are expensive and can take a long time. Sometimes they’re impossible (or extremely difficult) to perform.

Take, for example, the effect of alcohol on heart disease risk. It would be ideal to randomly assign participants to a heavy drinking group, a moderate drinking group or a non-drinking group, get them to follow that regime for a few years and then measure their levels of heart disease over the next decade or so. But that’s not feasible. For a start, no-one would agree to be in the non-drinking group. So instead, we need to observe people’s drinking behaviour over some years and their heart disease risks. We could get this data cheaply and fairly easily for example by surveying patients admitted to hospital with heart conditions and comparing that to a survey of drinking habits of the general population.

The problem is this: If we find a link between drinking patterns and heart disease, how do we know that it is the drinking habits that have caused the different rates of heart disease? It may be, for example, that the incidence of heart disease affects drinking habits (having a heart attack may prompt you to cut down on your alcohol intake) or it could be that some other factor causes both (perhaps having a stressful job or lifestyle encourages you to drink and increases your risk of heart disease).

This is exactly the problem that a recent study of alcohol and heart disease has uncovered. It’s become accepted wisdom over the last few years that moderate alcohol intake reduces your risk of heart disease. That’s not necessarily wrong, but this study suggests that many of the studies that this finding has been based on are flawed:

Researchers at UCSF pored through more than 30 years of studies that seem to show health benefits from moderate alcohol consumption, and concluded in a report released today that nearly all contained a fundamental error that skewed the results…

The common error was to lump into the group of "abstainers" people who were once drinkers but had quit.

Many former drinkers are people who stopped consuming alcohol because of advancing age or poor health. Including them in the "abstainer" group made the entire category of non-drinkers seem less healthy in comparison.

Without the error, the analyses shows, the health outcomes for moderate drinkers and non-drinkers were about the same.

The first lesson for us I think is to read scientific and economic research with a critical eye. The second lesson (one which I am ignoring right now!) is to read primary sources and not just media reports of studies, if the results of those studies are important to us. I don’t think journalists ever read the studies they write about.

The third lesson is that drinking a good red is its own reward.