Which Scientific Studies Should We Trust?

Back in the 1970s people used to joke that scientific researchers would soon be telling us that sex causes cancer, and at least one comedian went so far as to envision a future where underwear came with cancer warnings emblazoned across the crotches. The 1970s were a time when new studies seemed to be published almost daily, each one purporting to have discovered a link between cancer and something every household had in its pantry, medicine cabinet, or among its toiletries. Almost invariably, a few months, or years, after the original study was published, a new study would come along debunking the first study. And so it would go, on and on, and on, and on... .

We all knew, back then, that scientific studies were just that: studies. They were an investigation into a theory, and not a conclusion in and of itself. Even without any formal scientific education, a person could guess that these studies were not to be taken as one-hundred percent conclusive evidence of this or that 'fact'. Yeegads! One day one study would tell us that grapefruits were dangerous to our health, only to have a follow-up study tell us that, no, the first study was wrong and grapefruits were actually good for us! Who in their right mind would take them seriously?

Of course, back then, like now, the media jumped on these studies and rode with them as far as they could go. But in those days the media was quite different from what it is today, and people didn't mistake talk-show celebrities for gods who had come down from Mount Olympus. Talk shows were entertainment, and entertainment was different from reality. That said, when talk shows did bring on guests to speak about serious issues, they were of a completely different caliber from what we see today. The producers would usually tap the pool of local professionals whenever they needed someone to weigh in on this or that issue or topic. Doing so brought some degree of sanity to the discussion.

We also had a plethora of top-quality shows that specialized in consumer research and product investigation - at least in Canada, we did. The journalists were not afraid to ask hard questions, and shows such as 'Marketplace' and 'The Fifth Estate' were not the weak, wobbling shows that they are today. They raised questions for the viewer to think about, instead of simply providing cookie-cutter answers (as they sadly do now).

I may be remembering things through rose-colored lenses, and I am not denying that there were bad apples among these shows, but I recently came across a report on YouTube that the uploader deemed to be a fantastic piece of journalism. Guess what? It was an old segment from one of the shows mentioned above, and the journalist was Hannah Gartner. Even thirty years down the line her work still shone as an example of quality, unbiased journalism.

The point that I want to raise is that we no longer seem to know how to interpret the scientific information that we come across in magazines, on the news, and on the internet. Today's media, as well as talking heads from both the political right and left, don't seem to understand that scientific studies are not proof-carved-in-stone. They take one study - usually one that will get a lot of positive feedback from their community of viewers - and wave it around as the Holy Grail, finally discovered, regarding this or that illness or issue.

Scientific studies don't work that way. They work this way:

1. Scientist(s) submit a report of their work, and their conclusions based on that work, to a recognized scientific journal.

2. The scientific journal usually has the study reviewed by other scientists with knowledge of that particular discipline. If these scientists believe that the report has drawn valid conclusions based on the information outlined in it, they okay it for publication.

3. The scientific community then independently reviews the report, the data used, and the conclusions drawn, and weighs in on it.

4. Other scientists attempt to replicate the results of the study using identical methodology, and report their findings.

5. If subsequent studies cannot replicate the findings of the original study, then the original study is considered flawed and not credible. Please note that it takes several studies returning different results from the first one to confirm that the first study was indeed flawed.

So, in a nutshell, you cannot pick a study and wave it around as proof of whatever it is that you want to believe, as has happened with the Andrew Wakefield study on possible link between the MMR vaccine and autism. The only subsequent study that claimed to have found evidence supporting the link was also found to have serious flaws and had to be retracted. Meanwhile, there have been numerous other studies that show no link between autism and vaccines.

How should the general public interpret this, then?

Well, the answer is found in applying "Occam's Razor". Which makes more sense, in the least convoluted way? The original study, or the ones that came afterwards and contradicted it?

If the original study is riddled with conflicting interests and problematic data, that alone should make it suspect. The evidence of our own eyes, and of our own reason, should tell us which information to trust. If grapefruits will kill you, then shouldn't the majority of people who have consumed grapefruits drop dead?

Sadly, this is not the first time that the followers of a discredited study have battened on to it, and refused to accept subsequent research. In 1971 Linus Pauling published the book "Vitamin C and the Common Cold", which advocated the use of large doses of vitamins to stave off and/or cure illness. Subsequent research has not been able to confirm or convincingly deny the validity of Pauling's assertions.

Again, let's apply Occam's Razor to the argument. Which side presents the simplest, least convoluted evidence?

Well, we know that vitamin C prevents scurvy. The question is, does it cure the common cold? No, it doesn't. It may alleviate symptoms, but it does not provide a wonder cure.

In the end, scientific studies are only explorations into possibilities. You can't batten onto one and say 'Ah-ha! This proves it!'. You have to look at what all the evidence suggests, and that includes subsequent research into the same question. If ten studies agree, then it is likely that the conclusion is correct. If five agree, and five disagree, then there is something else at work - likely, a principle that science has yet to detect.

The dangers of clinging to discredited studies ought to be evident to everyone. Unfortunately, it appears that they are not. The overriding danger is that, by refusing to let go of erroneous information, we are preventing the discovery of new information that could possibly lead to a cure for a disease, or otherwise help us. We could also end up doing things that will cause ourselves even greater damage down the line.

That is why we have long been cautioned by the scientific and medical communities not to jump on passing bandwagons.

Today's hotly hyped clinical studies are all too often used to wrap our garbage in tomorrow.

Image: Pixabay