If we can’t trust science what can we trust?
This is a geeky post about how scientists do science. Science means many things to many people, but this post applies to virtually all branches of science, be that cancer research or global warning.
Science is a way of checking things out to see if they are true. It means you have to give up on your “hunch”, if the evidence proves you are wrong. This happens all the time in the lab, you think that “your favourite gene” will be up regulated in cancer cells, but when you actually look at the data it doesn’t change and a couple of times it goes down, so you have to go back to the drawing board, look for other genes that are changing or other genes that interact with “your favourite gene”. This takes time (months) and money (thousnds of pounds).
Once scientists have discovered something e.g. tumours that have mutations in both p53 and ATM respond to chemo, but those that have mutations in only one, don’t respond to chemo, then the scientist needs to write a scientific paper and have it published in a journal (a specialist magazine). In the example above the results were published in Genes and Development, a highly respected journal with a good impact factor (i.e. the stuff that appears in here is good science). This is an example of a research article, in general, good scientists write more research articles than bad scientists. So the scientists with the most research articles tend to get the best jobs at the best universities.
What about other types of article? As well as publishing their own research, scientists publish “Review Articles”, these don’t contain data, but they are an overview of a whole field of research. These articles give one particular scientists opinion on the state of current research and future questions that need to be answered. As a scientist, I like review articles, especially if you are new to a subject, they give you a great overview of a lot of research. BUT, these articles are subject to bias. Often these articles will mention the work of the scientist and his friends and ignore data that they don’t like (e.g. if it contradicts them). Selecting some studies but not others is called “cherry-picking”.
Okay, cherry picking your mates articles doesn’t sound very scientific, is there a better way of carrying out a review? Well, there is, it’s called a systematic review. Where you decide in advance what question you want to answer and look for ALL articles on the subject (whether you agree with them or not).
Recently the Soil Association responded to a systematic review which said that organic food was no better for your health than conventional food production (More information on the Bad Science website Over there!). On the same website there is an excellent blog post on “How Myths are Made“, this describes how academic reviews can bias scientific opinion. This blog post also mentions the recent scandal in America where a large drug company (Wyeth) were accused of employing “ghost writers” to write reviews which appeared as if they were from scientists at academic institutions rather than written by the drug company itself. You can read more on the PLoS Blog or In the Pipeline – Ghost Writing.
So where does this leave us?“All of science is subject to such exploitation because all of science is fundamentally characterized by uncertainty. No study is perfect; each one is subject to criticism both illegitimate and legitimate — and so if you wish, you can make any scientific stance, even the most strongly established, appear weak and dubious.”
Chris Mooney March 2008
Who can you trust? Well, science isn’t perfect but it is the best way we’ve got of tackling cancer. If I was looking for non biased medical information I would start by looking at the Cochrane Reviews. Of course, the newer the treatment, the less likely it is to be covered, simply because their haven’t been enough studies done on it.
I for one, would far rather put my money and my life in the hands of a scientific clinical trial than give my money and my health to someone who was trying to sell me something, at the end of the day even if I didn’t benefit from taking part in the trial I know that clinical trials will benefit others who come after me.