How do you judge a scientific discussion if you’re not actually an expert in the field? Of course, usually the answer is that you can’t. Scientific debates are typically rather involved and highly technical. Really the only way of coming up with a meaningful judgement just for our own satisfaction is to roll up our sleeves and spend a considerable period of time teaching ourselves the necessary groundwork. To be able to participate is vastly more challenging still.
We all know this, of course. And yet sometimes we just have to make a judgement. Sometimes we have to decide whether something is safe or not – e.g. mobile phones, living under transmission lines, eating BSE infected beef, MMR vaccine or just plain old flying. Occasionally our job is a lot easier because the data is actually pretty transparent. For example, we don’t really feel that we have to trust the science of flight any more because planes do land safely the vast majority of the time.
At other times the decision is harder as the potential costs occur a long time in the future and we know that data on actual outcomes is not even available to the scientists. Whether or not to eat beef from cattle infected with BSE is this sort of question. For these sorts of issues we don’t always really need to know the truth, because sometimes we can just err on the side of caution. (Some might say that this is wrong because you can’t know what counts as cautious without a proper understanding of the probabilities and the costs. In my view this misunderstands lots of things about what it means for an action to make sense, but there’s plenty of that elsewhere).
Sometimes such an easy escape route is not available because the potential costs of either decision are substantial. The MMR vaccine is one such case. Not because I believe that the costs of vaccination are high, but because there were people who believed that this could be the case. So for them to make a decision was not trivial. (It would have been a lot more trivial if they had listened to scientists rather than celebrities and newspapers – but this post is intended to be a discussion of exactly how someone can decide to do that.)
Whenever you need to make an active choice about what to do or believe there are really only a few options available to you:
- Go along with whatever the scientific consensus says, assuming that there is one – i.e. defer to authority. This sounds daft, but really isn’t for the simple reason that we all have to start somewhere. In my case I haven’t done any research into the MMR vaccine, but I believe it is safe. Sure I could go through the actual scientific literature. Many other people, I think, would struggle to do that at all. But it would take me a reasonable period of time and I know that I can’t do that for each such case.
- I can make an honest and determined attempt to understand the issue by reading around the subject. If I do this for long enough there is a chance that I would be adequately qualified to judge. Far more likely an outcome is that, with luck, I’ll be better placed to judge than I was, but still not truly competent. I may have figured out who to listen to however and sometimes that is hard enough.
- I can make no attempt whatsoever bar reading the headlines in the paper. This really is daft most of the time, but even so we all do it occasionally I am sure. If the paper has done a good job we won’t be too misled – but don’t be too hopeful on that score.
As alluded to above, even if I do make some effort to educate myself, this doesn’t really solve the problem because now I need to find some way to judge which participants in a debate I should believe. Occasionally debates crop up that illuminate this effect. Here is one such example. What am I, as an observer, supposed to make of this debate?
As always, we are influenced by more stylistic elements than we might like to admit. Here are the first thoughts that flashed through my mind when reading it.
- Here is a book written by a journalist which purports to declare that the actual scientists in a field are seriously misguided. We’ve had “a century of misguided obesity research”, apparently. I’ve seen a lot of books written by non-experts that claim to overthrow all sorts of scientific paradigms. Do they usually do any such thing? Virtually never.
- It’s a book and not a scientific paper. If this was climate science, for instance, there would be a presumption that any new research of any merit ought to be published in a peer reviewed academic journal – preferably a reputable one. The same standards would be expected of research into the purported efficacy of homeopathy, for instance, and a failure so to do might well be considered damning.
- The main complaint seems to be a lack of randomised trials combined with a belief that randomised trials are essential. So it is curious that the book author has managed to rustle up what appear to be 10 rather strong conclusions from a field containing a paucity reputable studies.
- The book “covers the literature right back to Samuel Johnson”. I’m struggling to think of any scientific discipline where a review so far in time is relevant for the current state.
Now, before we go any further I should make something very clear. I haven’t read the book and I’m extremely unlikely to do so. But even if I did read the book that would be in no way enough for me to judge its contents because I would have to read around the subject extensively. The blog author admits that he is “not an expert in nutrition”. He is at least involved in medical science. I’m no expert in anything very closely related to the matter at hand and I’m not going to pretend that I could judge the book even if I had read it.
In fact this is why, for myself, this is an interesting case. If the blogger in question did not have something of a reputation for sceptical assaults on some of the mumbo jumbo peddled these days, I would have dismissed the whole thing out of hand.
And, most critically, my point is that we all do this. Rapid dismissals of another’s purportedly scientific opinion are to be found everywhere including the supposedly methodical sceptical community. Why? Because you have to filter out the set of things that you personally are going to investigate. At some point you just need to trust that what you’re being told is accurate. One of the great benefits of science when it is working well, is that the challenge and debate is done for you already. That by no means makes it infallible, but it certainly makes it more believable than the latest author of a shocking new book – whether that is about global warming, alternative medicine or nutrition.
Leave a comment