Elements of truth

Stoat’s recent post about peer review, reminded me that there was something related that I had been considering writing about. There appears to be many climate “skeptics” (deniers some would call them) who regularly make claims about peer review being flawed (often referred to by them as pal-review), academia being a closed-shop that skeptics cannot enter, and that research funding is biased towards those who toe the anthropogenic global warming (AGW) party line (for example). What’s ironic – of course – is that most (if not all) who make these claims have virtually no experience with any of this. They’ve never published a paper (never had a paper reviewed or reviewed a paper), they’ve never been in an academic job or experienced academic hiring practices, and they’ve never come close to having a grant funded (never submitted a grant, reviewed a grant, or sat on a grants panel). So, in my opinion at least, most of what they say is essentially, ill-informed nonsense.

However, there is a grain of truth to some of what they say. Peer-review isn’t perfect. It’s, however, never been intended as a way of confirming scientific results; it’s simply meant to be a check for obvious errors and a way of filtering out what is clearly wrong or poor. It also helps to improve things even when they’re good. However, it is true that sometimes good papers can be held up by poor reviewers (who might have some kind of bias) and bad papers can get through because of lazy or bad reviewers. There are attempts to try different styles (open review, double blind) but, at the end of the day, it’s not really a major problem. The good papers will still get published and the bad ones that do get through will either get ignored or someone will publish something else that illustrates the errors. So, it’s not perfect but it doesn’t stop the good papers and doesn’t result in bad papers getting much more exposure and credibility than they deserve.

Academia also has its issues. There is an element of hiring people who fit a certain style. I, for example, don’t think it’s particularly diverse. Gender balance in the sciences is a particular issue that has still to be solved. I’ve heard people say “it’s okay, we only hire excellence and so we are not discriminating”. This sounds good, but often excellence is defined by those already in the system, and – consequently – it’s very hard to avoid unconscious biases. I believe that we should recognise that there are many different characteristic that can make up an excellent academic. So, in my opinion, this is a big issue that I think we should be aiming to do something about. Does it mean, however, that universities are hiring people who aren’t very good? No, I don’t think it does. The people being hired are still very good, do excellent research and, often, are very good teachers (although the balance between research and teaching is another issue that I won’t comment on here). It just means that we aren’t always hiring the best possible people.

What about research funding? Again there are issues here. There are areas that are topical and for which funding is easier than others (even within certain subjects). You’re often judged on metrics that may not be a particularly good indicator of the quality of your research. Your funding success may depend on the choice of reviewers and who’s on the panel. I even think that getting your name known to those on the panel can make a difference. Sometimes you have to try a few times before being successful. Some people don’t hide their disdain for certain areas that they are meant to be judging. So, it’s not perfect. Is it terrible and do we end up funding lots of really poor research? No, I don’t think we do. If anything, there’s probably much more that could be justifiably funded than there is funding available. It’s not perfect, but it’s pretty good and generally – from what I’ve seen – does a fair job of distributing research funding sensibly.

Maybe the biggest concern I have at the moment is that universities appear to be seeing themselves, more and more, as businesses in which income generation is a priority. Of course, you need money to pay for the costs of teaching and research, but there’s a big difference between knowing the research that you would like to be doing and deciding how best to fund that, and using research simply as a way of generating income. Similarly for teaching. Universities now introduce postgraduate degrees as a way of attracting overseas, fee-paying students. Are these degrees worth having? I’m sure they’re valuable and fine, from an academic perspective. Would we teach all of them if it wasn’t for the fee-paying students? Probably not. From what I’ve seen of typical climate “skeptics”, however, most would probably think this is the way universities should go – get out from the control of government and let the free-market operate. Personally, I think it’s a terrible way to go, but maybe I’ll be proven wrong. But – even so – from what I can tell, good universities are still full of good academics, who teach interesting and useful material to good students who will one day do interesting and valuables things because of what they’ve learned.

Anyway, this post has maybe gone off at a bit of a tangent. The point I was trying to make is that even though there are problems with peer-review, academia, and research funding, none of these problems means that we have poor researchers, publishing weak research in agenda-filled publications. There are things that could be done better, but that’s true of any complex system. Should we try to improve aspects of the system? Of course. Is it broken in some major way? I don’t think so. We should always aim to do things better and try to avoid biases and avoid being influenced by anything that will damage our ability to do good research. We don’t always succeed, but – by and large – it’s a pretty good system that could do with some tweaking (admittedly, more in some areas than in others). Climate “skeptics/contrarians” are entitled to their opinions, but it would seem silly to assume that those with no real experience of any aspect of academic research, are somehow able to make profound judgements about it’s credibility.