A while ago I wrote a post about a Nature commentary on Research Integrity and transparency, by Stephan Lewandowsky and Dorothy Bishop. Warren Pearce and colleagues wrote a brief response and have since expanded on this in a blog post.
Something that struck me when I read the Pearce et al. response is that I’d forgotten how antagonistic (okay, maybe too strong, but I can’t think of another term) debates can be in this context; I don’t think that the Pearce et al. response has anything positive to say about the Lewandowsky and Bishop article. This is a little surprising given that one of the Pearce et al. criticisms seems to be that one should be careful of over-simplifying what is a complex issue. My interpretation of the Lewandowsky and Bishop article is that we should indeed be careful of simplistically thinking that transparency in research is some kind of panacea. We shouldn’t think that more transparency will mean that science will somehow be free from attack, or that it will somehow make people more trusting of science in general. So Pearce et al. are correct that we should be careful of over-simplifying this issue, but it’s not clear in what way Lewandosky and Bishop did so.
Pearce et al. then discuss the issue of experts, saying:
the fundamental question is who counts as an expert, and under what conditions?
I realise that there might not be some kind of clear boundary between expert and non-expert, but surely it’s pretty easy to establish if someone qualifies as an expert, or not. Of course, it is quite possible for someone who isn’t formally an expert to make a positive contribution to a field, either by bringing some kind of fresh perspective, or by engaging with scientists who are then forced to think more about what they’re presenting, how they’d doing so, and the significance of their research. None of this, however, suggests that making all of one’s data and codes available is likely to lead to some layperson noticing something obvious that experts have missed. I agree that this is not an argument against doing so, but – similarly – doesn’t seem like a particularly good argument for doing so.
Pearce et al. then discuss the role of the public and suggest that [a] more fruitful approach to addressing public doubts about science was proposed by David Demeritt in 2001 who said:
“The proper response to public doubts is not to increase the public’s technical knowledge about and therefore belief in the scientific facts of global warming. Rather, it should be to increase public understanding of and therefore trust in the social process through which those facts are scientifically determined. Science does not offer the final word, and its public authority should not be based on the myth that it does, because such an understanding of science ignores the ongoing process of organized skepticism that is, in fact, the secret of its epistemic success. Instead scientific knowledge should be presented more conditionally as the best that we can do for the moment. Though perhaps less authoritative, such a reflexive understanding of science in the making provides an answer to the climate skeptics and their attempts to refute global warming as merely a social construction.”
I actually don’t find much to disagree with in the above quote; I think that a better understanding of the scientific process would be of benefit. However, it seems that my interpretation of the above is somewhat different to that of Pearce et al. One thing that concerns me about the drive for greater transparency is that it could lead to a greater trust in individual studies. I don’t think that this is necessarily a good thing. Typically we start to trust our understanding of a scientific topic when results from different studies by different people/groups start to converge in some way.
We don’t increase our trust in a particular study simply because we can’t find an error and – similarly – we shouldn’t necessarily dismiss a study because someone finds a mistake. Similarly, we shouldn’t trust something more because the authors have been completely transparent, and shouldn’t necessarily dismiss a study because the authors have not been as transparent as we might have like; we trust the overall scientific method, not individual studies or individual researchers. That to me is the social process through which those facts are scientifically determined. So, I don’t necessarily see anything in the above quote that is at odds with what was being suggested by Lewandowsky and Bishop.
Pearce et al. finish their post by saying
What is noticeable is how little these social sciences critiques have cut through to those in the natural sciences.
Well, there may be some reasons for this. From what I’ve seen, some of the social science critiques appear to be coming from those who seem to think that they’re in some kind of special position where they can observe and critique the natural/physical sciences. Why? We’re all researchers. Most of us work in the same environments with the same pressures and incentives. We all potentially suffer from biases. We can all make mistakes. We’re all expected to engage with the public and, potentially, with policy makers. So, maybe some natural scientists just don’t really see why they should pay much attention to some social scientists who seem to think they’re in some kind of position to critique how natural/physicists undertake their research. It also doesn’t help when some social scientists make it fairly clear that they don’t really understand what’s being presented by natural/physical scientists.
In my view, if social scientists want their critiques to be taken seriously by natural/physical scientists, they should put more effort into engaging with them directly, rather than appearing to be standing back and simply observing. On the other hand, I do remember a post in which some in the comments suggested that I simply didn’t understand what was being presented by the social scientists. This may be true, but – if so – I was not the only physicist to misunderstand what was being presented, and might suggest that social scientists are not putting sufficient effort into making their critiques understandable.
I’ll end by quoting their final two sentences
To be clear, there is no excuse for ignoring the existing evidence base. However, we believe that social scientists must be more proactive in using that evidence base in order to lead the debate from a position of strength.
What evidence and why should social scientists be aiming to lead the debate from a position of strength? I don’t really understand what is being suggested here, but maybe this is an opportune moment to re-highlight Michael Tobis’s post about swimming in your own lane.