If it seems obvious, it probably isn’t

There’s an interesting paper that someone (I forget who) highlighted on Twitter. It’a about when science becomes too easy. The basic idea is that there are pitfalls to popularising scientific information.

Compared to experts,

laypeople have not undergone any specialized training in a particular domain. As a result, they do not possess the deep-level background knowledge and relevant experience that a competent evaluation of science-related knowledge claims would require.

However, in the process of communicating, and popularising, science, science communicators tend to provide simplified explanations of scientific topics that can

lead[s] readers to underestimate their dependence on experts and conclude that they are capable of evaluating the veracity, relevance, and sufficiency of the contents.

I think that this is an interesting issue and it partly what motivated my post about public involvement in science.

However, I am slightly uneasy about this general framing. I think everyone is a layperson in some contexts; it’s not as if scientists (physicists, in particular) are immune from thinking that they can suddenly step into a new field without having developed the deep-level background knowledge. I think most are susceptible to thinking that they understand something better than they actually do.

One issue is that the norms of science are often presented as suggesting that one shouldn’t judge a scientific idea on the basis of who presents it. In a sense, science is open to all. However, this doesn’t mean that people can suddenly step into a field and make some kind of substantive contribution without developing the requisite skills and knowledge. Most fields require years of study. We really shouldn’t pretend otherwise.

My own view of the issue is that we probably don’t spend quite enough time explaining what it takes to do research, and how science is actually done. We often, as the paper suggests, tend to make it sound like everything is quite simple and straightforward, when this is typically far from the truth. Not only is the actual topic often more complicated than it sometimes seems, the processes involved are also typically quite complicated.

Rarely is it as simple as collect data, analyse data, present results. The instruments typically need to be calibrated, the data needs to be properly assessed, and the analysis methods need to be carefully checked. There might be models that also need to have been developed and tested. In fact, one of the hardest parts of doing research is deciding what question should be asked. It takes time, effort, experience, and often requires a set of skills that take years to develop.

If there was one thing that I would stress, it’s that if you think someone has missed something obvious, or made some kind of obvious mistake, they probably haven’t. This is especially true if you’re considering an entire research field. It’s not impossible that a layperson could notice something obvious that a large group of experts have missed, but it’s extremely unlikely.

So, even if science communicators have done too good a job of making science accessible, I suspect the issue of over-confident laypeople is much more complicated than that; hubris is a not uncommon human trait. My own view is that it’s important to make scientific information available and accessible. Maybe what we should do more of is make it clear that the process through which we develop scientific knowledge is far more complicated than it may, at first, seem. If it looks as though a group of experts have missed something obvious, it’s probably more complicated than it seems, and they probably haven’t.

Advertisements
This entry was posted in Pseudoscience, Research, Science, Scientists, Sound Science (tm), The philosophy of science, The scientific method and tagged , , , , , . Bookmark the permalink.

44 Responses to If it seems obvious, it probably isn’t

  1. As a statistician (of sorts) I frequently work with scientists in other fields, and one of the things I like most about statistics is learning enough about their science that I can understand what they are talking about, without them having to dumb it down so much that the fundamental research aspects of the work are lost. But of course that doesn’t mean that I understand the science at anything like their level, any more than they know about my statistical research.

    It is monumental hubris to think that the worlds climatologists have got the fundamental science wrong at the most basic level for 200 years, but that is the position that many climate skeptics seem confident to adopt!

  2. John Hartz says:

    ATTP: You did not mention the first rule of effective communication, “Know thy audience.” which translates to “Different strokes for different folks.”

  3. John Hartz says:

    ATTP: Here’s one way of communicating that you may not have envisioned when you wrote your OP.

    The show then cut to a segment with Nye—safety glasses on—standing behind a desk, on top of which stood a globe, a fire extinguisher a bucket of sand and a fire blanket.

    “By the end of this century, if emissions keep rising, the average temperature on Earth could go up another four to eight degrees. What I’m saying is the planet’s on f****** fire,” Nye said while nonchalantly setting the globe alight with a blowtorch which had been hidden under the desk.

    “There are a lot of things we could do to put it out, are any of them free?” he asked, gesturing towards the other items on the desk. “No of course not, nothing’s free you idiots, grow the f*** up. You’re not children any more. I didn’t mind explaining photosynthesis to you when you were 12 but you’re adults now and this is an actual crisis. Got it? Safety glasses off motherf******.”

    https://www.newsweek.com/bill-nye-global-warming-adults-planet-fire-1423540

  4. Paul Tikotin says:

    I can’t imagine anyone believing that science is a process where we collect data, analyse data, present results.

    Without a theory, data is just a collection of facts. How would we analyse data in the absence of a theory?

    Framing a theory is a fundamental part of the process. Do that well and you will know what data you should be collecting and how to frame your analysis.

  5. Everett F Sargent says:

    So, may the DK be with you.

    The paper you mentioned ….
    When science becomes too easy: Science popularization inclines laypeople to underrate their dependence on experts
    https://journals.sagepub.com/doi/abs/10.1177/0963662516680311?journalCode=pusa
    engages in p-hacking, because, you know I read it somewhere else.

    My scientific toolbox is filled with screwdrivers, that way I can screw over those SME’s whom display scientific hubris in their own papers and get to display my own scientific hubris at the same time.

  6. Snape says:

    My comments on climate blogs tend to follow a pattern. Someone with some level of background in science, maybe an undergraduate degree in geology, maybe expertise in Meteorology, will display hubris by forcefully disagreeing with experts in a field other than their own. I end up displaying hubris by arguing with them.

    Or maybe not. Is it hubris for a layman to try and defend a position shared by experts?

  7. John Hartz says:

    On this side of the pond, the Pretend President has sucked all of the hubris out of the population. 🙂

  8. well done! there is a lot to chew on with this post.

  9. Everett F Sargent says:

    Well, this appears to be rather timely, or not, from the NAS …

    Reproducibility and Replicability in Science
    https://www.nap.edu/download/25303
    (create a free account to get the free PDF)

    (1) Introduction
    (2) Scientific Methods and Knowledge
    (3) Understanding Reproducibility and Replicability
    (4) Reproducibility
    (5) Replicability
    (6) Improving Reproducibility and Replicability
    and Chapter …
    (7) Confidence in Science 117
    Research Syntheses 117
    Geoscience 119
    Genetics 121
    Psychology 122
    Social Science Research Using Big Data 124
    Public Perceptions of Reproducibility and Replicability 126
    Public Understanding of Science 126
    Public Trust in Science 127
    Media Coverage of Science 128

    For what it is worth, I was unable to reproduce or replicate this study. 😉

  10. John Hartz says:

    ATTP: Another facet of the issue you have raised in the OP…

    The third annual March for Science was in New York City on May 4. What follows here is my keynote speech about political attacks on science, the importance of safeguarding the role of science in policy making, the value of diverse scientists, the role of the ocean in the Green New Deal, and the critical need for building community around solutions.

    We Must Defend Science in the Face of Political Attacks Opinion by Ayana Elizabeth Johnson, Observations, Scientific American, May 13, 2019

  11. Steven Mosher says:

    “thinking that they can suddenly step into a new field without having developed the deep-level background knowledge. I think most are susceptible to thinking that they understand something better than they actually do.”

    I have a odd perspective on this. It occurs to me every time I start to write up results of
    merely collecting data and analyzing it.

    After writing I look at the thing I wrote… and realize… references? shit I have no references
    other papers are chock ful them.. OH ya, other guys actually spend years reading all the literature
    and they can PLACE their research in CONTEXT of what others have done.

    so I start reading..
    and reading..
    and reading..

    Now you might just think that you can merely publish your “findings” I looked and found x,
    But as you read through a science argument a large part of it is placing the research in context, or showing how it fits or doesnt fit with what other people have found.

    Doing the data? shit I have tons of it, knowing how it fits, how it contributes, ect etc, is a full time
    job. You might find an interesting thing or two on your own, but if you dont know the actual state of the science you have no clue how to write a compelling piece of science.

    I know it seems weird to judge a thing by it’s style, but when I look at the style I realize there is no way to “fake” the style ( references ect) without actually reading the background work past the abstract.

  12. Everett F Sargent says:

    Credibility, expertise and the challenges of science communication 2.0
    https://journals.sagepub.com/doi/10.1177/0963662517733368

    “Recently, wide-ranging discussions about so-called ‘post-truth’ have also significantly involved science-related topics and science communication.

    The issue of credibility and reliability of information is obviously central for science communication and public understanding of science. However, some themes deserve more attention in this context.

    We live in a communication environment that is radically different from the past, and nevertheless, we paradoxically continue to invoke traditional forms of certifying the trustworthiness of information. …

    The contemporary communication landscape clearly places new and greater responsibility on researchers and their institutions, who are increasingly active in communication to the ‘end user’ and not always prepared to deal with the dynamics and potential risks of such engagement. …

    We know from several historical, social and public perception studies that much has been changing, both in science and in society and at the intersection of the two. It is increasingly important for our field to raise the question of which communicative processes may have contributed to changes in the cultural and social status of science. … ”

    Read the entire opinion, it’s free. I found it to be interesting and more or less on topic.

  13. snape,

    Is it hubris for a layman to try and defend a position shared by experts?

    I would think that the sensible position, in most cases, is to accept the position presented by experts. They could be wrong, but if someone doesn’t have their own relevant expertise, how likely is it that they will be able to work this out?

  14. snape “Is it hubris for a layman to try and defend a position shared by experts? ”

    Hubris means “excessive pride or self-confidence.” whether it is hubris or not depends on whether your confidence in your understanding of the topic is justified. Having qualifications in atmospheric science doesn’t mean that you are necessarily free from hubris, for example Prof. Murry Salby is a genuine expert on atmospheric Rossby waves, but that doesn’t mean he understands the carbon cycle, and hence his claims about the cause of the post-industrial rise in CO2 are nonsense. You don’t have to move far from your field of expertise before it is appropriate to go back to the start and do your homework.

  15. Reblogged this on Symptoms Of The Universe and commented:
    …And Then There’s Physics’ post on science communication, reblogged below, very much struck a chord with me. This point, in particular, is simply not as widely appreciated as it should be:

    “Maybe what we should do more of is make it clear that the process through which we develop scientific knowledge is far more complicated than it may, at first, seem.”

    There can too often be a deep-seated faith in the absolute objectivity and certainty of “The Scientific Method”, which possibly stems (at least in part) from our efforts to not only simplify but to “sell” our science to a wide audience. The viewer response to a Sixty Symbols video on the messiness of the scientific process, “Falsifiability and Messy Science”, brought this home to me: https://muircheartblog.wordpress.com/2018/07/08/the-truth-the-whole-truth-and-nothing-but/

    …but I’ve worried for a long time that I’ve been contributing to exactly the problem ATTP describes: https://muircheartblog.wordpress.com/2013/08/09/philip-moriarty-perform-or-perish-guilty-confessions-of-a-youtube-physicist/

    By the way, if you’re not subscribed to ATTP’s blog, I heartily recommend that you sign up right now.

  16. David B. Benson says:

    Advection versus convection.

  17. Marco says:

    “…lead[s] readers to underestimate their dependence on experts and conclude that they are capable of evaluating the veracity, relevance, and sufficiency of the contents.”

    I see similarities with the Dunning-Kruger effect here…

  18. Snape says:

    ATTP, DM
    Some of the viewpoints I’ve come across within the past year or so:

    – water vapor has an overall cooling effect.
    – A few degrees warming will be almost unnoticeable.
    – monthly anomalies should only be reported to a 10th of a degree.
    – climate change played almost no part in California Wildfires
    – global warming is the result of a natural cycle – deep ocean warmth moving upwards, replaced by cooler water cycling downwards.
    – the “lazy jet stream” hypothesis should be fully embraced, even if the IPCC has a more conservative stance.

    All of the bloggers who held these opinions had at least a 4 year degree in some area of science, followed by science related careers. My training amounted to a combined handful of math and science courses in college, yet I argued with all of them. Hubris on their part for sure (none were experts in the topic discussed) and probably on mine too for even taking part.

    Now, my taking part may have been justified if I understood and presented the science correctly. Not hubris. The problem is, I’m not qualified to make that call.

  19. John Hartz says:

    Snape:

    Thanks for all that you do.

    Re the academic credentials of bloggers, always take what they say about their backgrounds with a large dollop of salt.

    Re the soundness of your responses to them, you can find a science-based rebuttal for just about any climate science denier myth on SkepticalScience.com.

  20. Snape says:

    John
    Thanks, but it’s all for fun. I don’t think I’ve ever changed someone’s mind.

  21. John Hartz says:

    Snape: You may not be able to change a climate science denier’s mind, but you may be able to educate onlookers.

  22. John Hartz said:

    “Snape:

    Thanks for all that you do.”

    Agree, thanks from here too. Snape has been battling the AGW skeptics for years as I recall.

  23. John Hartz says:

    Back in the day when I crossed swords with climate science deniers on the comment threads of articles, I invariably came across posts by a mean-spirited blogger, “meme” — supposedly a religious fanatic living in Canada. I now wonder whether or not “meme” was a prototype Russian bot. Regardless, the Russians could have easily used the climate commentary wars to create and hone their bot expertise.

  24. “go emeritus” is basically what happens when senior academics lose their self-skepticism and spoil their academic reputation by promulgating and “unusual” hypotheses (not difficult to think of examples). The phrase is borrowed from an interesting Science Fiction book called “The Languages of Pao” (Jack Vance?). It seems reasonable to suggest this happens for a variety of reasons, but I suspect age-related change in cognitive function can easily be one of them (at fifty I find it a bit more understandable than when I was a freshly minted academic half a lifetime ago). I would venture this has become a far greater problem with the development of the WWW which makes finding a receptive audience for the theory so much easier. I think we have a duty to treat people who have “gone emeritus” the way we would like to be treated if it happened to us (in my case have my errors clearly explained to me, but with consideration, rather than derision).

  25. [Mod] I’d quite like that last comment to stay if possible as I think it is relevant to the topic of hubris even without the now deleted context, however I understand if it is deleted.

  26. Dikran,
    Yes, that one seems fine. I’m less comfortable with discussions that reflect on the sanity of actual individuals.

  27. Yes, me too. Much appreciated!

  28. John,
    Russia has been involved in the climate arena for years, they make a lot of money from it.
    https://www.nytimes.com/2014/12/01/world/russian-money-suspected-behind-fracking-protests.html

  29. John Hartz says:

    jeffnsails850:

    Thanks for the link. That particular article snuck under my adar in 2014,

  30. John Hartz says:

    Seems to me that it’s now time to hit the panic button…

    It was 84 degrees near the Arctic Ocean this weekend as carbon dioxide hit its highest level in human history by Jason Samenow, Capital Weather Gang, Washington Post, May 14, 2019

    PS – Please pass the Prozac.

  31. Dave_Geologist says:

    “The Languages of Pao” (Jack Vance?). Correct, dikran.

    And then, inexorably the dominie would approach his Emeritus status: he would become less precise, more emotional; egocentricity would begin to triumph over the essential social accommodations; there would be outbursts of petulance, wrath, and a final megalomania—and then the Emeritus would disappear.

    Finisterle propounded another apparently paradoxical law of nature: “The more forceful and capacious the brain of a dominie, the more wild and violent its impulses when it succumbs to sclerosis and its owner becomes an Emeritus.”

    Reason? Age-related decline you touched on, and was Vance’s explanation. In some cases, where an academic is in decline before retirement age but had a stellar career or was perhaps a really good department head or “rain-maker”, I suspect it is used as a way to ease someone out of (causing-)harm’s way without too many hurt feelings. That can happen well before 67. I had an aunt (non-blood-relative) whose dementia began at 60. It was a familial thing and different from Alzheimer’s, which when I read around I found has led to controversies over whether studying individuals and families with that (because of high signal-to-noise and the ability to explore the relevant genes, and to test early intervention when the body is still physically sound) is a red-herring when it comes to understanding and treating late-onset dementia. Not that it’s wrong to treat those victims of course: it’s just that, like it or not, niche minority ailments don’t get big-bucks funding.

    The “bigger-they-are, harder-they-fall” aspect is probably down to ego and to there being more of an audience for a big name. Pauling and Vitamin C, for example. I could see the level of biological and medical knowledge he had to acquire to place his work on chemical bonds and crystal structures in context as analogous to dikran’s comment on what he has to learn to help researchers with their statistics. A little knowledge can be a dangerous thing, at least if it’s not coupled with humility. A scientist dabbling in another field can be as much a fish out of water as a layperson

    “The devil finds work for idle hands” probably also applies. Most people take up some sort of hobby when they retire. That’s their’s.

  32. “A little knowledge can be a dangerous thing, at least if it’s not coupled with humility.”

    If only humility was easy. It’s taken me years to become the world’s best at it. ;o)

  33. Dave_Geologist says:

    I sometimes use palaeontology as an example (I think this is my first time here though). I had a reasonable grounding as an undergraduate, although some of it is out of date. IIRC Dinoflagellates were still classed as Problematica in the Treatise of Palaeontology, it being unclear whether they were carnivorous plants, or animals which had co-opted photosynthesisers. A single-celled Venus flytrap, or a single-celled coral? Or neither, because they’re intermediate in some respects between eukaryotes and prokaryotes. So maybe the last common ancestor dates to before plants were distinct from animals. But there I go, speculating outside my knowledge-base 😉 . I spent a lot of time in industry working closely with palaeontologists (who often made use of dinoflagellate cysts for correlation and to determine palaeo-environment), and attended various courses to keep me up to date with what I needed to know of developments since I’d graduated. We called dinos plants, although I see they still appear in both plant and animal classification schemes. Like my take on wavicles, perhaps the right answer is “neither”. I also maintained a lifelong interest, to a degree where I can read and reasonably well understand academic papers (I’m currently half-way through Hadrosaurs (Life of the Past), which despite its pretty cover is a collection of dry academic papers, not a coffee-table book, and an absolute bargain as an e-book).

    So, I’m not a palaeontology specialist, but how much more do I know and, more important, understand about the subject than Joe Public, who’s just read some popular books or seen some TV shows? Probably about a hundred times, I reckon. My professional colleagues probably knew 100 times what I did, although in a narrow, specialist area. If they were very job-focused, I probably knew more about dinosaur phylogeny than they did. However, they had the skills to challenge a dinosaur cladogram if they wanted to, where I just have to rely on peer-review. My near-contemporary, who was doing a post-doc when I was doing my PhD and is now an internationally renowned museum scientist, probably knows 100 times as much as my industry colleagues. Mainly because his career has been all about doing new stuff, in a broader area, as opposed to doing the same thing again and again but doing it very well (their middle name is Unix 🙂 ). Joe Public probably can’t conceive of the idea that there is someone who knows a million times more about a subject than he does. Especially one like palaeontology, which is perhaps the poster-child for science which can be and has been popularised. Joe thinks he’s pretty knowledgeable about that. After all, his mates scoff when he tells them birds are dinosaurs, but he knows he’s right and they’re wrong. Because I know just how much more I know than Joe does, but also how much more the real experts know, and know just how dumbed-down even the best popular books and TV are, I have no problem conceiving of knowledge gaps covering many orders of magnitude. And that’s without even getting into areas were I’m an actual expert.

  34. John Hartz says:

    For “There has to be law..” thread:

    The longer a comment thread is active, the more likely the discussion will become esoteric. 🙂

  35. Dave said:

    “In some cases, where an academic is in decline before retirement age but had a stellar career or was perhaps a really good department head or “rain-maker”, I suspect it is used as a way to ease someone out of (causing-)harm’s way without too many hurt feelings. That can happen well before 67.”

    This CBS News (and longer 60 Minutes) story from last week on FTD. Scary stuff

  36. Dave_Geologist says:

    Dinos are one esoterica (esotericum?) we’ll have to get used to John. They love hothouse conditions. Dead Zones, Red Tides etc.

    Alexandrium fundyense is a species of dinoflagellates. It produces toxins that induce paralytic shellfish poisoning (PSP), and is a common cause of red tide.

    Some species nibble on fish, to the point where they’re nibbled to death if they aren’t suffocated first (since they’re unicellular that tends to be called parasitism, with predation reserved for when they engulf other single-celled organisms). The oil industry treats them as plants because it’s the keratinous cysts we use, which are processed in acid in the same way as pollen and identified by a palynologist. Micropalaeontologists deal with calcified organisms so don’t process using strong acids. Drillers hate palynologists because they use HF, which brings a bunch of HSE hurdles with it.

    They bloomed during the PETM. The North Sea was one vast, permanent Red Tide. It would have been unfishable, uncrossable except in specially protected boats or submarines, and its coastal regions would have been unlivable due to toxic fumes. The seabed was euxinic, like the Black Sea, contrasting with an oxygenated seabed before and after. The PETM Sele Shale is black and laminated, unlike the underlying Lista Shale, because there were no bottom-dwelling burrowers to stir it up and eat the infalling organic matter. Karma struck, because that makes it very difficult to drill other than vertically. Not just the lamination, but even bits solid enough to cut a one-inch core sample out of have a six-to-one strength anisotropy. It’s reckoned to have cost the North Sea oil industry a billion dollars in lost drilling time. I’ve looked at a lot of Sele Shale in core, and seen nary a fish fossil. Unlike, for example, the euxinic Devonian lake deposits of northern Scotland, where you can’t avoid tripping over perfectly preserved fish. So maybe the entire water column was unlivable by other than single-celled organisms. As I’ve said many times about the PETM, we really, really don’t want to go there.

  37. John Hartz says:

    Dave: After esoteric comes goofy. 🙂

  38. Dave_Geologist says:

    Thanks for the link Paul. As you say, very scary stuff. More shocking than Alzheimers because it came on very quickly, and she’d been very active physically and intellectually – a head teacher, Club-Medal-competing golfer. She had relatives who were struck down younger. They never had children, and I’ve wondered (but not asked although my uncle is still alive, lives nearby and I see him often) if that was a deliberate choice. The prospect of seeing your adult child succumb to dementia when you were still OK would make me think twice, especially as in some cases there’s a 50/50 chance of each child having the faulty gene. Nowadays there are genetic tests for at least some variants, but then you’d be going into difficult territory around selective IVF, testing the foetus and perhaps an abortion, when actually the condition might not have struck until the mid-60s, by which time the child would have had a full and fruitful life.

    I’d obviously done some reading about it at the time (but, trying to stay on-topic, I’m definitely not an expert), and from a quick Google to catch up, see that there’s still controversy about it not getting the research bucks, other than as an “in” to understanding more profitable (and, in fairness, more prevalent, late-onset dementia), and resentment that the “guinea pigs” are not benefiting from the research.

  39. Joshua says:

    Related:

    In this age of emotional political conflict, there is less and less to agree upon. Experts are no longer respected as impartial; public debate is reduced to attack and counter-attack; the boundary between facts and propaganda seems to be dissolving. We live in a world not quite at war but nor exactly at peace.

    How did things reach this point, and what can we do about it? In this enlightening, far-reaching and provocative book, William Davies explores how physical and emotional feeling came to reshape our world today, destabilising governments and placing us all on high-alert. Drawing on a 400-year history of scientific and political ideas, he shows how our sensations were once treated with suspicion, before being seized enthusiastically as a path to mass mobilisation in war.

    As we enter a new technological and political era, this book reveals the origins of the nervous states in which we now live.

    Related:

    https://www.theguardian.com/books/2018/nov/24/nervous-states-how-feeling-took-over-the-world-william-davies-review

  40. Joshua says:

    Related:

  41. dikranmarsupial says:

    @dave_geologist – I also have an interest in palaeontology – it was a childhood ambition achieved when I published a journal paper (well an extended comment) on dinosaurs (on allometry for estimating body mass) ;o)

    I occasionally get my self an undergraduate textbook on palaeontology (mostly invertebrate), or cosmology, or climate, or physics, or ….

  42. Dave_Geologist says:

    Heheh. One of my long-ago interns is active in allometry and numerical biomechanics, and referenced in the Hadrosaur book I’m reading. She was very numerate (by geology standards) so we gave her a project trying to derive geological insights from analytical well tests (engineers tend to stop at spherical, cigar-shaped or sugar-cube cows). She didn’t fancy the oil industry, but I’m glad she made it in academia.

  43. dikranmarsupial says:

    Fortunately not all palaeontologists are as numerically adept :o)

    Not really related, but:

  44. Thanks for this post: I’d been trying to put this sentiment into words in a way that wouldn’t be interpreted as arrogant or condescending.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.