Her lines start with a Capital B, mine are lower case and in italic.
* * *
B. I am available today for ex but I know it’s late notice. Once I know how next week looks, I’ll let you know!
could be later on if you want to get it over, i have an outline
B. Ah sure!
could be soon
B. I’m here largely doing nothing Smiling face with open mouth and cold sweat
could be now let me get a coffee
B. Cool cool! I got mine, sure!
btw i’m on my laptop and my space key sticks i try tocorrect but ifi miss i hope youcatch them see this is how it is when i don’t correct
ah, ok i’ll send you the transcript beforehand now, let’s see my notes
*becomes more serious*
still on the boat?
B. Not today
Today at friends’ place
Ah we started 🙂
OK I can be more explicit and clear than that. We’re at the boat 3-4 days a week but unfortunately the marinas around here have live aboard limitations 😦
where are you
B. San Mateo, CA
so you made it all around the world
B. That’s sort of an open wound. We made it here from northern WA coast and were supposed to be heading south to Mexico right about now. But Covid plus some technical problems on the boat led to a change of plans. Right now we’re trying to figure out what we can/cannot realistically do this year.
We = My husband, our two cats, and myself
i thought i caught a glimpse of you in turkey, where you’re from
B. I was there last summer. I don’t get to visit home that often. Maybe 4-5 times in total in the last 17 years.
so you mix and match the photos you post or i misremember
B. But we sure hope to sail there sometime. It’s a matter of timing. Yeah I may have posted something but I can’t remember either. I post too many photos I suppose and sometimes out of order.
my favorite ones are of course your dishes and your cocktails we talked about this chat around the time i wrote about the Auditing Problem
B. Ah yes we did. I remember that post!
that post was about replication, you wrote a piece on this with friends or colleagues
B. Yes. I have by now 4 papers that somewhat touches on or is directly related to replication and reproducibility issues. Two preprints, two published. I have not written anything specifically on replication markets other than on twitter though.
can you summarize your main takeaways
B. Can I start a bit earlier? From our motivation?
you can start wherever you please
B. Thank you 🙂
When I got first interested in the replication crisis and the following developments, I was very much fed up with my own field. My PhD was on experimental consumer behavior, which is sort of a spin-off of social psychology with a marketing flavor. Anyway it is a field experiencing the same problems social psych has. And I knew that wasn’t what I wanted to continue doing.
Luckily I also have a Masters degree in Stats and a desire to put that to better use. Also great collaborators.
At first I was following the replication conversations with delight, hoping that this meant things could change. But then I felt like these new discussions were still more superficial than I would have liked to see and somewhat hasty at replacing new heuristics with new ones. To me it wasn’t all that different from everything that preceded it. Still lots of talking, not a whole lot of depth. When people referred to reproducibility of results, I didn’t see a clear, precise definition. I didn’t see the ideas carefully evaluated, dissected. I didn’t feel like the right questions were being asked. I wanted to see the basics first, the foundations laid out.
B. Anyway, this desire to actually understand the dynamics driving the state of affairs we observe was the motivation behind my work. I wanted to find out for myself what reproducibility meant. Why it mattered. Or when it did. Whether we can tell anything meaningful about what we observe in replication studies. Whether it would be actually good for science to pursue higher reproducibility. etc.
I don’t or rather I can’t take any of these for granted. Work needs to come before insights. I observe lots of insights not based in anything but faith. So I digress. What was the question again?
that looks like a good justification for that paper
B. Ah cool the first paper 🙂 Yes! Exactly.
I’m both very fond of and proud of that paper. It’s everything I wanted it to be. Probably has something to do with interdisciplinarity of the team and 4 years of work behind the scenes.
4 years is a lot of work; it’s my favorite paper, i think
B. Let me find the second. It’s a homeless preprint. There are two versions and I like the first better. But it’s a weird one. Doesn’t follow the usual template.
This is somewhat of a conceptual analysis of some ideas about open science and reproducibility using basic statistical concepts.
ah, i thought it was the same
B. Tells me I need to learn how to write better titles.
well, you got to sell your model centrism
why do you call your approach model-centric
B. Yes that was the core idea in the first two papers really. That we need to get away from focusing on hypotheses so much. Model-centric approach is much richer and it subsumes hypotheses.
what is the big difference
B. Good question. Let me think. I think it’s the way one sees how science makes progress. One way of viewing thinking about it is: We keep testing and rejecting hypotheses. These tests would guide us toward some ultimate truth (or an approximation of it) by accumulating loosely related facts. Another is: We keep updating our view of the world (or phenomena of interest). That is, we make scientific progress by exploring, selecting, comparing, evaluating, updating our models.
When you think in terms of models, you have to acknowledge the underlying assumptions, the variables of interest, the parameters, the constants, the model structure… It’s a complete picture. In a lot of hypothesis-centric literature, you cannot find a single assumption explicitly acknowledged. It’s obscure, underdetermined, underspecified, and not quite theoretically motivated. At least in the part of social science I am familiar with, which may not generalize outside. It seems to focus too much on binary outcomes and doesn’t talk much about how they relate to the phenomena. And doesn’t quite explain what’s going on. That was wordy and not very clear, I’m afraid but I’m not writing a paper so I’ll cut myself some slack 🙂
allow me to help: your paper’s main result is that “the scientific process may not converge to truth even if scientific results are reproducible and that irreproducible results do not necessarily imply untrue results” — so there is a need to accept that science is an open-ended quest
B. That’s one, for sure. Some others have been received much more interest though! Absolutely. Yes, I like that way of putting things.
and the model-centric approach seems to be more suited for exploratory research
B. Sure is. I imagine us searching through an infinite model universe. Trying to find the best strategy for an efficient search.
so you don’t dispute that there is a need for confirmatory research,
you just ask that we approach the problem like scientists would, not cops
B. In my view, so much of what we do is description and exploration. But a hypothesis-centric view makes it seem like each hypothesis test is out to confirm something. I think that is deeply flawed.
it echoes an old divide
B. Oh yes, of course. There are times we will need to confirm. And yes, I do not like a cop’s view of science for us.
that seems to lead naturally to your proposal
B. Yes now that you say so, it does seem like a natural transition to this new project. Similar underlying themes, different approaches. I am not a particularly “fun” presenter. I tend to be too serious and frown a lot. Maybe better to read the paper after all!
i like to listen to voices and am a visual learner
and think leaving a textual trace is very important,
but i also think that papers are on their way out,
we should have sites
B. I tried to keep that one accessible to most audiences because I wasn’t sure who would be listening. Also, I am not the theoretical statistician in the team so I cannot pretend to derive the proofs myself.
we should mention this other paper
The case for formal methodology in scientific reform
and there’s the other kind of research we could mention
Robust diversity and modeling in cognitive science
B. I like talking about these ideas though. Obviously because I think they matter but also I know they spark some reaction in the listeners. Some things we talk about are not easy to digest right away.
Ah yes, that’s a more collaborative piece. based on a small workshop organized by Joachim Vandekerckhove and Michael Lee from UC Irvine.
the theme seems to be similar
B. The theme is similar but that’s understandable. I was invited because of that PLOS one paper. Because i had a different voice and new things to say. the idea about the workshop was, which of these new scientific reforms could possibly have a role in cognitive modeling and which did not vibe.
so there’s some kind of fight over computational models
B. I have to declare upfront that I am not a cognitive scientist. I wouldn’t possibly dare speak on their behalf. I have met great cog scientists and collaborated with them recently. But I have a different perspective on things and am not too familiar with their internal wars.
(we’re almost done, btw)
B. Seems like there is an overall tendency to marginalize modelers in psychology though. It’s not considered mainstream work even if it should be.
were i to find a theme, it’d be pluralism,
but then i’m very biased, both toward you and toward pluralism
i might as well plug that presentation by hasok
B. I’d love to talk about pluralism. So far my contribution to these science reform discussions have brought a model-centric and a formal perspective. We do need these perspectives. We cannot have a single approach dominate. That’s another takeaway from the Plos One paper too. Epistemic diversity is the key to avoid traps.
i suppose that’s how we add robustness eric winsburg reached similar conclusions
B. I am a fan of Hasok Chang’s. Inventing Temperature, this talk, … Wonderful!
i find him a very good counterpoint to STS theorizing
who decided that philosophers can’t experiment?
B. It does make sense, no? I was just earlier having a chat with Iris van Rooij about a model she is developing with her colleagues. She uses deterministic models. We use stochastic. She tries to stay away from stats as much as possible. I run toward it
But we’re interested in similar problems. We necessarily have different approaches. At the end our conversation converged on how much we can learn from each other’s approaches while continuing to pursue our own. That’s beautiful to me.
that’s a great way to end the serious part
B. I gotta read that link but have not yet. But I thought philosophers do experiment nowadays. OK let us end it on that inspiring note 🙂
sometimes they do, sometimes they don’t
B. Right of course.
[inaudible digression about philosophy and philosophers]
B. I was just commemorating a review we received on the plos one paper. To paraphrase: “But you just have math and simulations. Where’s the science? Why should we believe you?”
*Where Is Science* is a project i have to make
it’s a good question, but it’s not to be asked for that kind of silly gotcha game
B. Best approach to me is to ignore and disregard all hierarchies. They’re misleading and largely useless.
that makes you happier, and it shows
B. Yup it’s best to not partake of those games at all! It does. I feel kinda outside of the game. Idk why though. Where does this feeling come from?
i can play these games, and my tweeting character is good at that,
it turns these games into an art form
insecurity, mostly feelings of inferiority, the tournament structure, the little payoffs
B. It is an intriguing character. One that makes me feel ok with the anonymity. I have been increasingly suspicious of many anonymous accounts lately.
twitter in general suffers from that
W is a good clown
i like him
B. 🙂 He is likeable.
“no u” would he reply
B. Haha! Idk that I am but I also lack that kind of awareness.
you are yourself, that should be enough
one thing i wonder tho
do you like music?
B. It would be paralyzing for me to think about my twitter as a persona and guess how people perceive it/me. Does anyone say no?
i like to end these chats with music
B. I do! So have I noticed 🙂 Nice!
then hit me with your best shot
B. Well it’s 2020 and the world is going batshit and politics is giving us nightmares everyday. So my cure is Boney M. Rasputin. Or Daddy Cool. Just don a cape, watch, dance like the front man, and have fun. (Also read up about the strange story of the whole project.) I have found that Boney M is not as popular in the US as it is/was in Europe but who can say no to disco?
Just watch and try to not smile.
why would i
so there you have it
wasn’t that hard, was it
B. No it wasn’t 🙂 It was a pleasure. And thank you! Hope I made at least some sense.
you made perfect sense
there was this song in a dance game on the old wii
B. Haha really? Awesome choice! Thank you!
thank you for your time dms are always open if you need to vent,
B. And likewise!
or if there’s a reply guy that bothers you
it’s my social service
B. Thank you 🙂 Haha I did tell someone off just yesterday. They do occasionally show up but I’ve gotten better at managing the trolls in general
i can’t promise i won’t look, but i won’t respond
i am off tweeting until tomorrow
B. Take care 🙂
enjoy your evening
B. Thank you!