118 — Science and Decision Making under Uncertainty, A Conversation with Prof. John Ioannidis

In this episode, I had the privilege of speaking with John
Ioannidis, a renowned scientist and meta-researcher whose
groundbreaking work has shaped our understanding of scientific
reliability and its societal implications. We dive into his
influential 2005 paper, Why Most Published Research Findings Are
False, explore the evolution of scientific challenges over the
past two decades, and reflect on how science intersects with
policy and public trust—especially in times of crisis like
COVID-19.


John had and has major impacts in our understanding of medical
research, research quality in general, public health and how to
handle critical situations under limited knowledge. His work was
and is highly influential and extraordinarily important for our
understanding where medical science, but really, science in
general is standing, how we dealt with the Covid crisis, and how
we could get our act together again. He is professor of medicine
at Stanford, expert in epidemiology, population health and
biomedical data science.


We begin with John taking us back to 2005, when he published his
paper in PLOS Medicine. He explains how it emerged from decades
of empirical evidence on biases and false positives in research,
considering factors like study size, statistical power, and
competition that can distort findings, and why building on shaky
foundations wastes time and resources.


“It was one effort to try to put together some possibilities, of
calculating what are the chances that once we think we have come
up with a scientific discovery with some statistical inference
suggesting that we have a statistically significant result, how
likely is that not to be so?”


I propose a distinction between “honest” and “dishonest”
scientific failures, and John refines this. What does failure
really mean, and how can they be categorised?


The discussion turns to the rise of fraud, with John revealing a
startling shift: while fraud once required artistry, today’s
“paper mills” churn out fake studies at scale. We touch on cases
like Jan-Hendrik Schön, who published prolifically in top
journals before being exposed, and how modern hyper-productivity,
such as a paper every five days, raises red flags yet often goes
unchecked.


“Perhaps an estimate for what is going on now is that it accounts
for about 10%, not just 1%, because we have new ways of massive…
outright fraud.”


This leads to a broader question about science’s efficiency. When
we observe scientific output—papers, funding—grows exponentially
but does breakthroughs lag? John is cautiously optimistic,
acknowledging progress, but agrees efficiency isn’t what it could
be. We reference Max Perutz’s recipe for success:


“No politics, no committees, no reports, no referees, no
interviews; just gifted, highly motivated people, picked by a few
men of good judgement.”


Could this be replicated in today's world or are we stuck in red
tape?


“It is true that the progress is not proportional to the massive
increase in some of the other numbers.”


We then pivot to nutrition, a field John describes as “messy.”
How is it possible that with millions of papers, results are
mosty based on shaky correlations rather than solid causal
evidence? What are the reasons for this situation and what
consequences does it have, e.g. in people trusting scientific
results?


“Most of these recommendations are built on thin air. They have
no solid science behind them.”


The pandemic looms large next. In 2020 Nassim Taleb and John
Ioannidis had a dispute about the measures to be taken. What
happened in March 2020 and onwards? Did we as society show
paranoid overreactions, fuelled by clueless editorials and media
hype?


“I gave interviews where I said, that’s fine. We don’t know what
we’re facing with. It is okay to start with some very aggressive
measures, but what we need is reliable evidence to be obtained as
quickly as possible.”


Was the medicine, metaphorically speaking, worse than the
disease? How can society balance worst-case scenarios without
paralysis.


“We managed to kill far more by doing what we did.”


Who is framing the public narrative of complex questions like
climate change or a pandemic? Is it really science driven, based
on the best knowledge we have? In recent years influential
scientific magazines publish articles by staff writers that have
a high impact on the public perception, but are not necessarily
well grounded:


“They know everything before we know anything.”


The conversation grows personal as John shares the toll of the
COVID era—death threats to him and his family—and mourns the loss
of civil debate. He’d rather hear from critics than echo
chambers, but the partisan “war” mindset drowned out reason. Can
science recover its humility and openness?


“I think very little of that happened. There was no willingness
to see opponents as anything but enemies in a war.”


Inspired by Gerd Gigerenzer, who will be a guest in this show
very soon, we close on the pitfalls of hyper-complex models in
science and policy. How can we handle decision making under
radical uncertainty? Which type of models help, which can lead us
astray?


“I’m worried that complexity sometimes could be an alibi for
confusion.”


This conversation left me both inspired and unsettled. John’s
clarity on science’s flaws, paired with his hope for reform,
offers a roadmap, but the stakes are high. From nutrition to
pandemics, shaky science shapes our lives, and rebuilding trust
demands we embrace uncertainty, not dogma. His call for dialogue
over destruction is a plea we should not ignore.


Other Episodes


Episode 126: Schwarz gekleidet im dunklen Kohlekeller. Ein
Gespräch mit Axel Bojanowski

Episode 122: Komplexitätsillusion oder Heuristik, ein
Gespräch mit Gerd Gigerenzer

Episode 116: Science and Politics, A Conversation with Prof.
Jessica Weinkle

Episode 112: Nullius in Verba — oder: Der Müll der
Wissenschaft

Episode 109: Was ist Komplexität? Ein Gespräch mit Dr. Marco
Wehr

Episode 107: How to Organise Complex Societies? A
Conversation with Johan Norberg

Episode 106: Wissenschaft als Ersatzreligion? Ein Gespräch
mit  Manfred Glauninger

Episode 103: Schwarze Schwäne in Extremistan; die Welt des
Nassim Taleb, ein Gespräch mit Ralph Zlabinger

Episode 94: Systemisches Denken und gesellschaftliche
Verwundbarkeit, ein Gespräch mit Herbert Saurugg

Episode 92: Wissen und Expertise Teil 2

Episode 90: Unintended Consequences (Unerwartete Folgen)

Episode 86: Climate Uncertainty and Risk, a conversation with
Dr. Judith Curry

Episode 67: Wissenschaft, Hype und Realität — ein Gespräch
mit Stephan Schleim



References


Prof. John Ioannidis at Stanford University 

John P. A. Ioannidis, Why Most Published Research Findings
Are False, PLOS Medicine (2005)

John Ioannidis, A fiasco in the making? As the coronavirus
pandemic takes hold, weare making decisions without reliable data
(2020)

John Ioannidis, The scientists who publish a paper every five
days, Nature Comment (2018)

Hanae Armitage, 5 Questions: John Ioannidis calls for more
rigorous nutrition research (2018)

John Ioannidis, How the Pandemic Is Changing Scientific
Norms, Tablet Magazine (2021)

John Ioannidis et al, Uncertainty and Inconsistency of
COVID-19 Non-Pharmaceutical1Intervention Effects with Multiple
Competitive Statistical Models (2025)

John Ioannidis et al, Forecasting for COVID-19 has failed
(2022)

Gerd Gigerenzer, Transparent modeling of influenza incidence:
Big data or asingle data point from psychological theory? (2022)

Sabine Kleinert, Richard Horton, How should medical science
change? Lancet Comment (2014)

Max Perutz quotation taken from Geoffrey West, Scale,
Weidenfeld & Nicolson (2017)

John Ioannidis: Das Gewissen der Wissenschaft, Ö1 Dimensionen
(2024)