Delineamento Experimental - Parte 1: Filosofia da Ciência e Reprodutibilidade
Overview
Bruno Martynhak connects experimental design in animal research to the reproducibility crisis, explaining how publication bias, cherry-picking, and low statistical power can make chance findings appear convincing, as in the Jelly Bean example where one color reaches p < 0.05. He contrasts Francis Bacon’s inductivism with Karl Popper’s falsifiability and Thomas Kuhn’s paradigm shifts, then argues that clear methods and repeatable results are essential for reliable animal experiments and responsible animal use.
Key takeaways
- Testing many hypotheses and publishing only the one that reaches p < 0.05 can produce a seemingly significant result by chance, as the green Jelly Bean and acne example illustrates.
- Low statistical power particularly undermines studies of small effects, which typically require larger sample sizes to detect reliably.
- Popper’s falsifiability distinguishes testable scientific claims, while Kuhn’s account recognizes that anomalies can persist within a paradigm before prompting a scientific shift.
- Animal-to-human conclusions rely on inductive generalization, so findings in mice, rats, or fish do not automatically establish effects in people.
- Reproducibility has two connected requirements: methods must be described well enough for others to repeat them, and similar methods should yield comparable results.
- Unrepeatable animal experiments risk both unreliable science and wasteful animal use, making clear protocols and repeat testing important parts of responsible research.
Chapters
- In the Jelly Bean example, testing many colors produces one apparent association between green jelly beans and acne at p < 0.05, illustrating how selective reporting can create misleading claims.
- Publication bias favors positive findings, so published associations may be overrepresented in the research scientists use to plan new studies.
- Low statistical power means a study has a low chance of detecting an effect; small effects generally require more experimental animals than large effects.
- A small study that reports a small effect may reflect chance or bias rather than a reliable treatment effect.
- Francis Bacon’s inductivism treats observations as the basis for general theories, but inductive conclusions extend beyond their evidence and remain probabilistic.
- The raven paradox, associated with Carl Hempel, shows how an observation such as a non-black water bottle can appear to support the claim that all ravens are black.
- Karl Popper defines scientific claims by falsifiability: they must be open to tests that could show them to be false.
- Thomas Kuhn describes scientific change through paradigms, anomalies, crises, and revolutions; researchers may accommodate conflicting evidence before replacing an established explanation.
- Translating findings from mice, rats, or fish to humans is a major inductive leap, not a conclusion guaranteed by the experiment itself.
- An experiment should produce comparable results when repeated, including within the same laboratory; otherwise, animal use may fail to yield reliable conclusions.
- Reproducibility includes both sufficiently detailed methods for another team to repeat the procedure and similar results when using methods like the original study.
- A Nature survey found researchers report failures to reproduce results from other labs and from their own work, including when new students attempt to continue prior experiments.
- The Brazilian Reproducibility Initiative, coordinated by Olavo Amaral and funded by Instituto Serrapilheira, highlighted challenges in obtaining exact methodological details from published papers.
- Martynhak frames reproducibility as necessary for experiments intended to support conclusions about humans, and points to improved procedural descriptions as one route toward better practice.
Transcript excerpt
0:00 Olá eu sou o professor Bruno Martinha aqui do departamento de fisiologia da UFPR e eu vou falar nesse curso de manipulação animal sobre delineamento experimental e um pouquinho sobre cálculo de amostra vamos lá essa aula está dividida em três partes Vamos começar com a primeira que se chama crise de reprodutibilidade eu gosto desse exemplo dessa tirinha aqui em que uma pessoa chega para outra e fala que umas gominhas causam acne E aí a pessoa manda os cientistas investigarem e a princípio o resultado diz que não não tem nenhuma relação entre as gominhas e acne mas parece que apenas uma certa cor é responsável por esse efeito vamos lá os cientistas
0:48 investigar diversas e diversas cores são testadas e nenhuma atinge aquele valor mágico da estatística que é o p menor do que 0,05 nenhuma delas exceto uma exceto a verde com a verde dá uma associação a Jelly Bean que veio de causa acne valor estatisticamente significativo E é isso que aparece no artigo publicado numa matéria da mídia e esse é um problema que acontece na ciência de maneira muito generalizada que é o Chery picken de dados a gente tem um viés de publicação muito intenso em que resultados positivos com associações positivas com coisas que testaram efeitos e encontrar
1:36 um efeitos são publicados nos periódicos que a gente cientista lê e vai se basear para fazer novos estudos tem muito fator envolvendo sorte e acaso nestes achados às vezes uma associação por mais que seja estatisticamente significativa ela aconteceu simplesmente por acaso por um azar estatístico pode ter um efeito de baixo poder estatístico junto com viésis que queria tamanhos de efeito não realistas Como assim um baixo poderes estatístico significa que você tem uma baixa chance de encontrar um efeito Digamos que um efeito de um
This is the start of the transcript. Save this video free for the whole transcript, searchable and time-coded.
Related notes
Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, Bruno Martynhak.