The 2026 replication project: how much of social science holds up. · published 11 September 2026
The replication rate is the headline everyone fights over, but the more unsettling number is what happened to the findings that did survive — at the median their effect size dropped from 0.25 to 0.10, roughly an 82 per cent cut in shared variance, meaning the real story is not fake effects but real effects that are a fraction of the size they were sold as.
These are the working notes, unedited. Everything the video
states as fact comes from here, and nothing else may be stated as fact.
The sections on limits and on what the finding does not show are
part of the brief, not an afterthought — see
the method.
The study: “Investigating the replicability of the social and behavioural sciences”, published in Nature in April 2026 (volume 652). It ran under SCORE, a programme funded by the US Defense Advanced Research Projects Agency whose stated aim is to work out how confidently a published claim can be expected to hold up.
What they did. They took 274 claims of positive results from 164 quantitative papers, published between 2009 and 2018 across 54 journals in the social and behavioural sciences, and tried to reproduce them. The replications were not cheap shots: they were powered to a median of 99.6 per cent to detect the original effect if it was there, they used the original materials where those were relevant and available, and the protocols were peer reviewed in advance rather than written after the result was known.
The headline result. 151 of the 274 claims came out statistically significant in the same direction as the original — 55.1 per cent, with a confidence interval of 49.2 to 60.9 per cent. Counted by paper rather than by claim, and weighted for papers carrying more than one, it was 80.8 of 164, or 49.3 per cent. Roughly half.
The part that gets left out of the headline. Replicating at all is a low bar, and the surviving findings came back much weaker than they were first reported. For the claims measured as a correlation, the median effect was 0.25 in the original studies and 0.10 in the replications — a reduction of about 82 per cent in shared variance. So the question is not only whether an effect is there. It is that the effects which are there are typically a fraction of the size that was published.
Variation by field was modest: replication rates ran from 42.5 to 63.1 per cent across disciplines, and the paper notes that some of those individual estimates carry high uncertainty. It does not support ranking fields against each other.
Limits
The sample is published positive results from 2009 to 2018 — before preregistration and registered reports became common in many of these fields, so it is a measurement of a particular decade’s literature rather than of work being published now. A failed replication is also not proof the original was wrong: it can mean the effect is smaller than either study could resolve, or that it depends on conditions nobody has identified yet. And 55 per cent is not “half of everything ever published” — it is the rate for claims selected to be testable and retested one time each.
What this does not show
It is not evidence that science is broken, or that research in general cannot be trusted, and it must not be used to wave away findings someone finds inconvenient. The exact opposite is the honest reading: this is the field measuring itself, at scale, in public, and publishing a number that does not flatter it. No other kind of institution does that. The finding that should unsettle a viewer is the shrinking effect size, not the idea that scientists are unreliable people.