Science published an essay arguing that "in Western science, the purpose of research is to obtain and convey information, whereas in Indigenous science, information must come with a purpose."
Every clinical trial I've worked under says otherwise. Helsinki, Belmont, IRBs and the Asilomar pause on recombinant DNA all put purpose and harm review before research begins.
The real lesson for AI is that it skipped the purpose-first review that science already requires. Splitting science into "Western" and "Indigenous" camps distracts from that shared standard and from the shared remedy: binding review before release.
I submitted this as a Science eLetter on Nov 6. As of today, Science has published one eLetter on the essay, and it simply builds on the essay's framing. No alternative perspective or views. Given no response from Science, I withdrew mine eLetter and am posting it here openly.
15
1 patient out of 821 decides whether this stroke trial is significant.
ECASS III tested the clot-buster alteplase given 3 to 4.5 hours after a stroke. Recovery with little or no disability: 219 of 418 on alteplase vs 182 of 403 on placebo. p = 0.043.
Score just one alteplase patient as not recovered, and p becomes 0.051.
That doesn't make the trial wrong. It means its "significant" label is unstable, and one trial can't carry the conclusion alone.
1
17
CSE: unstable significance with discordant robustness. This is not durable evidence.
* unstable: changing 1 pt outcome could flip significant to NS.
* discordant robustness: the findings are close to a RR = 1, the point of no-effect; this is discordant from p < 0.05.
1
6
ECASS III (NEJM 2008): doi.org/10.1056/NEJMoa080465…
Documentation & how to compute CSE: fragilitymetrics.org/documen…
6
Huge study, tiny p-value: make these 4 checks before you believe the headline.
1️⃣ The p-value. With large sample sizes, almost any nonzero difference comes out "significant". A tiny p tells you the significance call isn't in doubt. It tells you nothing about how big the effect is, or if the effect is meaningful.
2️⃣ RQ, the risk quotient. For a yes/no outcome it measures how far the results table sits from no effect, on a 0 to 1 scale. It is resistant to sample size. Near is below 0.075; far is above 0.227. A tiny p with a near RQ is discordant, meaning that although the finding is significant, it likely is a very large study picking up a negligible effect.
3️⃣ Fragility at scale. In a mega-study, a low fragility quotient isn't really saying the call is shaky. It's reporting a small absolute effect. More data won't change it, because the quotient has already settled at a value set by the size of the effect.
4️⃣ The absolute numbers: absolute risk difference and number needed to treat. Relative risks like “42% higher” hide the baseline. Halving a risk from 2% to 1% is a 50% relative drop, and you still treat 100 people to help one.
Big N settles whether there's a difference. It doesn't tell you whether the difference matters.
“Highly significant, p < 0.01.” It sounds like the matter is settled.
But we need to consider NNR, the Number Needed to Reverse. It's the number of identical repeat trials you'd expect to run to get one reversal of the significance call. Think number needed to treat, but for replication. It comes straight from the p-value.
At p = 0.05, NNR = 2.0. Rerun the identical trial and you'd expect one reversal for every two tries. A coin flip.
At p = 0.01, NNR = 3.7 (about 1 out of every identical 4 trials expected to be nonsignificant).
At p = 0.001, NNR = 10.9. (about 1 out of 11 identical trials expected to be nonsignificant).
At p = 0.0001, NNR = 37.
The p-value after all is just a probability, although too often it is treated as an absolute.
So p = 0.01 means roughly one identical repeat in four would be expected to come back not significant. That's not settled. It's a good start.
One caution: NNR is the p-value translated into counted trials, not new information. It says nothing about how big the effect is, or how few patients' outcomes would need to change to flip the result. For those you still need robustness and fragility.
16
Two trials. Both with p<0.001. Yet they are NOT equally convincing. Consider this scenario:
Trial A: 80 patients per arm with event rate of 50% in one arm and 22.5% in the other = {40, 40, 18, 62}
Trial B: 40,000 patients with event rate of 9% vs 10% = {1800, 18200, 2000, 18000}
1
1
101






