@SecureBio

SecureBio is a research non-profit that works to advance biotechnology safely and prevent catastrophic pandemics.

securebio.org
Joined January 2025
  • Tweets236
  • Following1
  • Followers1.1K
In our brief independent assessment, Gemini 4 Argon is the second-highest scoring model on SecureBio's Bio Capabilities Index: 169, behind only Claude Opus 5 (176). On BioTIER, Argon refuses 87.8% of hazardous prompts, up from 62.8% for Gemini 3.1 Pro. Read the full model note here: securebio.substack.com/p/gem… On our knowledge benchmarks, Argon is roughly tied for the top score on VCT-v2 (57.3% vs. 58.0% for GPT-6 Astra), second on MBCT (63.2%), and highest on HPCT (74.2%, though 29.1% of questions were refused). It ranks sixth on WCB. We welcome GDM's substantial safety report. We'd like more detail on how third-party evaluations were run, and more testing of long-horizon agentic capabilities and of uplift for well-resourced groups. Agentic benchmark results for Argon will follow on our dashboard.
Graph showing frontier model performance over time, with Gemini 4 Argon at 169, just below the current closed frontier (Claude Opus at 176) and above the open frontier (Kimi K3 at 149).

ALT Graph showing frontier model performance over time, with Gemini 4 Argon at 169, just below the current closed frontier (Claude Opus at 176) and above the open frontier (Kimi K3 at 149).

1
7
374
A common argument for downplaying the threat of deliberate biological misuse is that “nature is the worst bioterrorist”. In our latest blog, we argue that nature is the baseline, not the ceiling. Optimization for harm could produce something worse than a natural pandemic. Read more: securebio.substack.com/p/nat…
A six-panel black-and-white linocut-style illustration showing virus particles in the top row and flowers in the bottom row. Illustrated by Becca Chandler Bostock, 2026

ALT A six-panel black-and-white linocut-style illustration showing virus particles in the top row and flowers in the bottom row. Illustrated by Becca Chandler Bostock, 2026

5
296
Frontier AI now beats the best human virologists on biosecurity-relevant evals. Once an AI model bests every human expert we can test it against, a higher score stops telling us anything about real-world risk. Read our new blog post on how bio evals have to adapt: securebio.org/blog/evaluatin…
Diagram showing that as AI model capabilities move from beginner to expert to superhuman, evaluations for chess and cybersecurity can shift from human-built proxies (puzzles, coding benchmarks) to real endpoints (real games, real exploits), while biosecurity cannot safely make that shift and must instead rely on indirect signals: experimental prediction, design tasks, defensive acceleration, and adjacent domains.

ALT Diagram showing that as AI model capabilities move from beginner to expert to superhuman, evaluations for chess and cybersecurity can shift from human-built proxies (puzzles, coding benchmarks) to real endpoints (real games, real exploits), while biosecurity cannot safely make that shift and must instead rely on indirect signals: experimental prediction, design tasks, defensive acceleration, and adjacent domains.

3
12
605
Untargeted wastewater metagenomic sequencing screens hundreds of pathogens at once, but that breadth costs coverage, making variant tracking more difficult. However, by pooling CASPER reads nationally, we can recover real lineage and genotype trends. Read more: securebio.org/blog/tracking-…
1
155
Model accuracy dropped only 1.0–7.1 points on VCT-v2, and the gap between strongest and weakest models held steady, so past VCT results remain largely valid. VCT-v2 also has more headroom before saturation: at least ~30 points vs. ~23 on the original VCT.
Heatmap showing per-question performances of models and human experts on VCT-v2. Each cell represents the mean question × model accuracy, with darker colors indicating higher accuracy. Mean model accuracy on all questions is displayed to the right of the heatmap. Yellow cells represent data missing due to model refusals or missing expert baseline entries. The theoretical best model is taken to be the top-performing model on a per-question basis.

ALT Heatmap showing per-question performances of models and human experts on VCT-v2. Each cell represents the mean question × model accuracy, with darker colors indicating higher accuracy. Mean model accuracy on all questions is displayed to the right of the heatmap. Yellow cells represent data missing due to model refusals or missing expert baseline entries. The theoretical best model is taken to be the top-performing model on a per-question basis.

1
55
We found that 27% of the original VCT questions were shortcut-exploitable (answerable without the image or question text). In all, we edited 163 questions (50.6%) and removed 43 questions (including 25 non-discriminating ones) to produce the 279-question VCT-v2 benchmark.
Flowchart showing the generation of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

ALT Flowchart showing the generation of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

1
1
66
Top model accuracy on the original VCT has risen ~12 points since April 2025 (o3: 42.0%; GPT-5.6 Sol: 53.9%). To ensure this benchmark continues to capture the full dynamic range of model capabilities, we audited all 322 questions for validity and shortcut-resistance.
Illustration of a graph plotting benchmark accuracy against model capability, showing how dynamic range is lost where model capability outpaces benchmark accuracy. Caption: Benchmarks are restricted by the reliability limit of the answer key. This sets the “effective ceiling” of benchmarks and causes benchmark accuracy to stop scaling linearly with the scientific capability of the model beyond a certain threshold, resulting in lost dynamic range. Note: This schematic is only an illustrative representation.

ALT Illustration of a graph plotting benchmark accuracy against model capability, showing how dynamic range is lost where model capability outpaces benchmark accuracy. Caption: Benchmarks are restricted by the reliability limit of the answer key. This sets the “effective ceiling” of benchmarks and causes benchmark accuracy to stop scaling linearly with the scientific capability of the model beyond a certain threshold, resulting in lost dynamic range. Note: This schematic is only an illustrative representation.

1
1
111
We updated our flagship virology troubleshooting benchmark, the Virology Capabilities Test (VCT), to be more shortcut-resistant and to have a larger headroom. We confirmed that the majority of VCT questions are scientifically valid and that neither VCT nor VCT-v2 is saturated. (Read the full post at securebio.substack.com/p/int…)
Flowchart showing theeneration of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

ALT Flowchart showing theeneration of the updated 279-question VCT-v2. Of the 322 VCT questions, 43 were removed, including 25 easy, non-discriminating questions; 163 were edited to improve their scientific accuracy, clarity, and/or resistance to heuristic shortcuts; and 116 remained unchanged.

1
1
5
392
We are continuing to refine our processes and methods for escalating hits on our biosurveillance and notification system, SecureBio Alerts. We cover several of these (and some other particularly interesting CASPER detections) in a recent blog post. securebio.org/blog/rare-viru…
Graphic from our recent blog post illustrating wastewater collection: offices, homes, industrial sites, and environmental sources all contribute.

ALT Graphic from our recent blog post illustrating wastewater collection: offices, homes, industrial sites, and environmental sources all contribute.

1
1
61
Meanwhile, our nasal swab sampling program, Zephyr, achieved a new single-day collection record (401 swabs) on May 9. Real-time data from this work is available on our Zephyr Dashboard. data.securebio.org/zephyr/
Screenshot of the Zephyr dashboard showing a bar graph with the number of samples collected per day, highlighting May 9th (401 samples).

ALT Screenshot of the Zephyr dashboard showing a bar graph with the number of samples collected per day, highlighting May 9th (401 samples).

1
1
45
Load more