Evidence table
AI Scientist Claims vs Reality Tracker
A filterable, sourced table for separating AI scientist headlines from demonstrated capabilities and replication evidence, including Anthropic's 2026 science examples.
Key facts
Before you use this tool
Rows use primary-source baselines where possible: papers, lab announcements, and official documentation.
The table distinguishes workbenches, hypothesis engines, code loops, robotic agents, self-driving labs, and vendor science examples.
Replication status is descriptive, not a final judgment on scientific value. Vendor-announced results are labeled as such.
11 rows
| System | Claim | Reality | Autonomy | Status | Source |
|---|---|---|---|---|---|
| Claude ScienceResearch workbench | Claude as a workbench for scientific discovery | Official positioning describes a scientific workbench that can help organize research work; experimental autonomy depends on connected tools, permissions, and review gates.Not enough on its own to claim closed-loop autonomous experimentation. | Human-directed workbench | Official announcement | Anthropic |
| Venus elevation modelVendor science example | Claude built a digital elevation model of Venus from Magellan data | Anthropic describes an elevation model derived from public Magellan radar data and released under a Creative Commons license. The artifact is open, so third parties can inspect it.How much of the pipeline the model drove versus a human planetary scientist is not documented publicly. | Undisclosed division of labor | Vendor announcement | Anthropic |
| Protein binder validationVendor science example | A protein binder designed with Mythos 5.1 was experimentally validated | Anthropic announced an experimentally validated protein binder in work associated with Mythos 5.1 trusted access. No hit rate, baseline comparison, or run record has been published.Highest-stakes claim with the thinnest public record. Dual-use domain; methods are not covered on this site. | Undisclosed division of labor | Vendor announcement | Anthropic |
| Terminal-Bench-Science 0.1Vendor science example | Fable 5.1 scores 52.6% on a science terminal benchmark | Anthropic reports 52.6% for Fable 5.1, 29.0% for Opus 5, and 24.7% for Fable 5, alongside measured GPU kernel speedups. Roughly doubling in one generation is a real signal for operating scientific tooling.Version 0.1 benchmark, self-reported by the model vendor, and a proxy for tool operation rather than scientific judgment. | Benchmark task execution | Vendor-reported benchmark | Anthropic |
| Anthropic scientists programResearch workbench | Anthropic expands Claude access for scientists | A seat program giving human researchers access to Claude with Claude Science as the workbench, with biology and chemistry researchers described as limited to Opus-class models.An access program, not an autonomy claim. Seats and pricing are covered at clauderesearcher.com/scientists-program. | Human researcher access | Official announcement | Anthropic |
| Sakana AI ScientistCode experiment loop | Fully automated open-ended scientific discovery | Agentic workflow for idea generation, experiment execution in code, paper writing, and automated review in constrained machine-learning research settings.Autonomy is strongest in bounded code experiments, not physical lab actuation. | Bounded code loop | Paper and follow-up literature | arXiv |
| AI Scientist-v2Code experiment loop | Workshop-level automated scientific discovery via agentic tree search | Follow-up system expands search and review mechanics for machine-learning research tasks with automated paper-style outputs.Workshop-level output is not the same as independently confirmed scientific discovery. | Agentic tree search | Preprint | arXiv |
| Google AI co-scientistHypothesis engine | A multi-agent AI partner to accelerate research | Multi-agent system for generating, ranking, and refining hypotheses with scientist interaction and supporting evidence workflows.Designed as a co-scientist partner, not an unsupervised lab operator. | Human-in-the-loop co-scientist | Paper and official announcement | Google DeepMind |
| CoscientistRobotic chemistry agent | Autonomous chemical research with large language models | LLM-based agent planned and executed parts of chemical research workflows by using tools and interfacing with automated chemistry equipment.Lab-connected autonomy requires strict boundaries, expert oversight, and domain-specific safety controls. | Lab-connected agent | Nature paper | Nature |
| A-LabSelf-driving laboratory | Autonomous laboratory for accelerated synthesis of novel materials | Closed-loop materials discovery workflow combining computation, robotic synthesis, characterization, and decision-making in a lab setting.The system is specialized infrastructure, not a general-purpose AI scientist. | Closed-loop materials lab | Nature paper | Nature |
| Self-driving laboratory literatureSelf-driving lab | Self-driving laboratories for chemistry and materials science | Review literature describes closed-loop experimentation patterns, automation stacks, optimization loops, and practical limits.A field pattern, not one universal system or guarantee of autonomous discovery. | Closed-loop experimentation | Review literature | Nature Communications |
FAQ
Claims vs Reality Tracker Questions
What counts as reality in the claims tracker?
Reality means the demonstrated capability in the cited source: what the system actually did, what tools it controlled, and what evidence was preserved.
Why include systems that are not fully autonomous?
Because many AI scientist headlines blur workbenches, co-scientists, and closed-loop agents. The tracker makes that boundary visible.
How often should the tracker be refreshed?
It should be refreshed when a primary paper, official release, independent replication, or safety guidance changes the evidence base. It was last refreshed on 2026-09-19 with Anthropic's 2026 science examples.
Why are Anthropic's 2026 science results marked as vendor announcements?
Because that is their evidence tier. The Venus elevation model, the protein-binder validation, and the Terminal-Bench-Science score were published by the model vendor without a run record or independent replication. That does not make them false; it means they should be cited as announcements.