Researching and Improving AI Agents for Biology
NeuroscienceBench: Benchmarking AI agents on the ability to analyze and interpret neuroscience research data
Preprint · NeuroscienceBench91 evals · 32.2% top scoreRead paper
Neuroscience research increasingly depends on large datasets and novel experimental methods that demand specialized expertise. Here we introduce NeuroscienceBench, a verifiable benchmark that tests whether an AI agent can reach correct scientific conclusions from diverse neuroscience data. The benchmark comprises 91 evaluations, each focused on a research task testing a distinct skill. Agents receive the same data a wet- or dry-lab neuroscientist would and return a structured answer that is graded deterministically. Evaluations are classified along three axes (functional system, process class, biological level) and draw on 43 peer-reviewed studies spanning next-generation sequencing, electrophysiology, optical physiology, electroencephalography, brain-computer interface decoding, cell and tissue imaging, behavioral scoring, and clinical biomarkers. Across 34 model-harness configurations (21 models, 4 harnesses, 9,212 graded attempts) the mean pass rate was 21.0%. The strongest configuration, Claude Opus 5.5 with the Pi harness, passed 32.2% (95% CI 23.5–41.0). The type of data analyzed had the largest effect on performance, with mean pass rates ranging from 14% on less common omics readouts and 18% on clinical and biofluid data to about 25% on behavior and on EEG and sleep scoring. Accuracy rose with cost, but inexpensive configurations came within a few points. Current AI models can therefore help with some neuroscience analyses but cannot be relied on without expert judgment.