Autopoiesis Science

Autopoiesis Science is an evaluation framework designed to rigorously assess the scientific reasoning capabilities of advanced language models, with an initial focus on graduate-level physics. Built around the GPQA Diamond benchmark, it tests whether model answers are not only correct, but also supported by coherent, step-by-step reasoning grounded in appropriate physical principles. The primary purpose of Autopoiesis Science is to distinguish genuine conceptual understanding from pattern matching or memorization in complex scientific problem solving.

Autopoiesis Science automatically scores model responses against expert-authored solutions, evaluating both final answers and intermediate reasoning steps for accuracy and completeness. It verifies the logical structure of solutions, checking consistency with problem constraints and proper use of relevant physics concepts and laws. The system detects and classifies reasoning errors, unjustified leaps, and misuse or omission of key principles, while providing fine-grained analysis of where model reasoning diverges from expert standards across the full solution path. These capabilities enable precise benchmarking, targeted debugging, and systematic comparison of different models or training interventions.