SciPaths: Forecasting Pathways to Scientific Discovery
Researchers have introduced SciPaths, a new benchmark designed to forecast the pathways to scientific discovery by identifying enabling contributions and their dependencies on prior work. Unlike existing benchmarks that focus on simpler tasks like citation prediction, SciPaths requires models to reason backward from a target contribution to the necessary building blocks. Evaluations of current frontier and open-weight language models show that even the best models struggle with this complex reasoning, achieving only a 0.189 F1 score, indicating that accurately recovering methodological dependencies remains a significant challenge. AI
IMPACT This benchmark pushes AI capabilities towards complex scientific reasoning and dependency tracking, potentially accelerating AI-assisted research.
