LM Evaluation Harness

AI model evaluations are becoming a costly bottleneck, surpassing training expenses

AI model evaluations are becoming prohibitively expensive, with recent benchmarks costing tens of thousands of dollars and consuming thousands of GPU hours. This high cost is particularly pronounced for agent-based eval…