Benchmarking Coding Agents on Databricks' Multi-Million Line Codebase databricks.com ↗

Databricks built its own coding-agent benchmark from real pull requests against its multi-million-line codebase, graded by real test suites. No model won outright: Opus 4.8 scored 87% at $1.94 a task, GLM 5.2 tied it at $1.28, Sonnet 5 cost more despite a lower per-token price.