Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases withspecific.com ↗ HN ↗

Specific Labs’ Real-SWE benchmarks coding agents on private production codebases, not public repos. The best model, Fable 5.1 on Claude Code, resolves just 38.8% of tasks. Most failures are missed requirements, not broken code, and spending more doesn’t buy better results.