NPHardEval Leaderboard: New Benchmark Tests LLM Reasoning with Complexity Classes

A new leaderboard, NPHardEval, aims to evaluate the reasoning capabilities of large language models (LLMs) by focusing on complexity classes. Unlike standard benchmarks, NPHardEval dynamically updates its test set to prevent data contamination and accurately measure generalization. The benchmark covers problems from various computational complexity classes, including P, NP, and NP-hard, providing a nuanced view of model strengths and weaknesses. Early results show that while some models perform well on simpler tasks, they struggle with NP-hard problems, highlighting gaps in reasoning depth. The dynamic nature of the leaderboard ensures that models are continually tested on novel problems, making it a robust tool for tracking progress in LLM reasoning.