A new leaderboard called QIMMA (قِمّة) has been launched to evaluate Arabic large language models (LLMs) with a focus on quality. Unlike existing benchmarks that often emphasize breadth or speed, QIMMA aims to assess models on linguistic accuracy, cultural relevance, and task-specific performance in Arabic.
The initiative, whose name means "summit" in Arabic, brings together experts in NLP and Arabic linguistics to design rigorous evaluation criteria. Early results have shown significant variation among popular models, highlighting the need for specialized Arabic AI evaluation. QIMMA is expected to help researchers and developers identify the strongest models for Arabic-language applications.