A new benchmarking platform called TTS Arena has emerged to evaluate text-to-speech (TTS) models based on real-world user feedback. Unlike traditional metrics that focus on technical aspects like word error rate or latency, TTS Arena crowdsources ratings from listeners to rank models by naturalness, clarity, and overall quality.
Developed by researchers and open-source contributors, the platform lets users submit any text and hear it spoken by multiple TTS models side by side. Listeners then vote for the most natural-sounding version. This approach aims to capture subjective human preferences that automated metrics often miss.
Early results show that modern neural TTS systems, especially those based on large language model architectures, outperform older concatenative and parametric models. However, the gap between top commercial offerings and leading open-source models is narrowing.
TTS Arena covers a diverse set of models, including those from ElevenLabs, OpenAI, Microsoft, Google, and open-source projects like Bark and Tortoise TTS. The platform updates rankings periodically as new models are added or existing ones improve.
The creators hope TTS Arena will guide developers and businesses in choosing the right TTS engine for applications like virtual assistants, audiobooks, and accessibility tools, as well as drive further improvements in the field.