Sarvam AI, an Indian artificial intelligence startup, has announced a major breakthrough in speech recognition for Indian languages. The company released open-source benchmarks designed to evaluate AI models on code-mixed speech—where speakers blend English with native languages like Tamil, Hindi, or Telugu.
Unlike traditional metrics such as Word Error Rate (WER), Sarvam AI's approach uses large language model (LLM)-based evaluation. A key innovation is "Intent Scoring," which assesses whether an AI correctly understands the user's intent rather than just transcribing words verbatim.
This development addresses a long-standing limitation of global AI models like GPT-4 and Gemini, which often struggle with India's unique linguistic diversity. By focusing on code-mixed speech, Sarvam AI aims to create a "gold standard" for Indian speech AI, potentially accelerating the country's voice-first economy and strengthening sovereign AI capabilities.
The benchmarks are available for developers and researchers, marking a significant step toward more inclusive and context-aware AI systems tailored to India's multilingual population.