Sarvam, an Indian AI laboratory, recently introduced a new series of large language models. The company is banking on these smaller, efficient open-source AI models to compete with the more costly systems from larger U.S. and Chinese competitors.
This launch, revealed at the India AI Impact Summit in New Delhi, supports India’s objective to lessen dependence on international AI platforms and develop models suited for local languages and specific applications.
The new models from Sarvam feature 30-billion and 105-billion parameter versions, alongside a text-to-speech model, a speech-to-text model, and a vision model for document analysis. This represents a significant advancement from their earlier 2-billion-parameter Sarvam 1 model, launched in October 2024.
Both the 30-billion and 105-billion parameter models utilize a mixture-of-experts architecture. This design activates only a portion of their parameters at any given time, leading to substantial reductions in computing expenses. The 30B model is designed for real-time conversations with a 32,000-token context window, while the larger model provides a 128,000-token window for intricate, multi-step reasoning.
These new AI models were developed from the ground up, not merely fine-tuned from existing open-source systems. The 30B model underwent pre-training on approximately 16 trillion text tokens, and the 105B model was trained on trillions of tokens across various Indian languages.
The startup stated that these models are built to support real-time applications, such as voice assistants and chat systems in Indian languages.
The models were trained utilizing computing resources from India’s government-supported IndiaAI Mission, with infrastructure provided by data center operator Yotta and technical assistance from Nvidia.
Executives at Sarvam indicated that the company intends to scale its models cautiously, prioritizing real-world applications over simply increasing their size.
Pratyush Kumar, Sarvam’s co-founder, commented at the launch, “Scaling will be approached thoughtfully, not indiscriminately. The focus is on identifying and developing for tasks that truly matter at scale.”
Sarvam announced plans to open-source both the 30B and 105B models. However, it was not specified if the training data or the complete training code would also be publicly released.
The company also detailed intentions to develop specialized AI systems, such as coding-focused models and enterprise tools under “Sarvam for Work,” and a conversational AI agent platform named “Samvaad.”
Established in 2023, Sarvam has secured over $50 million in funding, with investors including Lightspeed Venture Partners, Khosla Ventures and Peak XV Partners (previously Sequoia Capital India).
