OpenAI launches IndQA: New benchmark to measure AI understanding of Indian culture and languages

New Delhi:

OpenAI recently introduced a new initiative called IndQA. This project includes a unique collection of questions designed to evaluate how effectively artificial intelligence can grasp and think about various aspects of Indian culture, languages, and daily life.

IndQA is a collection of 2,278 questions available in 12 different languages, focusing on 10 areas of culture. It was created with input from 261 experts in India. IndQA stands out because the questions are created in the languages that people in India actually use, instead of being translated from another language. This approach makes the questions more relatable and true to how people in India think and talk with one another.

Srinivas Narayanan, CTO, B2B Applications, OpenAI, highlighted the company's vision, telling reporters:

"At OpenAI, we believe AI should be useful for all of humanity. That means AI must understand local cultures, languages, histories and contexts—not just the Western world. India is a country of immense diversity, with many languages, traditions, and cultural nuances. For AI to be truly valuable here, it must understand that richness".

Captures India's cultural and historical context

Announcing the launch, Narayanan emphasised that IndQA was created with a curated dataset that accurately captures India's cultural and historical context. "This dataset helps our models understand Indian nuances more deeply," he explained. "The experts also provide evaluation rubrics, so we can measure how well the AI performs on culturally grounded questions. Our goal is to take this as a playbook and use it in other countries too".

Narayanan added that IndQA also underscores OpenAI's growing commitment to the Indian ecosystem, where local developers, educators, and creators are shaping AI adoption globally.

IndQA covers a broad range of culturally relevant topics: architecture and design, arts and culture, everyday life, food and cuisine, history, law and ethics, literature and linguistics, media and entertainment, religion and spirituality, and sports and recreation. The items are written natively in twelve languages: Bengali, English, Hindi, Hinglish, Kannada, Malayalam, Marathi, Odia, Tamil, Telugu, Gujarati, and Punjabi.

Each piece of information is very detailed. It includes a prompt or question written in an Indian language, along with an English translation for clarity. There are also guidelines for evaluating responses and an example of a perfect answer that meets expert standards.

ALSO READ: BSNL's indirect price hike: Validity slashed on popular low-cost prepaid plans