Google Gemini is the first global test to measure the compatibility of artificial intelligence models with the Emirati identity

The Office of Artificial Intelligence, Digital Economy and Remote Work Applications in the UAE government announced that the “Google Gemini” model won in the “Google Gemini” index.AI “In the Field”, which is the first global test aimed at measuring the compatibility of artificial intelligence models with Emirati culture, and the extent of their familiarity with local dialects, customs and national values.

The index evaluates large linguistic models through a challenge that focuses on elements of cultural intelligence in the UAE.

“Gemini” won after a series of evaluation tests that included more than 400 questions and 5,200 responses from 11 large artificial intelligence linguistic models. A specialized Emirati committee, which includes experts from various concerned authorities, undertook the process of evaluating the results to determine the models most trained in Emirati culture.

The list of the top five language models after Gemini, which showed outstanding performance, included ChatGPT-OpenAIGPT4O, ChatGPT-OpenAIO1, Coher, and Grok.

Omar Sultan Al Olama, Minister of State for Artificial Intelligence, Digital Economy and Remote Work Applications, stressed that the UAE government attaches great importance to consolidating the national identity as the foundation and pillar of society, pointing to the importance of strengthening the Emirati national identity in modern technologies in light of the accelerating digital transformation, to ensure that development is consistent with the country’s authentic values, and to train technological systems and artificial intelligence on the components of the national identity to preserve the legacy for future generations.

He said that the index focuses on adopting the concept of national identity in technology, which represents the roots of man and the mirror of his history that preserves the values ​​of society, and reflects the UAE’s keenness to enhance awareness of national identity and its importance to empower the Emirati person, preserve his culture, and build a future in which technological progress is enjoyed with an awareness that ensures its sustainability and reflects the spirit of the nation in the UAE society.

The models were evaluated to measure their ability to showcase the spirit and culture of the Emirates and preserve the language, heritage and values, through a direct comparison between the leading international models, to assess the extent of their understanding and connection to the Emirati identity in seven different dimensions that included the historical context, creative and poetic expression, the Emirati language and dialect, cultural symbols and connotations, awareness of the social context, behavioral differences, social and religious topics, and Emirati values ​​and ethics.

The index followed the Human AI Response Team methodology, which is a process of deliberately and directly challenging the model to uncover any gaps, risks, or cultural and behavioral errors before using it.

The models were tested using a custom platform that was designed taking into account bias and activation of different mechanisms of action when abnormal behavior was detected, and the identity of the linguistic models was completely anonymized from the evaluation committee during the methodology.

It is noteworthy that less than 5% of the total Arabic content in the world is available in digital form, and 48% of the new generation “Z” rely on artificial intelligence as a primary source of information, which requires training linguistic models with accurate data, especially related to Emirati culture, to preserve the national heritage for new and future generations.