Google has unveiled a new feature in its PaliGemma 2 AI model family: the ability to "identify" emotions. This feature, announced at a recent event, allows the model to analyze images and generate captions that describe actions, emotions, and the overall scene in the photo. While emotion detection may seem like a breakthrough, experts are expressing concerns about the potential issues it may cause.
PaliGemma 2 is based on Google’s Gemma open model set and works by analyzing images, offering more than simple object identification. It generates detailed captions that include not only objects but also emotional and contextual aspects of the scene. However, emotion recognition is not immediately available; the model needs to be fine-tuned for this specific purpose.
Experts, however, are concerned about the implications of emotion-detecting AI. Sandra Wachter, a professor of data ethics and AI, argued that interpreting emotions through AI could be problematic. She compared it to asking a Magic 8 Ball for advice, suggesting that such assumptions about reading emotions could be misleading.
The technology behind emotion detection relies on the work of psychologist Paul Ekman, who identified six basic human emotions: anger, surprise, disgust, enjoyment, fear, and sadness. However, further research has challenged this idea, showing that emotional expression can vary greatly across different cultures and backgrounds. Mike Cook, an AI research fellow at Queen Mary University, explained that while detecting emotions through facial expressions may be possible in certain cases, it is not a fully reliable or universal solution.
One of the key concerns about emotion recognition systems is their reliability and the biases they may carry. A 2020 study by MIT found that face-analyzing models could develop biases, such as favoring certain facial expressions like smiling. Furthermore, recent studies have shown that emotion-detection models tend to associate more negative emotions with Black individuals compared to white individuals.
Google claims that it has conducted extensive testing on PaliGemma 2 to evaluate potential biases, finding low levels of toxicity and profanity compared to industry standards. However, it has not disclosed the full set of benchmarks or tests used in the evaluation. The only benchmark Google has revealed is FairFace, a dataset of headshots from a few racial groups. Some researchers have criticized FairFace for not being representative enough, suggesting that it may not adequately address biases in emotion detection.
Emotion detection has raised regulatory concerns, especially in high-risk areas such as law enforcement and hiring practices. The EU's AI Act, for example, prohibits the use of emotion detection in schools and workplaces, though law enforcement is exempt. Heidy Khlaaf, chief AI scientist at the AI Now Institute, cautioned that relying on AI for emotional interpretation could lead to harmful consequences, particularly for marginalized groups.
The open availability of PaliGemma 2, including through platforms like Hugging Face, increases the risk of misuse. Khlaaf warned that it could lead to discrimination, especially in sensitive areas such as law enforcement, hiring, or border control. Despite Google’s assurances that it has tested the model for safety and ethics, experts remain wary of the potential for abuse.
Wachter emphasized that responsible innovation requires considering the consequences of a product from its inception. She raised concerns about a future where emotional analysis could affect important life decisions, such as job hiring, loan approvals, or university admissions, based on a potentially flawed understanding of emotions.
In conclusion, while Google’s new emotion recognition feature in its AI models shows technical advancement, it also brings with it significant ethical and practical concerns that need careful consideration.
