OpenAI is funding academic research aimed at developing algorithms capable of predicting human moral judgments. This initiative, revealed in an IRS filing, includes a grant awarded to researchers at Duke University for a project titled “Research AI Morality.” According to a press release, this grant is part of a broader $1 million, three-year funding program dedicated to creating "moral AI." The funding is set to conclude in 2025.
Leading this research is Walter Sinnott-Armstrong, a professor specializing in practical ethics at Duke, alongside co-investigator Jana Borg. Both researchers have previously explored how AI could function as a "moral GPS," aiding humans in making better ethical decisions. Their work includes studies on morally aligned algorithms, such as one designed to allocate kidney donations fairly, as well as examining public preferences for AI's role in moral decision-making.
The stated aim of the OpenAI-backed project is to train AI systems to predict human moral judgments in complex scenarios involving conflicts in fields such as medicine, law, and business. However, the challenge lies in the inherently subjective nature of morality. Philosophers have debated ethical frameworks for centuries, and no universal consensus has emerged. This complexity raises questions about whether AI can meaningfully engage with moral reasoning at all.
Previous efforts highlight the difficulties in this area. In 2021, the Allen Institute for AI introduced a tool called Ask Delphi to provide ethical recommendations. While Delphi could handle simple moral dilemmas, it struggled with nuance, often giving contradictory or inappropriate answers depending on how questions were phrased.
This inconsistency stems from the limitations of modern AI systems. Machine learning models operate by recognizing statistical patterns in data gathered from the internet. While this enables them to predict language patterns, they lack the ability to comprehend ethical concepts or the emotions and reasoning that shape human morality. Consequently, AI models often reflect biases prevalent in their training data, which is disproportionately influenced by Western, educated, and industrialized perspectives.
Such biases can exclude diverse viewpoints, leading to moral judgments that fail to represent all individuals. For instance, Delphi once deemed being straight as more “morally acceptable” than being gay, highlighting the ethical pitfalls of AI.
Developing an algorithm to predict human moral judgments would require accounting for these biases, as well as grappling with the philosophical debate over moral frameworks. While some models lean toward utilitarianism, prioritizing the greatest good for the greatest number, others align with absolutist approaches like Kantian ethics. Deciding which framework is "better" depends largely on individual perspectives, further complicating the task.
OpenAI’s research initiative reflects an ambitious attempt to bridge the gap between AI and human ethics. Yet, the question remains: can morality, with all its complexities and subjectivity, ever be encoded into an algorithm? Whether such a goal is achievable will only become clearer as this research progresses.
