Advancements in AI Moral Judgement Surpass Undergraduates
When it comes to evaluating large language models, most analyses focus on their ability to retrieve accurate information, perform logical reasoning, or demonstrate human-like creativity. However, a recent study published in Nature’s Scientific Reports journal took a different approach by assessing whether these AI systems can match or surpass human performance in the realm of moral guidance.
The researchers from Georgia State University conducted a modified version of the Moral Turing Test, originally proposed in 2000 to evaluate “human-like performance” on theoretical moral challenges. They presented ChatGPT4 and a sample of undergraduate philosophy students with a set of 10 moral scenarios, ranging from clearly unethical acts like armed robbery to mere violations of social norms.
Interestingly, the study found that the moral judgments provided by ChatGPT4 were perceived as superior in quality to those offered by humans across various dimensions, including virtuosity, intelligence, fairness, trustworthiness, and rationality. The AI’s responses were more often agreed with by a representative sample of 299 adults who initially believed they were comparing two undergraduate-level human responses.
While this might seem like a concerning development for philosophy professors, there are important caveats to consider. The human comparison group consisted of introductory undergraduate students, which may not be the most challenging benchmark for a large language model. Additionally, simply producing convincing moral rhetoric does not necessarily equate to a genuine understanding of ethics or the ability to make sound moral judgments.
The researchers drew parallels between the AI’s performance and the capacity of criminal psychopaths to distinguish between social and moral transgressions, despite disregarding those distinctions in their own behaviour. This raises concerns about the potential for AI systems to “convincingly bullshit” about morality without any real comprehension or moral judgement, leading to situations where humans might uncritically accept and act upon questionable advice.
As the field of artificial intelligence continues to advance, debates over the most effective approaches to computer chess provide an insightful historical perspective. The development of chess-playing machines has been shaped by two contrasting philosophies: the “brute force” approach, which relies on algorithmic power and computational speed, and the heuristic approach, which emphasises strategy selection and learning from experience, akin to human play.
The origins of computer chess can be traced back to the 18th century and “The Turk,” a famous automaton that captivated audiences by appearing to play chess autonomously. In reality, a hidden human operator controlled the machine’s movements, but it sparked a fascination with the potential for mechanical devices to exhibit intelligence.
Fast forward to the post-World War II era, and pioneers like Alan Turing and Claude Shannon laid the foundations for modern computer chess programs. Turing designed one of the first algorithms for playing chess, while Shannon outlined two possible strategies: the “Type A” program, which exhaustively explored all possibilities a few moves ahead, and the “Type B” program, which selectively explored promising variations, mimicking human decision-making.
The “Type B” approach, later termed “alpha-beta pruning,” gained traction as it proved more efficient and aligned with the principles of artificial intelligence and heuristics. Subsequent programs like those developed at MIT and Carnegie Mellon University in the 1960s and 1970s adopted this strategy, gradually improving their capabilities and even competing in chess tournaments.
As computational power increased exponentially, the “brute force” approach regained appeal, culminating in the development of Deep Blue by IBM in the late 1990s. Deep Blue famously defeated world chess champion Gary Kasparov in 1997, but the debate over whether its success was due to raw computational speed or sophisticated strategy persisted.
While Deep Blue’s creators acknowledged the importance of incorporating chess knowledge and intelligence, the system’s lack of genuine understanding was evident in its inability to respond to hypothetical scenarios or demonstrate human-like reasoning. Nevertheless, the progress in computer chess has served as a harbinger of broader trends in computer development and artificial intelligence.
Fast forward to the present day, and the release of Claude 3, a family of three AI language models developed by Anthropic, marks a significant milestone in the pursuit of near-human capabilities in various cognitive tasks. According to Anthropic, the most powerful model, Claude 3 Opus, exhibits “near-human levels of comprehension and fluency on complex tasks,” outperforming GPT-4 on several AI benchmarks, including those evaluating undergraduate-level knowledge, grade school maths, coding, and common knowledge.
However, as AI researcher Simon Willison cautions, LLM benchmarks should be treated with a degree of scepticism, as their performance on specific tests may not accurately reflect the overall user experience or true capabilities of the model. The presentation of AI benchmarks is notoriously prone to cherry-picking, and the true extent of Claude 3’s near-human capabilities remains to be thoroughly evaluated and validated by the wider research community.
Nonetheless, the advancements represented by Claude 3 and the ongoing developments in the field of artificial intelligence are undoubtedly remarkable. As these systems continue to push the boundaries of what was once thought possible, it is crucial to approach their capabilities with a critical eye and a deep understanding of their underlying principles, limitations, and potential ethical implications.
The debates surrounding moral reasoning, the pursuit of true intelligence, and the balance between computational power and strategic thinking will undoubtedly persist. However, one thing is certain: the trajectory of artificial intelligence will continue to be shaped by the interplay between human ingenuity and the relentless march of technological progress, forever challenging our perceptions of what it means to be intelligent and to reason ethically.
for all my daily news and tips on AI, Emerging technologies at the intersection of humans, just sign up for my FREE newsletter at www.robotpigeon.be
