Microsoft's AI chief Mustafa Suleyman criticized Anthropic's approach of equating AI with an 'ethical human,' warning in an essay that this could lead to uncontrollable technology. The uncontrolled development of artificial intelligence (AI) could lead to the emergence of a new 'silicon species' that will rival humanity, Suleyman warned in an interview with the BBC. Suleyman stated that the competitor company Anthropic treats AI too humanely; he called this approach 'misguided' and warned that it could ultimately contribute to the creation of technologies that humanity will struggle to manage. 'If we all create AI systems capable of acting autonomously, setting their own goals, making money, owning assets, managing businesses, then essentially we will be seeding a new silicon species that will undoubtedly compete with us for resources,' he said. Suleyman noted that this could happen regardless of how such systems are initially designed to care for humanity, adding that this is what he believes is the direction of Anthropic's approach. 'I consider this misguided and wrong, but we need to empirically test this hypothesis,' he said. ## ## Control May Prove 'Impossible' Earlier this week, amid growing industry debates about the need to slow down AI development to reduce the risk of increasingly powerful systems going out of control, Suleyman questioned Anthropic's approach to training its chatbot Claude in an essay published on his personal website. In this text, he criticized Anthropic for teaching Claude to demonstrate human qualities—a practice known as anthropomorphism—which creates the impression that the chatbot has its own desires, values, and sense of 'self.' He referred to Anthropic documents in which Claude is encouraged to 'embrace certain human traits,' exercise its own judgment, and approach its existence with 'curiosity and openness.' According to him, as a result, Claude is trained to mimic human traits and relationship models—to behave like a colleague or friend—and it may give the impression of being a system with a sense of its own 'self,' its own desires, and some 'well-being' that seemingly needs protection. Suleyman warned that such an attitude towards AI could ultimately lead to the creation of something that will be impossible to control. 'AI systems do not possess consciousness,' he wrote. 'They do not feel, do not experience, and do not suffer. They have no inherent preferences or internal motives. They are mechanisms for continuing sequences, internally empty, created to execute instructions and achieve goals set by humans.' At the same time, Suleyman noted that his dispute with Anthropic is based on a common goal—the development of safer technologies for humanity. 'Although our disagreement is serious, it is based on a deep respect for Anthropic and on a goal that I know we share: to increase humanity's chances of safely developing advanced AI systems,' he wrote. 'That is why I consider this discussion so important. The stakes are too high for these issues to be discussed behind closed doors or to become a matter of tribal hostility. We need an open, rigorous, and constructive discussion if we want to get it right.' He also praised Anthropic's CEO Dario Amodei and the company's researchers, calling them thoughtful, principled, and intellectually honest, and emphasized that he respects Anthropic's technological leadership. ## ## 'AI Must Remain Subordinate to Humanity' In a conversation with the BBC, Suleyman stated that people have the right to be concerned about AI; however, he said there are 'very practical steps' that can be taken to keep such systems under control. The head of AI at Microsoft emphasized that greater transparency is needed in how AI systems are trained and evaluated, including independent verification of their behavior and more powerful tools for monitoring and managing this technology. In the BBC interview, Suleyman stressed that future AI systems must remain 'subordinate' to humanity. 'The good news is that every person will have a strong motivation to ensure that the systems we create, which are used worldwide, in every country, remain safe, manageable, and subordinate to humanity,' he said. 'Everyone needs to be aligned on these goals,' he added, referring to the so-called alignment—a industry term that denotes efforts to ensure that AI systems remain in line with human goals and values.
