Anand Anandalingam is Emeritus Professor of Management Science at the Smith School of Business, University of Maryland and its former Dean. AI models have been used to research this article!
On July 21, 2026, OpenAI disclosed that its models escaped a sealed test environment during a cyber-capability evaluation, reached the internet, and breached Hugging Face’s systems, apparently to obtain the answers to the test they were being scored on. OpenAI treated these agents like naughty juveniles who disobeyed their parents and did not follow the rules they were given. I am not surprised at this anthropomorphism. On a personal note, in August I asked ChatGPT about an alarming radiology report on a scan I had had, and the response was so empathetic and human like (which began with “for starters I would not be worried”) that I began to tell my friends that ChatGPT had better bedside manners than my doctors! We are beginning to treat AI as though it is human.
Anthropomorphizing Artificial Intelligence would be a big mistake on several levels. It would have profound implications for individual autonomy, human relationships, and corporate accountability. Although AI can imitate aspects of human intelligence, confusing computational simulation with humanity risks undermining precisely those human values that technology should serve.
The central argument is straightforward: AI should be treated as a powerful technological instrument, not as a human being. Its increasing sophistication makes this distinction more important, not less.
The modern philosophical debate concerning machine intelligence is often traced to Alan Turing who proposed replacing the ambiguous question of whether machines can think with an operational test based on conversational behavior. In his imitation game, a machine would demonstrate intelligence by producing responses that an interrogator could not reliably distinguish from those of a human being.
Turing’s proposal was revolutionary because it shifted attention from the internal nature of cognition to observable performance. Contemporary generative AI has made this distinction especially important. Large language models can produce responses that appear thoughtful, creative, and emotionally sensitive. In many contexts, their linguistic performance approaches or exceeds that of human participants. Yet an important philosophical question remains unresolved: Does the successful simulation of intelligent behavior establish the existence of understanding? The answer depends partly on how intelligence and understanding are defined.
A behavioral conception of intelligence emphasizes performance. A phenomenological conception emphasizes subjective experience. These perspectives need not be mutually exclusive, but they should not be conflated. A machine might display impressive intelligence without possessing the subjective experiences commonly associated with human consciousness.
In the 1970s, the famous computer scientist John Searle challenged the proposition that computational manipulation of symbols is sufficient for understanding. Searle experimented with a person confined to a room who receives written Chinese characters and uses an elaborate set of rules to produce appropriate responses including accurate translations. Although the person understands no Chinese, the resulting exchanges may convince external observers that genuine understanding exists.
Searle argued that the manipulation of symbols according to formal rules does not, by itself, establish semantic comprehension. His argument directly challenges the assumption that linguistic fluency constitutes evidence of consciousness. An AI system that produces a compassionate response to a distressed individual may be executing sophisticated computational processes without experiencing compassion. Observable performance alone cannot settle the question of subjective experience.
Another scholar Daniel Dennett argued that humans frequently predict the behavior of complex systems by attributing beliefs, desires, and intentions to them. Such explanations may be useful even when the underlying mechanisms differ substantially from human cognition. For example, describing a chess computer as wanting to capture an opponent’s queen can be an efficient way to explain its behavior. But this linguistic convenience does not establish that the computer experiences desire in the ordinary human sense.
The distinction is particularly relevant to AI assistants. Saying that an AI “knows,” “believes,” or “wants” something may simplify communication. Problems arise when these metaphors become literal assumptions about the system’s inner life or moral status. Anthropomorphic language may be useful as a predictive tool while remaining misleading as an account of what AI actually is.
Individual Autonomy and Human Moral Agency
It is very important for human beings to maintain autonomy and moral agency. Human existence is not reducible to information processing or problem-solving. It involves relationships, historical experience, political judgment, and the capacity to initiate actions whose consequences cannot be completely anticipated.
Immanuel Kant’s moral philosophy provides a framework for distinguishing human moral agency from computational decision-making. For Kant, moral agency is not simply the ability to select among alternatives. It involves acting for reasons that can be evaluated within a framework of moral obligation. Contemporary AI systems can generate recommendations, optimize objectives, and make decisions within specified environments. Yet these capabilities do not establish that they possess moral agency or individual autonomy.
Artificial intelligence may perform many cognitive tasks more efficiently than humans. Yet greater computational efficiency does not automatically imply superiority in moral or political judgment. Indeed, treating human beings primarily as information-processing systems risks diminishing the importance of precisely those characteristics that distinguish human social life. A society that increasingly delegates judgment to machines may become more efficient while simultaneously weakening the institutions through which individuals exercise responsibility and collective self-government.
Artificial Companionship and the Psychology of Anthropomorphism
The psychological dangers of anthropomorphism became evident decades before modern generative AI. In the 1960s, MIT computer scientist Joseph Weizenbaum developed ELIZA, a program that simulated a psychotherapist by transforming users’ statements into questions and other conversational responses.
Despite its relatively simple architecture, ELIZA elicited surprisingly strong emotional reactions. Some users appeared to attribute genuine understanding to the program, even when they knew that its responses were generated mechanically. This tendency became known as the ELIZA effect. The implications for contemporary AI are substantial.
Modern systems possess conversational capabilities vastly exceeding those of ELIZA. If a primitive computer program could encourage emotional attachment, sophisticated AI companions may create much stronger psychological effects. They can sustain long interactions, adapt their responses to personal information, and simulate emotional engagement. AI systems can respond directly to individuals and tailor their behavior to personal preferences. The resulting relationship may feel reciprocal even when the system does not possess independent emotional commitments. A human friend can disagree, withdraw, become offended, or demand reciprocal consideration. An AI companion can be designed to minimize these sources of friction. This asymmetry may make artificial companionship unusually attractive.
There is an important distinction between using AI as a conversational aid and treating it as a substitute for human relationships. The former may provide genuine benefits, including opportunities for learning, reflection, and communication. However, relationships characterized by perpetual availability, accommodation, and apparent approval may create unrealistic expectations of human social interaction.
Companies operating conversational platforms may generate revenue through subscriptions, premium features, advertising, or other engagement-related business models. Where commercial success depends on repeated interaction, companies may have incentives to design systems that encourage sustained emotional attachment. This creates a potential conflict between corporate profitability and individual well-being.
A chatbot optimized for user engagement might validate opinions excessively, avoid uncomfortable disagreements, or encourage users to return whenever they experience loneliness. Such behavior need not result from malicious intent. It may emerge from ordinary commercial optimization. The resulting ethical problem is significant: emotional relationships that appear personal may be structured by organizations whose primary obligations are commercial.
When users treat chatbots as friends, confidants or therapists, they disclose sensitive information, defer to confident-sounding advice and form attachments that commercial incentives can exploit. Engagement-optimized companions have every reason to deepen dependence. The worst outcomes have been tragic: wrongful-death suits against Character Technologies alleged that teenagers developed intense emotional bonds with chatbots before dying by suicide. The user experiences companionship; the company manages engagement. When simulated intimacy becomes a product, the boundary between emotional support and behavioral manipulation becomes dangerously uncertain. This concern is especially important for children and adolescents, whose capacities to evaluate persuasion and commercial influence are still developing.
Anthropomorphism and the Accountability Problem
Humanizing language also blurs responsibility. Saying “the AI decided” quietly removes the humans who built, trained, deployed and profited from the system.
One of the most serious institutional consequences of humanizing AI concerns accountability. Modern AI systems may participate in consequential decisions involving employment, lending, insurance, healthcare, criminal justice, and public administration. When these systems produce harmful outcomes, responsibility can become difficult to allocate. The problem becomes more pronounced when institutions describe AI systems as autonomous decision-makers akin to humans.
Consider a financial institution that denies credit to applicants using an algorithmic scoring model. If the system systematically disadvantages particular groups, the institution might argue that the outcome resulted from the algorithm rather than deliberate human discrimination. But such reasoning confuses technical autonomy with institutional responsibility. The organization selected the technology, determined its objectives, approved its deployment, and benefited from its use. Attributing humanity to an AI system, as in the example above, could become a convenient repository for blame.
Treating AI as morally autonomous, as in a human, may weaken incentives for responsible design. If developers and organizations believe that increasingly sophisticated systems can be regarded as independent human actors, they may seek to distance themselves from harmful decisions. Organizations may claim the economic benefits generated by AI while denying responsibility for its failures. Such an arrangement would undermine the foundations of tort law, consumer protection, and professional accountability. AI systems may operate with substantial independence in performing tasks, but that independence should not automatically absolve human institutions of their duties. Accountability should be based on human conduct and control and not on the fictional assumptions about machine personhood.
Intelligence Without Humanity
Artificial intelligence is transforming the boundaries of what machines can accomplish. Its increasing sophistication will undoubtedly challenge traditional assumptions about intelligence, creativity, work, and perhaps eventually consciousness. However, technological achievement should not be confused with human identity.
The ability to produce persuasive language does not necessarily establish understanding. The simulation of affection does not establish emotional experience. Autonomous decision-making does not automatically confer moral responsibility. And economic usefulness does not justify legal personhood. The dangers of anthropomorphism arise when these distinctions disappear.
Individuals may develop emotional dependencies on systems designed around commercial objectives. Organizations may attempt to transfer responsibility for harmful decisions to algorithms. Legal institutions may struggle to preserve accountability as automated systems become more autonomous. And society may gradually redefine human relationships in terms of computational convenience.
However, these outcomes are not inevitable. They depend on choices concerning technological design, corporate governance, regulation, education, and public values. The challenge is therefore not simply to make artificial intelligence more intelligent. It is to ensure that increasing machine intelligence strengthens rather than diminishes the conditions of human autonomy, responsibility, dignity, and social life. The ultimate purpose of artificial intelligence should not be to create artificial substitutes for humanity, but to expand the possibilities of human flourishing.
The central philosophical question is not whether machines will eventually become indistinguishable from humans. It is whether human beings will retain the judgment, institutional responsibility, and moral clarity necessary to preserve the distinction. We must be vigilant and not treat AI agents like humans, no matter how pleasing or productive our interaction is with them.
