The Dark Side of Anthropomorphism within AI-Driven Customer Experience: Manipulation, Data Disclosure and Consumer Vulnerability

Abstract

Marketing and information systems scholarship has largely treated anthropomorphism in AI-based customer experience as a positive design lever, linking human-like agent cues to trust, engagement, and conversion. A smaller but expanding body of work, however, suggests that the same cues may manipulate consumers, induce disclosure of sensitive data, and disproportionately harm vulnerable users. These critical findings remain scattered across marketing, human-computer interaction, law, and gerontechnology, with little theoretical integration. Drawing on the Computers Are Social Actors paradigm, the three-factor theory of anthropomorphism, mind perception theory, and the consumer vulnerability framework, this paper develops an integrative conceptual model of the dark side of anthropomorphism in AI-enabled customer experience. We introduce manipulative anthropomorphism as a relational subclass of AI-mediated dark patterns; theorise an anthropomorphism-induced distortion of the privacy calculus that questions informed consent under the GDPR and the EU AI Act; and articulate how anthropomorphic design interacts with consumer vulnerability — particularly among older, lonely, and cognitively impaired consumers — in banking and healthcare. Twelve testable propositions and a multi-method research agenda are advanced. The paper offers a dialectical reframing of anthropomorphism as an ethically contested design practice that warrants regulatory, managerial, and design-level safeguards rather than uncritical adoption.