Winning Warmth in AI Systems
When it comes to Artificial Intelligence, users prefer warmth over competence
Users Prefer the Warmth of an AI System Over Its Competence
A Study by three Technion researchers reveals that AI systems’ competence isn’t enough: for users to choose a system, it needs to have warmth.
Spotify or Apple Music? Waze or Google Maps? Alexa or Siri? Consumers choose between artificial intelligence (AI)-based systems every day. How exactly do they choose which systems to use? Considering the amount of money and efforts spent on AI performance enhancement, one might expect competence and capability to drive users’ choices. Instead, a recent study conducted by researchers from the Faculty of Industrial Engineering and Management at the Technion – Israel Institute of Technology shows that the “warmth” of a system plays a pivotal role in predicting consumers’ choice between AI systems.
Technion AI researchers: (l-r) Dr. Ofra Amir, Dr. Liat Levontin, Zohar Gilad
Research findings from a study featuring more than 1,600 participants, recently published in the Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, offer some insight into the psychology of potential users. The researchers, Zohar Gilad, Prof. Ofra Amir, and Prof. Liat Levontin from the Faculty of Industrial Engineering and Management at the Technion, examined the effects of users’ perception of AI systems’ warmth, that is, the systems’ perceived intent (good or ill), and AI systems’ competence, that is, the systems’ perceived ability to act on those intentions, on the choices they made.
Most of the research done to date regarding warmth perceptions of AI-based systems addressed systems with a virtual or physical presence, such as virtual agents and robots. The current study, though, focused on “faceless” AI systems, with little or no social presence, such as recommender systems, search engines, and navigation apps. For these types of AI systems, the researchers defined warmth as the primary beneficiary of the system. For example, a navigation system can prioritize collecting data about new routes (benefitting the system) over presenting the best-known route, or vice versa.
The research showed that the users’ preference for warmth persisted even when the highly warm system was lacking competence.
Image source: Wikimedia Commons
The researchers found that the system’s warmth was important to potential users, even more than its competence, and they favored a highly warm system over a highly competent system. This preference for warmth persisted even when the highly warm system was overtly deficient in its competence. For example, when asked to choose between two AI systems that recommend car insurance plans, most participants favored a system with low-competence (“using an algorithm trained on data from 1,000 car insurance plans”) and high-warmth (“developed to help people like them”), over a system with high-competence (“using a state-of-the-art artificial neural network algorithm trained on data from 1,000,000 car insurance plans”) and low-warmth (“developed to help insurance agents make better offers”). That is, consumers were willing to sacrifice competence for higher warmth.
These findings are similar to what is known of human interactions: warmth considerations are often more important than competence considerations when judging fellow humans. In other words, people use similar basic social rules to evaluate AI systems and people, even when assessing AI systems without overt human characteristics. Based on their findings, the researchers concluded that AI system designers consider and communicate the system’s warmth to its potential users.
