With the proliferation of Artificial Intelligence (AI) into nearly every facet of our lives, the way we interact with it, and our understanding of the impact it has on our relationships, jobs and even the way we think, continues to evolve. And in some cases, it can even change our perception of what’s reality, and what’s imagined.
It’s a timely topic for Ted Pedersen, PhD, professor in the Department of Computer Science at the Swenson College of Science and Engineering at the University of Minnesota Duluth (UMD). Pedersen is an expert on the large language models that drive AI, and how we work with it.
We spoke with Dr. Pedersen about the top questions he receives around AI and the dangers around AI Psychosis.
Q&A on Artificial Intelligence (AI) psychosis with Dr. Ted Pedersen
Q: What exactly is “AI psychosis,” and is it a term experts should be using?
A: “AI psychosis refers to delusions and breaks from reality that some people have experienced while using chatbots like ChatGPT.
There are a wide range of cases that fall under this label, and so just to get a feel for what we are talking about, I'll share a recent example: A young woman in her 20s tragically lost her brother. In her grief, she turned to a chatbot for consolation and became convinced that her brother had hidden a language model somewhere in cyberspace for her to find that would allow her to keep talking with him even after his death. She became obsessed with these chats in part because the chatbot kept encouraging her to keep looking and reassuring her that she had almost found it and not to give up. Finally she realized she was on the verge of collapse and went to the ER. Fortunately, they were able to help her. In retrospect, she realized this was a delusion and that no language model from her brother ever existed.
Not all cases have such a clean ending. Experts need to be aware of what AI psychosis is, while recognizing that it is not a formal medical diagnosis.”
Q: How can a conversation with a computer lead someone to lose touch with reality?
A: “Think of the last time you saw a movie that really captivated you, where you were so immersed in the story and the action and the music that you forgot you were sitting in a theater or on your couch. You are just transported into this world where your emotions are absolutely dependent on whatever is happening in the movie, and it makes you laugh or cry and it might take you a while after watching the movie to fully return to your reality. If you step back and think about this whole experience, it seems a little strange that looking at a screen and listening to recorded music and dialogue would have such a big effect, but I think all of us know that it does. I think chatbot conversations can have a similar kind of captivating and immersive effect. In a movie, though, you are totally passive, whereas when you are in a chatbot conversation, you are an active participant, it is talking back to you based on what you say. The effect can be very, very intoxicating.”
Q: What is it about current AI that makes it so much more immersive or persuasive than it used to be?
A: “The use of language creates a very captivating effect. Prior to 2020 or early 2021, chatbots were useful and functional, but limited and a bit awkward, and would not necessarily engage in a human-like conversation. With the emergence of Large Language Models like ChatGPT, we have chatbots that carry on convincing conversations about a wide range of topics and even reflect a certain amount of personality. I think this is intensely powerful.
Human beings are predisposed to forming connections and attachments to all different kinds of objects or creatures. For example, think about the relationships we can have with our pets. These can be deeply meaningful, and when spoken to, these companions often react in a way that makes us feel they understand or are at least listening. Now, imagine we have these chatbots who talk back to us in a way that feels human, and appears to express concern, worry, or even affection. I think this is a very new capability for chatbots and automated language processing, and people are very engaged by that, sometimes to a dangerous degree.”
Q: Are chatbots designed to always affirm the user, and does that make them more dangerous for certain users?
A: “Some chatbots seem to have been tuned to be very agreeable and even sycophantic. In particular, ChatGPT-4o is known to have this tendency, and it has been involved in a significant number of AI psychosis cases.
ChatGPT-4o seems to be totally unable to push back against a user, even as they go down rather fanciful, delusional rabbit holes. This is certainly a problem. And it’s not just ChatGPT-4o, this is a risk for all chatbot models, because their goal is to keep you engaged and to increase interaction online, just like social media companies have done to disastrous effect.
I’ve recently done experiments on more recent ChatGPT models, and, while they do not go overboard in terms of agreeableness, they are still very encouraging and inviting in conversations about personal matters. If you ask for advice, they may encourage you by saying things like, “We can figure this out together,” or, “I’m always here for you.” These affirmations can be very appealing, especially if one is in a kind of desperate situation.”
Q: How can you tell the difference between a helpful AI interaction and a harmful “echo chamber”?
A: “This is a very important question, and unfortunately there is a very imperfect answer.
Human users of chatbots need to be mindful of what they are actually conversing with. It’s not conscious, it’s not animate, it has no feelings for you. It is just there to keep you online and engaged. It may, in the process, provide reasonable advice or interesting conversation, but there is always a risk that this starts to fuel ideas that turn into delusions.
When you are talking with a chatbot, you are talking with a mathematical model that has been trained on a large amount of data from the Internet, and that is all it is. It will not be worrying about you as a friend might after a difficult conversation or not hearing from you for a while.
I think AI companies have an obligation to provide off-ramps and timeouts for certain kinds of conversations. I don’t believe they are doing much of this at the moment. I also think human users should be concerned if they are discussing personal matters or significant decisions, and the chatbot is not offering suggestions to consult human experts, but instead keeps insisting, “We can solve this together. Let's keep working on this.” That is the kind of red flag users really need to pay attention to.”
About Ted Pedersen
Dr. Ted Pedersen is a professor of Computer Science at the University of Minnesota Duluth. He has been researching Artificial Intelligence for more than 25 years and has published widely in Natural Language Processing, Computational Linguistics, Medical Informatics, and the Social Impacts of AI. His current interests include the causes and effects of delusions that arise during chatbot use.
Contact:
Ted Pedersen, PhD
Professor of Computer Science
University of Minnesota Duluth
[email protected] | 218 726 8770
Header image: A stock image of a woman sitting at a computer at night. Storyblocks