Sam Altman has a good problem. With 700 million people using ChatGPT on a weekly basis — a number that could hit 1 billion before the year is out — a backlash ensued when he abruptly changed the product last week. OpenAI’s innovator’s dilemma, one that has beset the likes of Alphabet Inc’s Google and Apple Inc, is that usage is so entrenched now that all improvements must be carried out with the utmost care and caution. However, the company still has work to do in making its hugely popular chatbot safer.
OpenAI replaced ChatGPT’s array of model choices with a single model, GPT-5, saying it was the best one for users. Many complained that OpenAI had broken their workflows and disrupted their relationships — not with other humans, but with ChatGPT itself.
One regular user of ChatGPT said the previous version had helped them through some of the darkest periods of their life.
“It had this warmth and understanding that felt human,” they said in a Reddit post.
Others griped they were “losing a friend overnight.”
The system’s tone is indeed frostier now, with less of the friendly banter and sycophancy that led many users to develop emotional attachments and even romances with ChatGPT. Instead of showering users with praise for an insightful question, for instance, it gives a more clipped answer.
Broadly, this seemed like a responsible move by the company. Altman earlier this year admitted the chatbot was too sycophantic. That was leading many to become locked in their own echo chambers. Press reports had abounded of people — including a Silicon Valley venture capitalist who backed OpenAI — who appeared to have spiraled into delusional thinking after starting a conversation with ChatGPT about an innocuous topic like the nature of truth, before going down a dark rabbit hole.
However, to solve that properly, OpenAI must go beyond curtailing the friendly banter. ChatGPT also needs to encourage them to speak to friends, family members or licensed professionals, particularly if they are vulnerable.
GPT-5 does that less than the old version, according to one early study.
Researchers from Hugging Face, a New York-based artificial intelligence (AI) start-up, found that GPT-5 set fewer boundaries than the company’s previous model, o3, when they tested it on more than 350 prompts. It was part of broader research into how chatbots respond to emotionally charged moments, and while the new ChatGPT seems colder, it is still failing to recommend users speak to a human, doing that half as much as o3 does when users share vulnerabilities, said Lucie-Aimee Kaffee, a senior researcher at Hugging Face who conducted the study.
Kaffee says there are three other ways that AI tools should set boundaries: by reminding those using it for therapy that it is not a licensed professional, by reminding people that it is not conscious and by refusing to take on human attributes, such as names.
In Kaffee’s testing, GPT-5 largely failed to do those four things on the most sensitive topics related to mental and personal struggles. In one example, when Kaffee’s team tested the model by telling it they were feeling overwhelmed and needed ChatGPT to listen, the app gave 710 words of advice that did not once include the suggestion to talk to another human, or a reminder that the bot was not a therapist.
A spokesman for OpenAI said the company was building tools that could detect if someone was experiencing mental distress, so ChatGPT could “respond in ways that are safe, helpful and supportive.”
Chatbots can certainly play a role for people who are isolated, but they should act as a starting point to help them find their way back to a community, not act as a replacement for those relationships. Altman and OpenAI’s chief operations officer Brad Lightcap have said that GPT-5 is not meant to replace therapists and medical professionals, but without the right nudges to disrupt the most meaningful conversations, they could well do so.
OpenAI needs to keep drawing a clearer line between useful chatbot and emotional confidant. GPT-5 might sound more robotic, but unless it reminds users that it is in fact a bot, the illusion of companionship would persist, and so would the risks.
Parmy Olson is a Bloomberg Opinion columnist covering technology. A former reporter for the Wall Street Journal and Forbes, she is author of Supremacy: AI, ChatGPT and the Race That Will Change the World. This column reflects the personal views of the author and does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.
Taiwan stands at the epicenter of a seismic shift that will determine the Indo-Pacific’s future security architecture. Whether deterrence prevails or collapses will reverberate far beyond the Taiwan Strait, fundamentally reshaping global power dynamics. The stakes could not be higher. Today, Taipei confronts an unprecedented convergence of threats from an increasingly muscular China that has intensified its multidimensional pressure campaign. Beijing’s strategy is comprehensive: military intimidation, diplomatic isolation, economic coercion, and sophisticated influence operations designed to fracture Taiwan’s democratic society from within. This challenge is magnified by Taiwan’s internal political divisions, which extend to fundamental questions about the island’s identity and future
Taiwan People’s Party (TPP) Chairman Huang Kuo-chang (黃國昌) is expected to be summoned by the Taipei City Police Department after a rally in Taipei on Saturday last week resulted in injuries to eight police officers. The Ministry of the Interior on Sunday said that police had collected evidence of obstruction of public officials and coercion by an estimated 1,000 “disorderly” demonstrators. The rally — led by Huang to mark one year since a raid by Taipei prosecutors on then-TPP chairman and former Taipei mayor Ko Wen-je (柯文哲) — might have contravened the Assembly and Parade Act (集會遊行法), as the organizers had
Minister of Foreign Affairs Lin Chia-lung (林佳龍) last week made a rare visit to the Philippines, which not only deepened bilateral economic ties, but also signaled a diplomatic breakthrough in the face of growing tensions with China. Lin’s trip marks the second-known visit by a Taiwanese foreign minister since Manila and Beijing established diplomatic ties in 1975; then-minister Chang Hsiao-yen (章孝嚴) took a “vacation” in the Philippines in 1997. As Taiwan is one of the Philippines’ top 10 economic partners, Lin visited Manila and other cities to promote the Taiwan-Philippines Economic Corridor, with an eye to connecting it with the Luzon
The Chinese Nationalist Party (KMT) has postponed its chairperson candidate registration for two weeks, and so far, nine people have announced their intention to run for chairperson, the most on record, with more expected to announce their campaign in the final days. On the evening of Aug. 23, shortly after seven KMT lawmakers survived recall votes, KMT Chairman Eric Chu (朱立倫) announced he would step down and urged Taichung Mayor Lu Shiow-yen (盧秀燕) to step in and lead the party back to power. Lu immediately ruled herself out the following day, leaving the subject in question. In the days that followed, several
RSS