On Feb. 28 last year Sewell Setzer III, a 14-year-old boy from Florida, killed himself at the urging of a lifelike artificial intelligence (AI) character generated by Character.AI, a platform that is also reportedly hosting pro-anorexia AI chatbots that encourage disordered eating among young people. Clearly, stronger measures are urgently needed to protect children and young people from AI.
Of course, even in strictly ethical terms, AI has immense positive potential, from promoting human health and dignity to improving sustainability and education among marginalized populations. However, these promised benefits are no excuse for downplaying or ignoring the ethical risks and real-world costs. Every violation of human rights must be seen as ethically unacceptable. If a lifelike AI chatbot provokes the death of a teenager, that AI could play a role in advancing medical research is no compensation.
Nor is the Setzer tragedy an isolated case. In December last year, two families in Texas filed a lawsuit against Character.AI and its financial backer, Google, alleging that the platform’s chatbots sexually and emotionally abused their school-age children, resulting in self-harm and violence.
Illustration: Mountain People
We have seen this movie before, having already sacrificed a generation of children and teens to social-media companies that profit from their platforms’ addictiveness. Only slowly did we awaken to the social and psychological harms done by “anti-social media.” Now, many countries are banning or restricting access, and young people themselves are demanding stronger regulation.
However, humanity cannot wait to rein in AI’s manipulative power. Owing to the huge quantities of personal data that the tech industry has harvested from us, those building platforms such as Character.AI can create algorithms that know us better than we know ourselves. The potential for abuse is profound. AIs know exactly which buttons to press to tap into our desires, or to get us to vote a certain way. The pro-anorexia chatbots on Character.AI are merely the latest, most outrageous example. There is no good reason why they should not be banned immediately.
Yet time is running out, because generative AI models have been developing faster than expected — and they are generally accelerating in the wrong direction. The “Godfather of AI,” the Nobel laureate cognitive scientist Geoffrey Hinton, continues to warn that AI could lead to human extinction.
“My worry is that the invisible hand is not going to keep us safe. So just leaving it to the profit motive of large companies is not going to be sufficient to make sure they develop it safely. The only thing that can force those big companies to do more research on safety is government regulation,” Hinton said.
Given big tech’s consistent failure to uphold ethical standards, it is folly to expect these companies to police themselves. Google poured US$2.7 billion into Character.AI last year despite its well-known problems. However, while regulation is obviously needed, AI is a global phenomenon, which means we should strive for global regulation, anchored in a new global enforcement mechanism, such as an international data-based systems agency at the UN, as I have proposed.
The fact that something is possible does not mean that it is desirable. Humans bear the responsibility to decide which technologies, which innovations, and which forms of progress are to be realized and scaled up, and which ought not be. It is our responsibility to design, produce, use, and govern AI in ways that respect human rights and facilitate a more sustainable future for humanity and the planet.
Sewell would almost certainly still be alive if a global regulation had been in place to promote human rights-based AI, and if a global institution had been established to monitor innovations in this domain. Ensuring that human rights and the rights of the child are respected requires governance of technological systems’ entire life cycle, from design and development to production, distribution, and use.
Since humans already know that AI can kill, there is no excuse for remaining passive as the technology continues to advance, with more unregulated models being released to the public every month. Whatever benefits these technologies might someday provide, they would never be able to compensate for the loss that all who loved Sewell have already suffered.
Peter G. Kirchschlager, professor of ethics and director of the Institute of Social Ethics at the University of Lucerne, is a visiting professor at Federal Institute of Technology Zurich.
Copyright: Project Syndicate
As strategic tensions escalate across the vast Indo-Pacific region, Taiwan has emerged as more than a potential flashpoint. It is the fulcrum upon which the credibility of the evolving American-led strategy of integrated deterrence now rests. How the US and regional powers like Japan respond to Taiwan’s defense, and how credible the deterrent against Chinese aggression proves to be, will profoundly shape the Indo-Pacific security architecture for years to come. A successful defense of Taiwan through strengthened deterrence in the Indo-Pacific would enhance the credibility of the US-led alliance system and underpin America’s global preeminence, while a failure of integrated deterrence would
It is being said every second day: The ongoing recall campaign in Taiwan — where citizens are trying to collect enough signatures to trigger re-elections for a number of Chinese Nationalist Party (KMT) legislators — is orchestrated by the Democratic Progressive Party (DPP), or even President William Lai (賴清德) himself. The KMT makes the claim, and foreign media and analysts repeat it. However, they never show any proof — because there is not any. It is alarming how easily academics, journalists and experts toss around claims that amount to accusing a democratic government of conspiracy — without a shred of evidence. These
The Executive Yuan recently revised a page of its Web site on ethnic groups in Taiwan, replacing the term “Han” (漢族) with “the rest of the population.” The page, which was updated on March 24, describes the composition of Taiwan’s registered households as indigenous (2.5 percent), foreign origin (1.2 percent) and the rest of the population (96.2 percent). The change was picked up by a social media user and amplified by local media, sparking heated discussion over the weekend. The pan-blue and pro-China camp called it a politically motivated desinicization attempt to obscure the Han Chinese ethnicity of most Taiwanese.
On Wednesday last week, the Rossiyskaya Gazeta published an article by Chinese President Xi Jinping (習近平) asserting the People’s Republic of China’s (PRC) territorial claim over Taiwan effective 1945, predicated upon instruments such as the 1943 Cairo Declaration and the 1945 Potsdam Proclamation. The article further contended that this de jure and de facto status was subsequently reaffirmed by UN General Assembly Resolution 2758 of 1971. The Ministry of Foreign Affairs promptly issued a statement categorically repudiating these assertions. In addition to the reasons put forward by the ministry, I believe that China’s assertions are open to questions in international
RSS