An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
Through the noise of rushing papers and whirring belts at a print factory in Kyoto, two creators watch their photo essay come to life in broadsheet form — part of an effort to win new audiences in the age of artificial intelligence (AI). Despite the decline of the publishing industry, self-publication and handmade “zine” magazines are growing in popularity in Japan, reflecting the nation’s enduring love of paper in the digital era. While speaking to Agence France-Presse at the plant, his hands black with ink, one of the creators, Kazuma Obara, said: “I think [paper] is a medium that engages all five
‘ABSURD MISTAKE’: The election commission said that there had been a failure to anticipate turnout after 14 polling stations ran short of ballot papers South Korean riot police yesterday cleared protesters from a Seoul polling station after a 35-hour blockade sparked by a shortage of ballot papers during local elections earlier this week. Wednesday’s election was the first nationwide vote since South Korean President Lee Jae-myung took office following the ouster of Yoon Suk-yeol over his short-lived martial law declaration. Lee’s ruling Democratic Party swept most races, but failed to flip the crucial Seoul mayoral seat. The South Korean National Election Commission apologized, blaming a failure to anticipate turnout after 14 polling stations in Seoul ran short of ballot papers. Some polling stations stayed open until 10pm to
France experienced its hottest spring on record, the French weather service said on Tuesday, after an exceptional early heat wave that also broke highs for the season in England and Wales. Meteo-France said the average nationwide temperature over March to May was 13.8°C — about 1.7°C above the norm, and surpassing records set in 2011 and 2020. “The warmest spring since records began in 1900,” it said in a bulletin. All three months were warmer than average, but the onset of an “unprecedented heatwave” late last month pushed the mercury to highs typically seen at the height of the summer. “Our country had never
A Sherpa guide was found crawling to base camp on Mount Everest a week after he went missing and was reunited with his family, who had given up hope he would return. Dawa Sherpa was last seen on Friday last week descending the mountain, but he did not reach base camp even though his client did. The pair were among the last climbers on the mountain as the climbing season came to an end and the route was dismantled. Dawa was located by a cleaning crew on Thursday morning as he was crawling down the snowy slopes around the Khumbu Icefall, just above
RSS