An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
MINERAL DEPOSITS: The Pacific nation is looking for new foreign partners after its agreement with Canada’s Metals Co was terminated ‘mutually’ at the end of last year Pacific nation Kiribati says it is exploring a deep-sea mining partnership with China, dangling access to a vast patch of Pacific Ocean harboring coveted metals and minerals. Beijing has been ramping up efforts to court Pacific nations sitting on lucrative seafloor deposits of cobalt, nickel and copper — recently inking a cooperation deal with Cook Islands. Kiribati opened discussions with Chinese Ambassador Zhou Limin (周立民) after a longstanding agreement with leading deep-sea mining outfit The Metals Co fell through. “The talk provides an exciting opportunity to explore potential collaboration for the sustainable exploration of the deep-ocean resources in Kiribati,” the government said
Romania’s electoral commission on Saturday excluded a second far-right hopeful, Diana Sosoaca, from May’s presidential election, amid rising tension in the run-up to the May rerun of the poll. Earlier this month, Romania’s Central Electoral Bureau barred Calin Georgescu, an independent who was polling at about 40 percent ahead of the rerun election. Georgescu, a fierce EU and NATO critic, shot to prominence in November last year when he unexpectedly topped a first round of presidential voting. However, Romania’s constitutional court annulled the election after claims of Russian interference and a “massive” social media promotion in his favor. On Saturday, an electoral commission statement
Chinese authorities increased pressure on CK Hutchison Holdings Ltd over its plan to sell its Panama ports stake by sharing a second newspaper commentary attacking the deal. The Hong Kong and Macau Affairs Office on Saturday reposted a commentary originally published in Ta Kung Pao, saying the planned sale of the ports by the Hong Kong company had triggered deep concerns among Chinese people and questioned whether the deal was harming China and aiding evil. “Why were so many important ports transferred to ill-intentioned US forces so easily? What kind of political calculations are hidden in the so-called commercial behavior on the
The head of Shin Bet, Israel’s domestic intelligence agency, was sacked yesterday, days after Israeli Prime Minister Benjamin Netanyahu said he no longer trusts him, and fallout from a report on the Oct. 7, 2023, Hamas attack. “The Government unanimously approved Prime Minister Benjamin Netanyahu’s proposal to end ISA Director Ronen Bar’s term of office,” a statement said. He is to leave his post when his successor is appointed by April 10 at the latest, the statement said. Netanyahu on Sunday cited an “ongoing lack of trust” as the reason for moving to dismiss Bar, who joined the agency in 1993. Bar, meant to
RSS