An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
ROCKY RELATIONS: The figures on residents come as Chinese tourist numbers drop following Beijing’s warnings to avoid traveling to Japan The number of Chinese residents in Japan has continued to rise, even as ties between the two countries have become increasingly fractious, data released on Friday showed. As of the end of December last year, the number of Chinese residents had increased by 6.5 percent from the previous year to 930,428. Chinese people accounted for 22.6 percent of all foreign residents in Japan, making them by far the largest group, Japanese Ministry of Justice data showed. Beijing has criticized Tokyo in increasingly strident terms since Japanese Prime Minister Sanae Takaichi last year suggested that a military conflict around Taiwan could
Japan is to downgrade its description of ties with China from “one of its most important” in an annual diplomatic report, according to a draft reviewed by Reuters, as relations with Beijing worsen. This year’s Diplomatic Bluebook, which Japanese Prime Minister Sanae Takaichi’s government is expected to approve next month, would instead describe China as an important neighbor and the relationship as “strategic” and “mutually beneficial.” The draft cites a series of confrontations with Beijing over the past year, including export controls on rare earths, radar lock-ons targeting Japanese military aircraft and increased pressure around Taiwan. The shift in tone underscores a deterioration
A retired US colonel behind a privately financed rocket launch site in the Dominican Republic sees the project as a response to China’s dominance of the space race in Latin America. Florida-based Launch on Demand is slated to begin building a US$600 million facility in a remote region near the border with Haiti late this year. The project is designed to meet surging demand for the heavy-lift rockets needed to put clusters of satellites into orbit. It is also an answer to China’s growing presence in the region, said CEO Burton Catledge, a former commander of the US Air Force’s 45th Operations
Germany is considering Australia’s Ghost Bat robot fighter as it looks to select a combat drone to modernize its air force, German Minister of Defense Boris Pistorius said yesterday. Germany has said it wants to field hundreds of uncrewed fighter jets by 2029, and would make a decision soon as it considers a range of German, European and US projects developing so-called “collaborative combat aircraft.” Australia has said it will integrate the Ghost Bat, jointly developed by Boeing Australia and the Royal Australian Air Force, into its military after a successful weapons test last year. After inspecting the Ghost Bat in Queensland yesterday,
RSS