An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
PHISHING: The con might appear convincing, as the scam e-mails can coincide with genuine messages from Apple saying you have run out of storage For a while you have been getting messages from Apple saying “your iCloud storage is full.” They say you have exceeded your storage plan, so documents are no longer being backed up, and photos you take are not being uploaded. You have been resisting Apple’s efforts to get you to pay a minimum of ￡0.99 (US$1.33) a month for more storage, but it seems that you cannot keep putting off the inevitable: You have received an e-mail which says your iCloud account has been blocked, and your photos and videos would be deleted very soon. To keep them you need
The Israeli military has demolished entire villages as part of its invasion of south Lebanon, rigging homes with explosives and razing them to the ground in massive remote detonations. The Guardian reviewed three videos posted by the Israeli military and on social media, which showed Israel carrying out mass detonations in the villages of Taybeh, Naqoura and Deir Seryan along the Israel-Lebanon border. Lebanese media has reported more mass detonations in other border villages, but satellite imagery was not readily available to verify these claims. The demolitions came after Israeli Minister of Defense Israel Katz called for the destruction of
A US YouTuber who caused outrage for filming himself kissing a statue commemorating Korean wartime sex slaves has been sentenced to six months in prison, a court in Seoul said yesterday. Johnny Somali, 25, gained notoriety several years ago for recording himself doing a series of provocative stunts in South Korea and Japan, and streaming them on platforms such as YouTube and Twitch. South Korean authorities indicted Somali — whose real name is Ramsey Khalid Ismael — in 2024 on public order violations and obstruction of business, and banned him from leaving the country. “The court has sentenced him to six months in
The death toll from a shooting in western Afghanistan rose to 11 on Saturday, after gunmen targeted civilians at a picnic spot in Herat, the provincial authority said. Bullet marks were visible on a wall of the Sayed Mohammad Agha Shia shrine, while bloodstains marked a blanket abandoned at the scene. “Eleven people have been recorded dead and eight others wounded from Friday’s incident, with the condition of two of the wounded reported as critical,” Herat’s information office said in a statement. The update raises a toll of seven killed provided on Friday by the Afghan Ministry of Interior Affairs
RSS