An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
Archeologists in Peru on Thursday said they found the 5,000-year-old remains of a noblewoman at the sacred city of Caral, revealing the important role played by women in the oldest center of civilization in the Americas. “What has been discovered corresponds to a woman who apparently had elevated status, an elite woman,” archeologist David Palomino said. The mummy was found in Aspero, a sacred site within the city of Caral that was a garbage dump for more than 30 years until becoming an archeological site in the 1990s. Palomino said the carefully preserved remains, dating to 3,000BC, contained skin, part of the
TRUMP EFFECT: The win capped one of the most dramatic turnarounds in Canadian political history after the Conservatives had led the Liberals by more than 20 points Canadian Prime Minister Mark Carney yesterday pledged to win US President Donald Trump’s trade war after winning Canada’s election and leading his Liberal Party to another term in power. Following a campaign dominated by Trump’s tariffs and annexation threats, Carney promised to chart “a new path forward” in a world “fundamentally changed” by a US that is newly hostile to free trade. “We are over the shock of the American betrayal, but we should never forget the lessons,” said Carney, who led the central banks of Canada and the UK before entering politics earlier this year. “We will win this trade war and
‘BODIES EVERYWHERE’: The incident occurred at a Filipino festival celebrating an anti-colonial leader, with the driver described as a ‘lone suspect’ known to police Canadian police arrested a man on Saturday after a car plowed into a street party in the western Canadian city of Vancouver, killing a number of people. Authorities said the incident happened shortly after 8pm in Vancouver’s Sunset on Fraser neighborhood as members of the Filipino community gathered to celebrate Lapu Lapu Day. The festival, which commemorates a Filipino anti-colonial leader from the 16th century, falls this year on the weekend before Canada’s election. A 30-year-old local man was arrested at the scene, Vancouver police wrote on X. The driver was a “lone suspect” known to police, a police spokesperson told journalists at the
North Korean leader Kim Jong-un has unveiled a new naval destroyer, claiming it as a significant advancement toward his goal of expanding the operational range and preemptive strike capabilities of his nuclear-armed military, state media said yesterday. North Korea’s state-run Korean Central News Agency (KCNA) said Kim attended the launching ceremony for the 5,000-tonne warship on Friday at the western port of Nampo. Kim framed the arms buildup as a response to perceived threats from the US and its allies in Asia, who have been expanding joint military exercises amid rising tensions over the North’s nuclear program. He added that the acquisition
RSS