An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
Kouri Richins, a Utah mother who published a children’s book about grief after the death of her husband is to serve a life sentence for his murder without the possibility of parole, a judge ruled on Wednesday. Richins was convicted in March of aggravated murder for lacing a cocktail given to her husband, Eric Richins, with five times the lethal dose of fentanyl at their home near Park City in 2022. A jury also found her guilty of four other felonies, including insurance fraud, forgery and attempted murder for trying to poison her husband weeks earlier on Feb. 14, 2022, with a
‘PERSONAL MISTAKES’: Eileen Wang has agreed to plead guilty to the felony, which comes with a maximum sentence of 10 years in federal prison A southern California mayor has agreed to plead guilty to acting as an illegal agent for the Chinese government and has resigned from her city position, officials said on Monday. Eileen Wang (王愛琳), mayor of Arcadia, was charged last month with one count of acting in the US as an illegal agent of a foreign government. She was accused of doing the bidding of Chinese officials, such as sharing articles favorable to Beijing, without prior notification to the US government as required by law. The 58-year-old was elected in November 2022 to a five-person city council, from which the mayor is selected
DELA ROSA CASE: The whereabouts of the senator, who is wanted by the ICC, was unclear, while President Marcos faces a political test over the senate situation Philippine authorities yesterday were seeking confirmation of reports that a top politician wanted by the International Criminal Court (ICC) had fled, a day after gunfire rang out at the Philippine Senate where he had taken refuge fearing his arrest. Senator Ronald “Bato” dela Rosa, the former national police chief and top enforcer of former Philippine president Rodrigo Duterte’s “war on drugs,” has been under Senate protection and is wanted for crimes against humanity, the same charges Duterte is accused of. “Several sources confirmed that the senator, Senator Bato, is no longer in the Senate premises, but we are still getting confirmation,” Presidential
HELP DENIED? The US Department of State said that the Cuban leadership refuses to allow the US to provide aid to Cubans, ‘who are in desperate need of assistance’ US Secretary of State Marco Rubio on Wednesday said that Cuba’s leadership must change, as Washington renewed an offer of US$100 million in aid if the communist nation agrees to cooperate. Cuba has been suffering severe economic tumult led by an energy shortage that plunged 65 percent of the country into darkness on Tuesday. Cuba’s leaders have blamed US sanctions, but Rubio, a Cuban American and critic of the government established by Fidel Castro, said the system was to blame, including corruption by the military. “It’s a broken, nonfunctional economy, and it’s impossible to change it. I wish it were different,” he told
RSS