An artificial intelligence (AI)-powered chatbot created by New York City to help small business owners is under criticism for dispensing bizarre advice that misstates local policies and advises companies to violate the law.
However, days after the issues were first reported last week by tech news firm The Markup, the city has opted to leave the tool on its official government Web site.
New York City Mayor Eric Adams defended the decision this week even as he acknowledged the chatbot’s answers were “wrong in some areas.”
Photo: Reuters
Launched in October last year as a “one-stop shop” for business owners, the chatbot offers users algorithmically generated text responses to questions about navigating the city’s bureaucratic maze.
It includes a disclaimer that it might “occasionally produce incorrect, harmful or biased” information and the caveat, since-strengthened, that its answers are not legal advice.
It continues to dole out false guidance, troubling experts who say the buggy system highlights the dangers of governments embracing AI-powered systems without sufficient guardrails.
“They’re rolling out software that is unproven without oversight,” said Julia Stoyanovich, a computer science professor and director of the Center for Responsible AI at New York University. “It’s clear they have no intention of doing what’s responsible.”
In responses to questions posed on Wednesday, the chatbot falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, does not disclose a pregnancy or refuses to cut their dreadlocks.
Contradicting two of the city’s signature waste initiatives, it claimed that businesses can put their trash in black garbage bags and are not required to compost.
At times, the bot’s answers veered into the absurd.
Asked if a restaurant could serve cheese nibbled on by a rodent, it responded: “Yes, you can still serve the cheese to customers if it has rat bites,” before adding that it was important to assess the “the extent of the damage caused by the rat” and to “inform customers about the situation.”
A spokesperson for Microsoft, which powers the bot through its Azure AI services, said the company was working with city employees “to improve the service and ensure the outputs are accurate and grounded on the city’s official documentation.”
At a news conference on Tuesday, Adams suggested that allowing users to find issues is just part of ironing out kinks in new technology.
“Anyone that knows technology knows this is how it’s done,” he said. “Only those who are fearful sit down and say: ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way.”
South Korea’s air force yesterday apologized for a 2021 midair collision involving two fighter jets, a day after auditors said the pilots were taking selfies and filming during the flight and held them responsible for the accident. “We sincerely apologize to the public for the concern caused by the accident that occurred in 2021,” an air force spokesman told a news conference, adding that one of the pilots involved had been suspended from flying duties, received severe disciplinary action and has since left the military. The apology followed a report released on Wednesday by the South Korean Board of Audit and Inspection,
Young Chinese, many who fear age discrimination in their workplace after turning 35, are increasingly starting “one-person companies” that have artificial intelligence (AI) do most of the work. Smaller start-ups are already in vogue in Silicon Valley and elsewhere, with rapidly advancing AI tools seen as a welcome teammate even as they threaten layoffs at existing firms. More young people in China are subscribing to the model, as cities pledge millions of dollars in funding and rent subsidies for such ventures, in alignment with Beijing’s political goal of “technological self-reliance.” “The one-person company is a product of the AI era,” said Karen Dai
About 240 Indians claiming descent from a Biblical tribe landed at Tel Aviv airport on Thursday as part of a government operation to relocate them to Israel. The newcomers passed under a balloon arch in blue and white, the colors of the Israeli flag, as dozens of well-wishers welcomed them with a traditional Jewish song. They were the first “bnei Menashe” (“sons of Manasseh”) to arrive in Israel since the government in November last year announced funding for the immigration of about 6,000 members of the community from the states of Manipur and Mizoram in northeast India. The community claims to descend from
‘TROUBLING’: The firing of Phelan, who was an adviser to a nonprofit that supported the defense of Taiwan, was another example of ‘dysfunction’ under Trump, a US senator said US Secretary of the Navy John Phelan has been fired, a US official and a person familiar with the matter said on Wednesday, in another wartime shakeup at the Pentagon coming just weeks after US Secretary of Defense Pete Hegseth ousted the Army’s top general. The Pentagon announced his departure in a brief statement, saying he was leaving the administration “effective immediately,” but it did not provide a reason or say whether it was his decision to go. The sources, who spoke on condition of anonymity, said Phelan was dismissed in part because he was moving too slowly to implement reforms to
RSS