In 2003, after a Pentagon plan to create a data-mining operation known as the Total Information Awareness program was disclosed, a firestorm of protest forced the Bush administration to back off.
However, since then, the intelligence community’s data-mining operations have grown enormously, according to industry and intelligence experts.
The confrontation in Ashcroft’s hospital room took place just one month after a Harvard undergraduate, Mark Zuckerberg, created a startup called Facebook; Twitter would not be founded for another two years. Apple’s iPhone and iPad did not yet exist.
“More and more services like Google and Facebook have become huge central repositories for information,” said Dan Auerbach, a technology analyst with the Electronic Frontier Foundation. “That’s created a pile of data that is an incredibly attractive target for law enforcement and intelligence agencies.”
The spy agencies have long been among the most demanding customers for advanced computing and data-mining software — and even more so in recent years, according to industry analysts.
“They tell you that somewhere there is an American who is going to be blown up,” said a former technology executive, and “the only thing that stands between that and him living is you.”
In 2006, the Bush administration established a program known as the Intelligence Advanced Research Projects Activity, to accelerate the development of intelligence-related technology.
Its stated purpose was to undertake “high-risk, high-payoff research programs that have the potential to provide the US with an overwhelming intelligence advantage over future adversaries.”
IBM’s Watson, the supercomputing technology that defeated human Jeopardy! champions in 2011, is a prime example of the power of data-intensive artificial intelligence. Watson-style computing, analysts said, is precisely the technology that would make the ambitious data-collection program of the NSA seem practical. Computers could instantly sift through the mass of Internet communications data, see patterns of suspicious online behavior and thus narrow the hunt for terrorists.
Both the NSA and the CIA have been testing IBM’s Watson in the last two years, said a consultant who has advised the government and asked not to be named because he was not authorized to speak.
Industry experts say that intelligence and law enforcement agencies also use a new technology, known as trilaterization, that allows tracking of an individual’s location, moment to moment. The data, obtained from cellphone towers, can track the altitude of a person, down to the specific floor in a building. There is even software that exploits the cellphone data seeking to predict a person’s most likely route.
“It is extreme Big Brother,” said Alex Fielding, an expert in networking and data centers.
In addition to opening the Utah data center, reportedly scheduled for this year, the NSA has secretly enlarged its footprint inside the US, according to accounts from whistleblowers in recent years.
In Virginia, a telecommunications consultant reported, Verizon had set up a dedicated fiber-optic line running from New Jersey to Quantico, Virginia, home to a large military base, allowing government officials to gain access to all communications flowing through the carrier’s operations center.