From dating Web sites and London trading floors to online retailing and Internet searches (Google’s search algorithm is now a more closely guarded commercial secret than the recipe for Coca-Cola), algorithms are increasingly determining our collective futures.
“Bank approvals, store cards, job matches and more all run on similar principles,” Ball said. “The algorithm is the god from the machine powering them all, for good or ill.”
So what is an algorithm?
Panos Parpas, a lecturer in the quantitative analysis and decision science (“QUADS”) section of the Department of Computing at Imperial College London, says that wherever we use computers, we rely on algorithms.
“There are lots of types, but algorithms, explained simply, follow a series of instructions to solve a problem. It’s a bit like how a recipe helps you to bake a cake. Instead of having generic flour or a generic oven temperature, the algorithm will try a range of variations to produce the best cake possible from the options and permutations available,” Parpas said.
Parpas added that algorithms are not a new phenomenon.
“They’ve been used for decades — back to Alan Turing and the codebreakers, and beyond — but the current interest in them is due to the vast amounts of data now being generated, and the need to process and understand it. They are now integrated into our lives. On the one hand, they are good because they free up our time and do mundane processes on our behalf. The questions being raised about algorithms at the moment are not about algorithms per se, but about the way society is structured with regard to data use and data privacy. It’s also about how models are being used to predict the future. There is currently an awkward marriage between data and algorithms. As technology evolves, there will be mistakes, but it is important to remember they are just a tool. We shouldn’t blame our tools,” he said.
The “mistakes” Parpas refers to are events such as the “flash crash” of May 6, 2010, when the Dow Jones Industrial Average fell 1,000 points in just a few minutes, only to see the market regain itself 20 minutes later.
The reasons for the sudden plummet has never been fully explained, but most financial observers blame a “race to the bottom” by competing quantitative trading (quants) algorithms widely used to perform high-frequency trading.
Scott Patterson, a Wall Street Journal reporter and author of The Quants, likens the use of algorithms on trading floors to flying a plane on autopilot. The vast majority of trades these days are performed by algorithms, but when things go wrong, as what happened during the flash crash, humans can intervene.
“By far the most complicated algorithms are to be found in science, where they are used to design new drugs or model the climate, but they are done within a controlled environment with clean data. It is easy to see if there is a bug in the algorithm,” Parpas said. “The difficulties come when they are used in the social sciences and financial trading, where there is less understanding of what the model and output should be, and where they are operating in a more dynamic environment. Scientists will take years to validate their algorithm, whereas a trader has just days to do so in a volatile environment.”
Most investment banks now have a team of computer science doctorates coding algorithms, said Parpas, who used to work on such a team.