Thu, Aug 20, 2009 - Page 9 News List

Now you don’t see us: Google algorithm blues

How a price-comparison Web site dropped out of view on Google raises questions of market dominance and unfair business practices

By Richard Wray  /  THE GUARDIAN

A British husband-and-wife team has been waging a three-year battle to get their price comparison Web site recognized by Google in a saga that sheds new light on the power of the world’s largest search engine.

Foundem.co.uk directs shoppers to online deals for goods such as TVs or flights, but has struggled since the day it suddenly disappeared from Google search results for these categories.

There is no evidence that Google is in any way being dishonest or unfair in the way that it ranks such Web sites, but Foundem’s fight to discover what happened has highlighted the ever-growing influence of its mysterious search algorithms.

Many consumers believe Google’s search engine works on a formula that was created by founders Sergey Brin and Larry Page, and that was that: They set it running and the rest is history.

In fact, as those in the Internet industry know, Google carries out regular “tweaks” of its algorithm. About 450 a year in fact. When they are made, the sheer scale of Google — it has an estimated 90 percent market share in Britain — means these can have huge and often unintended consequences.

Another example occurred about the time Foundem saw its traffic from the site drop off, when Google stopped returning results for Amazon.com for more than a day. The Internet retailer is a household name, so not appearing in Google’s results for a time is not the end of the world. However, for a start-up company, getting into Google’s results is a question of survival.

“A typical Web site in the UK receives around two in every five visits from search engines and obviously, the vast majority of those come via Google,” according to Robin Goad, research director at Hitwise UK, an Internet traffic monitor. “For a site that is selling things like Foundem then it is up to 60 percent to 70 percent of their traffic that comes from Google. It is definitely true that if you fall foul of the rules ... it can have a massive impact on your business.”

How Google works

Google’s search algorithm relies on more than 200 individual signals to decide what sorts of site are relevant to its Web searchers. These start with PageRank, the breakthrough bearing the surname of Google’s co-founder Larry Page, which measures a Web site’s relevance by the number of other sites linked to it, and extends to measures of unique content and whether text on the page is replicated — either on the site itself or elsewhere on the Web — and even whether it is spelt correctly.

One problem faced by Foundem is that, as a price comparison service, its raison d’etre is to pull in information from elsewhere on the Web, so a lot of text — such as product descriptions — will be replicated.

Google says its users do not want to be presented with a list of options on the site they visit, while the Foundem pair point out that this is, in essence, all that Google itself does. Meanwhile, Foundem results are appearing relatively highly on Yahoo and Bing — Microsoft’s search engine.

Relevance is also crucial for Google’s AdWords platform. It assigns a quality score for each bid for a keyword — the word used in a user’s search query that will trigger an ad. A high-quality score means a lower cost per keyword. That, plus a high click-through rate, will get the ad placed higher on the Google page than other ads even though rival advertisers may have spent more on their keywords.

Working out what works with Adwords is very complex, and like Google’s search system has spawned an industry of people who claim to know exactly how to place advertisers high on Google’s results.


Exactly what those rules are is far from clear because Google does not want to give too much information away in order to avoid Web developers “gaming” the system and promoting sites that are not, in fact, relevant to users. That would lead users to try other search engines and affect its traffic.

Trying to find out why exactly your site has dropped off is difficult. You can fill out an online reconsideration request, but its wording makes it quite clear that Google assumes you have done something untoward and tried to game its system.

Foundem’s Shivaun Raff explains: “We sent reconsideration requests then started sending e-mails to as many people as we could find, expecting each time that this was just a failure of process, expecting that once our case was in front of people with the power to do something it would get fixed.”

But Google insists its search rankings are only ever driven by a desire to make results useful.

“We can’t comment on individual cases,” a spokesman said. “But our systems are designed to produce the most relevant and useful results for the people who use Google search.

“Where sites are adding little value or original content, they are likely to fall in our ranking. Surveys of our users show that what they most dislike when they search is to receive multiple results from sites showing the same or very similar content,” he said.

This story has been viewed 2528 times.

Comments will be moderated. Keep comments relevant to the article. Remarks containing abusive and obscene language, personal attacks of any kind or promotion will be removed and the user banned. Final decision will be at the discretion of the Taipei Times.

TOP top