Imitation is Flattery, Isn’t It?

Google ignited a bit of a firestorm when it claimed that Bing was poaching its search results. To test their theory, they baited a honeypot and caught Bing with its hand in the jar.

Google calls it cheating and most media seem to agree.  I call it crowdsourcing.

When Google noticed that their search results for esoteric or misspelled (and then corrected by the user) terms were showing up in Bing’s results, they got suspicious.

We gave 20 of our engineers laptops with a fresh install of Microsoft Windows running Internet Explorer 8 with Bing Toolbar installed. As part of the install process, we opted in to the “Suggested Sites” feature of IE8, and we accepted the default options for the Bing Toolbar.

The crucial bit is that they “opted in to the Suggested Sites feature.”  This sends anonymous information about where you travel online to Bing.  They use the information to improve their search results figuring that people know better than algorithms which links are more interesting.

So, if I follow a link from one Web page to another the Suggested Sites feature makes note of that.  If enough people do the same, then Bing factors that into determining search results and the order in which they appear. I want to repeat something for clarity: this only happens if you opt-in to this feature in IE8 or install the Bing toolbar which opts you in by default.

Why did Google’s honeypot work so well? Actually, it didn’t.  They only got traction on about 8% of the bogus search queries they created.

Likely those terms were so random that vanishingly few had ever searched on that term before.  So, when 20 Google engineers start clicking through on Google’s bogus search result, Bing took notice.  Google.com does have a public page rank of 10 so Bing ought to take notice that a vast majority of folks who visit a single page (http://www.google.com/search?q=hiybbprqag) all go to the Wiltern’s seating cart.

Absent any other data on the term hiybbprqag, the traffic generated by Google’s honeypot quickly caused their bogus result to rise to the top of Bing’s search criteria.

The Real Question: Is This Cheating?

No.  Everyone providing information to Bing has opted in to the system.  (Whether they knew it or not is a different discussion.  I am a strong proponent of opt-in mechanisms which the Bing toolbar does not use, but IE8 does).  Bing simply uses that information to provide search results that are, in part, ranked by users clicks.

Interestingly, Google does something very similar: the page rank of any given page is affected by the pages that link to it.  So, if the New York Times were to link to this humble post, my site would get a huge boost – NYTimes.com has a page rank of 9.  Google would use the fact that the New York Times thinks I’m a credible source (really, I am!)  among 100’s of other variables to influence how my site compares to others when we show up in the same keyword searches.  The difference is Microsoft is using real-time user click data while Google is using more static HTML links.

Google hasn’t proven plagiarism, as some have claimed.  Bing does not behind-their-back run a parallel search on Google for every term entered into their site and use Google’s results to rearrange their own.  And even if they did, it would be hard to claim that is illegal – lazy, yes; but not illegal – as Google and Bing’s search results are not limited by copyright as sites such as Bing vs. Google have shown.

You can legitimately complain that Bing isn’t updating their search ranking algorithm rapidly enough. Were they doing so, they wouldn’t not have been caught so flat-footed.  It doesn’t take much to look for edge cases where, out of Bing’s 1000 factors, the clickthrough rate is the only one that registered > 0.0000.

This isn’t Microsoft being caught cheating, it’s Microsoft’s QA department being caught napping on the job.