Importance: High

It has been reveal by Henk van Ess that Google has a secret search quality lab that utilizes human testers with the goal of improving search quality. More info can be found on his SearchBistro blog. According to Henk, “It’s a lab of humans from all over the world (from China to The Netherlands, from Korea to Brasil) They are paid to check search results of Google every day. Most of the employees, called international agents by Google, were recruited through universities all over the world. The aim is to avoid spam, to get the right sites at the top of the listing and to test new features, not shown to the public yet.”

GoogleGuy responds to ethics issues surrounding the collection and posting of this information on the web:

I have serious reservations about Henk van Ess taking information from one of his own students (who presumably signed a non-disclosure agreement when the student agreed to help rate the quality of our results) and posting that information online. I also believe these web pages said things like “Google Proprietary and Confidential,” but it appears that the screenshots have been cropped to exclude that information. Those are the two things that really made me sad, not the “breaking news” the Google evaluates its own results quality. It shouldn’t be a surprise that Google evaluates the quality of its results in lots of ways–the fact is that every major search engine evaluates its relevance in many ways.

Henk van Ess response can be found here.

2006/06/02