Opportunities in Image Search

Below are the first 20 results of a Google Images search for a two-word phrase. What do you think the phrase is?

The answer: “computer salesman” (with the quotes)

I found one result (the Mac OS X guy) reasonable, and another (the woman with the iPod) on the borderline. That’s a 10% hit rate.

Among the other 90% were a wolf, a hydroplaning car, a stick figure, Elizabeth Dole, two books, and a guy standing by a telephone booth. Also present were several cartoons, some of which could be relevant if I wanted cartoons, but I didn’t. (The Advanced Image Search feature did not support filtering-out cartoon images, although adding “-cartoon” to the search got most of them.)

As far as I’m aware, Google Images’ results are primarily based on the text adjacent to the image, as opposed to deep analysis of image content. Thus, if we were to look at each result’s surrounding page, we would probably find something about a computer salesman. In some cases, this approach works—for example, searching “U.S. flag” yields good results. In others, per “computer salesman,” it doesn’t.

Two opportunities here:

1. Need an instant party game? Have someone search for terms on an image-search site and then, based only on the results, let the audience guess what is being searched. The searcher can give hints like “you’re getting warmer/colder.” For maximum fun, search a word or phrase with largley (but not totally) misleading and surreal results.
2. There is plenty of room for a better general-purpose image search engine. I say “general purpose” because specialized photo sites already do better by manually attaching keywords or tags to photos. Professional stock-photo sites employ people who do that; photo-sharing sites like Flickr spread the keywording burden among the user base. These efforts lead to better results, but they are limited to a much smaller universe of photos than all public photos Web-wide. So the opportunity is to make image search smarter while keeping the photo universe big.