Great New Search Engines

April 2009

The bad news is that I’m writing this with a broken arm. The good news: I have some lovely people waiting on me. I expect lunch to arrive at any moment.

So why would I write a column even though I have a broken arm (thanks to a nasty fall when playing tennis), and even though I haven’t really had the time or vitality to come up with a topic? I guess it’s become a habit. I’ve been doing this for over 14 years without missing.

In that 14 years the changes have been remarkable. The only search engine at that time that indexed web pages was WebCrawler, which indexed 4,000 pages. Google hit the 1 trillion mark last summer.

Search engines have become a part of our lives, and Google dominates. But it’s falling behind in a number of areas. My guess — and hope — is that it will emulate the technologies offered by those who have passed it.

The big thing Google lacks right now is a page preview function. I’ve already written about Searchme. When you do a search, it doesn’t just give you a list of results with capsule descriptions. Rather, it lets you spin through a preview of the pages on a carousel, letting you see at a glance what the page contains. Since I last wrote about Searchme, it has added an important feature: an unobtrusive “index” at left that is sort of like Google’s search result listing. One advantage is that it makes it easier to jump ahead rather than scroll page by page. Again, the beauty of this preview approach is that it’s so much faster. You aren’t having to wait for each page to load in the same way that you have to do when clicking on Google’s search results.

MelZoo is a new search engine that’s sort of a cross between Google and Searchme. It divides your screen in half and puts the Google-like search results in the left pane. And as you run your mouse over each search result, it puts up a preview of the page in the right pane. It’s simple and effective and fast.

A major problem with search engines, though, is that they typically only index web pages. Much Internet content does not now consist of web pages but of databases that you access. This is called the deep web. According to an article in the New York Times, it’s vastly larger than the 1 trillion web pages indexed by Google and includes things like shopping catalogs, flight schedules, and financial information.

The Times reported that there are now companies that are building search engines to access the deep web, which includes millions of databases. That, of course, is no small task. One such search engine in development is Kosmix, which matches searches with the databases most likely to have relevant information. Another is the humorously named DeepPeep, which has the astounding goal to eventually “crawl” and index all of the publicly available databases on the web.

In addition, specialized search engines continue to develop. One that I particularly like, and that I believe I’ve mentioned, is VideoSurf, which is a pretty astonishing search engine for searching for videos. When you type in a search term, it returns a list of videos. But for each video, VideoSurf has captured frames and presents them in a horizontal sequence — such that without watching the video, you can see at a glance what it contains from beginning to end. VideoSurf makes it easy, via a checklist, to refine your searches to particular content type, category, and video source. And you can sort the search results according to popularity, duration, and more.

For people search, Pipl seems to be the best. A search on my name finds an astonishing amount of information: addresses, phone number, e-mail address, age, social networking profiles, photos, blog posts, etc. etc. Of course, that’s partly because I have such a web presence. Part of Pipl’s savvy is that it doesn’t just search web pages but also areas of the deep web that have personal information. Obviously, you may not want to have your personal information out there. But at least Pipl lets you know what other people can find out about you.

Luckily, my name doesn’t come up in CriminalSearches. In some cases, you can use this to even find out if a person has had traffic offenses, though apparently it varies state by state.

Well, I’m glad I wrote this, despite my broken arm. Hopefully you’ll find something useful.

© 2009 by Jim Karpen, Ph.D.

E-mail Jim Karpen