Know your target

• Crawler-based engines: To this point, the search engines discussed fall largely into this category. A crawler-based search engine (like Google) uses an automated software agent (called a crawler) to visit, read, and index web sites. All the information collected by the crawler is returned to a central repository. This is called indexing. It is from this index that search engine results are pulled. Crawler-based search engines revisit web pages periodically in a time frame determined by the search engine administrator.
• Human-powered engines: Human-powered search engines rely on people to submit the information that is indexed and later returned as search results. Sometimes, humanpowered search engines are called directories. Yahoo! is a good example of what, at one time, was a human-powered search engine. Yahoo! started as a favorites list belonging to two people who needed an easier way to share their favorite web site. Over time, Yahoo! took on a life of its own. It’s no longer completely human-controlled. A newer search engine called Mahalo (www.mahalo.com) is entirely human-powered, however, and it’s creating a buzz on the Web.
• Hybrid engine: A hybrid search engine is not entirely populated by a web crawler, nor entirely by human submission. A hybrid is a combination of the two. In a hybrid engine, people can manually submit their web sites for inclusion in search results, but there is also a web crawler that monitors the Web for sites to include. Most search engines today fall into the hybrid category to at least some degree. Although many are mostly populated by crawlers, others have some method by which people can enter their web site information.

It’s important to understand these distinctions, because how your site ends up indexed by a search engine may have some bearing on when it’s indexed. For example, fully automated search engines that use web crawlers might index your site weeks (or even months) before a human-powered search engine. The reason is simple. The web crawler is an automated application. The human-powered search engine may actually require that all entries be reviewed for accuracy before a site is included in search results.

In all cases, the accuracy of search engine results will vary according to the search query that is used. For example, entries in a human-powered search engine might be more technically accurate, but the search query that is used will determine if the desired results are returned.