Answer: WebCrawler

The three core components of modern web search engines–crawling the web for new sites, indexing the results, and providing an interface for users to search that index–are so firmly embedded in the internet user experience that we now simply take them for granted.

In the early days of the web, however, there was nothing that even resembled the powerful search engines of today. The earliest search portals for the web were hand curated and limited in scope (such as W3Catalog and Aliweb). The first search engine to use a web robot–an automated script that crawls the web to discover and index web sites–was JumpStation. Technically speaking JumpStation did include the three elements of modern search engines but the delivery was lackluster because of hardware limitations and the index’s restriction to only titles and document headings–there was no way to search the body of a web page via JumpStation.

The first true full-service web search engine that encompassed crawling, full text indexing, and full text search, in a fashion that would seem natural to a modern user, was WebCrawler. Opened to the public in 1994, WebCrawler offered complete full text search, was the first search engine to be widely acknowledged and used by the public, and set the standard for future search engines.

After changing hands several times throughout the 1990s and early 2000s, WebCrawler continues to exist–albeit in name only–as a front end for Google and Yahoo! search results.