World Wide Web Wanderer
Script error: No such module "Infobox".Template:Template otherScript error: No such module "Check for unknown parameters".Script error: No such module "check for clobbered parameters".
The World Wide Web Wanderer, also simply called The Wanderer, was a Perl-based web crawler that was first deployed in June 1993 to measure the size of the World Wide Web. The Wanderer was developed at the Massachusetts Institute of Technology by Matthew Gray, who also created back in 1993 one of the 100 first web servers in history, www.mit.edu.[1] The crawler was used to generate an index called the Wandex later in 1993. The Wanderer charted the growth of the web until late 1995.
The Wanderer was probably the first web robot, and, with its index, clearly had the potential to become a general-purpose WWW search engine. The author, Matthew Gray, does not make this claim.[2] Elsewhere, it is stated that the purpose of the Wanderer was not to be a web search engine.[3]
References
External links
- ↑ Script error: No such module "citation/CS1".
- ↑ Matthew Gray's home page - Pertinent page on Matthew Gray's section on MIT site
- ↑ Brian LaMacchia's PhD thesis, section 1.2.3