The ancestor of all search engines is Archie FAQ invented by Allen Ntaghi, Peter deutsch and Bill Whelan of McGill University in Montreal in 1990. At that time, the World Wide Web had not appeared. Archie is the first program to automatically index anonymous FTP website files on the Internet, but it is not a real search engine. Archie is a searchable FTP file name list. The user must enter the exact file name search, and then Archie will tell the user which FTP address can download the file.

Inspired by Archie's popularity, Nevada University of System Computing Services developed a Gopher(Gopher FAQ) search Veronica(Veronica FAQ, at 1993. Jughead is another gopher search tool later.

Because the robot program specially used to retrieve information crawls on the network like a spider, the robot program of search engine is called spider FAQ program. The world's first spider program, MIT Matthew Gray's World Wide Web Rover, was used to track the scale of Internet development. At first, it was only used to count the number of servers on the Internet. Later, it was developed to capture web sites.

Corresponding to the rover, 1993 10 In June, Martijn Koster created Ali Web (Martijn Koster announces the availability of Ali Web), which is equivalent to the HTTP version of Archie. ALIWEB does not use the internet to search for robots. If the webmaster wants his own web pages to be included in ALIWEB, he needs to submit the introduction and index information of each of his own web pages, similar to Yahoo, which is well known later.

At the end of 1993, some search engines based on this principle began to appear one after another. The three most famous ones are: JumpStation in Scotland, World Wide Web Worm by Oliver McBryan of the University of Colorado (McBryan's first instruction) and Knowledge Base-based Software Engineering (RBSE) Spider by NASA.

1February, 993, the idea of six Stanford university students was to analyze the relationship between words in order to search a large amount of information on the Internet more effectively. This is so exciting. Later, he became famous for concept search. In May 2002, Excite, which was acquired by Infospace, stopped its own search engine and switched to the meta-search engine Dogpile.

1994 1 month, the first searchable and browsable directory EINet Galaxy(Tradewave Galaxy) was launched. In addition to website search, Gopher and Telnet search are also supported.

1In April, 994, two doctoral students from Stanford University, Yang Zhiyuan and david filo, founded Yahoo. With the increase of the number of visits and links, Yahoo Directory began to support simple database search. Because of Yahoo! The data of is manually input, so it can't really be classified as a search engine. In fact, it's just a searchable directory. The search efficiency is obviously improved. (Yahoo will use Altavista, Inktomi and Google to provide search engine services. )

1994 At the beginning of this year, Brian Pinkerton, a computer student at the University of Washington, started his small project "Brian Pinkerton announced the availability of web crawler". 1On April 20th, 994, WebCrawler only contained content from 6000 servers. WebCrawler is the first full-text search engine on the Internet that supports searching all words in documents. Before it, users can only search through URL and abstract, which usually come from manual comments or programs that automatically extract the first 100 words of the text. (Later, webcrawler was successively acquired by AOL and excite, and now it uses Dogpile, a meta-search engine like Excite. )

Lycos (Machine Translation Center of Carnegie Mellon University) is another important progress in the history of search engines. Michael Mauldin of Carnegie Mellon University connected John Leavitt's spider program with its indexing program and created Lycos. On July 20th, 1994, Lycos with a data volume of 54,000 was officially released. In addition to relevance ranking, Lycos also provides prefix matching and character similarity restrictions. Lycos is the first to use automatic summarization of web pages in search results. Its biggest advantage is that it far exceeds the data volume of other search engines: 65,438+0,994-394,000 documents in August; 1995 1 month-1.5 million documents; 1996 1 1 month-more than 60 million documents. (Note:1In April, 999, Lycos stopped its own spider and turned to Fast to provide search engine services. )

Infoseek is another important search engine. Although the company claimed that 1994 1 was established, its search engine did not meet the public until the end of the year. At first, Infoseek was just an obscure search engine, followed by Yahoo. Lycos, on the other hand, has no unique innovation. However, its development and subsequent word-of-mouth prove that it is not always important to be the first on stage at the beginning. Infoseek's friendly user interface and a large number of additional services (such as ups tracking, news, directories, etc.) make it more and more famous. On June1995+February 65438, the strategic agreement with Netscape made it a powerful search engine: when users click the search button on Netscape browser, Infoseek's search service will pop up, and Yahoo! Provide this service. (Note: Infoseek later became famous for its relevance. In February of 200 1 year, Infoseek stopped its own search engine and began to use Overture search results. )

1995, a new form of search engine-a summary of meta-search engine appeared. Users only need to submit a search request once, and the meta-search engine is responsible for the conversion and submission to multiple pre-selected independent search engines, and all the query results returned by independent search engines are returned to users after collection and processing. The first meta-search engine was Eric Selberg, a graduate student of Washington University, and Metacrawler of Oren Ezioni. Meta search engine sounds good in concept, but the search effect is always unsatisfactory, so no meta search engine has ever had a strong position.

DEC's altavista (some netizens need to visit AltaVista through p-roxy from 200 1 summer, and qbseach searches AltaVista without p-roxy, so they can only display the first page of search results) is a latecomer, and it didn't appear until 1995 (AltaVista public beta press release). But a large number of innovative functions made it quickly reach the peak of search engines at that time. The most prominent advantage of Altavista is its speed (search engine 9238: funny, altavista is to show the powerful computing power of DEC Alpha chip). Other new features of Altavista have changed the definition of search engines forever. AltaVista is the first search engine to suppORt natural language search, and AltaVista is the first search engine to implement advanced search syntax (such as and, or, NOT, etc.). Users can use AltaVista to search newsgroups and get articles from the Internet. They can also search for words in picture names, titles, Java applets and ActiveX objects. AltaVista also claims to be the first search engine that supports users to submit or delete URLs to the web index database, and it can be started within 24 hours. One of the most interesting new features of AltaVista is to search all websites with URL links. AltaVista has also made many innovations in the user-oriented interface. It puts "tips" in the search box area to help users better express their search style. These tips are updated frequently, so that users will see many interesting functions that they may never know after searching for a few times. This series of functions are gradually widely adopted by other search engines. 1997, AltaVista released a graphic demonstration system, LiveTopics, to help users find what they want from thousands of search results.

Then HotBot. 1On September 26th, 995, Eric Brewer, a CS teaching assistant in Berkeley, California, and Paul Gauthier, a doctoral student, founded inktomi (UC Berkeley announcements inktomi). 1on may 20th, 996, Inktomi company was established, and a powerful HotBot appeared in front of the world. It claims that it can crawl more than 65438+ 1 100 million pages every day, so it has new content far beyond other search engines. HotBot also uses cookie to store users' personal search preferences. (Hotbot is one of the most popular search engines in the following years, which was acquired by Lycos)

Northernlight company was established in Cambridge, Massachusetts on September 1995. 1995. In August, 1997, the Northernlight search engine officially appeared. It used to be one of the largest search engines in the database. It has no stop word. It has excellent current news, a special collection of more than 7 100 publications, and a good advanced search grammar. It is the first to support simple and automatic classification of search results. (June 5438+1October 65438+June 6, 2002, the search engine of Northern Lights was shut down, and it was subsequently acquired by divine. However, in Nlresearch, "World Wide Web Only" is selected, and the Northernlight search engine can still be used. )

Before 1998 10, Google was only a small project of Stanford University. 1995, doctoral student Larry Page began to study search engine design, and registered the domain name of google.com on September 15, 1997. At the end of 1997, with the participation of sergey brin, Scott Hassan and Allen Strumberg, BachRub began to provide Demo. 1February, 999, Google completed the transformation from Alpha version to Beta version. Google regards1September 27th, 998 as its birthday.

Google's innovations in integrated search, multilingual support, user interface and other functions, such as Pagerank, dynamic summary, webpage snapshot, DailyRefresh, multi-document format support, map stock dictionary search, etc., once again changed the definition of search engine forever like Altavista.

Before mid-2000, Google was highly respected for its search accuracy, but its database was not as large as other search engines and lacked advanced search syntax, so its use value was not very high and its promotion was not fast. It was not until the database was upgraded in mid-2000 that it was selected as a search engine by Yahoo.

Fast(Alltheweb) Company was founded in 1997, which is a by-product of academic research of Norwegian University of Science and Technology (NTNU). 1May, 999, released its own search engine AllTheWeb. The goal of Fast is to be the largest and fastest search engine in the world, and it has been close in recent years. Fast(Alltheweb) can automatically classify web pages according to ODP, support Flash and pdf search, support multilingual search, and also provide news search, picture search, video, MP3 and FTP search, which has extremely powerful advanced search function.

Teoma originated from a project of 1998 Rutgers University. Professor Apostolos Gerasoulis led Chinese professor Tao Yang and others to establish Teoma in Taway, Pisca, New Jersey, which made its debut in the spring of 20001year, was acquired by Ask Jeeves in September 20001year, and was released again in April 2002. Teoma's database is still small, but it has two brilliant functions: supporting Refine is similar to automatic classification; At the same time, it also provides professional link directory resources.

Wisenut was founded by Korean-born Yeogirl Yun. The beta version was released in the spring of 200 1, and the official version was released on September 5th, 20065438+0. In April 2002, it was acquired by looksmart, a catalog provider. Wisenut also has two brilliant functions: WiseGuide contains similar automatic classification and related search words; Preview search results.

Gigablast was founded by Matt Wells, a former Infoseek engineer. Pre-Beta version was displayed in March 2002, and released on July 2, 20021day. Gigablast's database is still small, but it also provides snapshots of web pages. A special function is to index web pages in real time, and your web pages can be searched as soon as they are submitted (note: the meat buns function of spammers is temporarily turned off).

Openfind was founded in 1998 65438+ 10, and its technology originated from GAIS laboratory led by Professor Wu Sheng of Chung Cheng University in Taiwan. At first, Openfind was just a Chinese search engine. Used to be the best Chinese search engine. At its peak, it provided Chinese search engines for Sina, Qimo and Yahoo, but after 2000, the market was gradually divided by Baidu and Google. In June, 2002, Openfind re-released the beta version of Openfind search engine based on GAIS30 project, and launched PolyRankTM, and announced that it had accumulated 3.5 billion web pages and started to enter the English search field. Since then, the technology upgrade has obviously accelerated.

Skynet of Peking University is the research result of the National Key Science and Technology Project "Chinese-English Chinese Coding and Distributed Information Discovery" in the Ninth Five-Year Plan. Developed by the Network and Distributed Systems Research Office of Peking University Computer Department, the service was officially provided on CERNET on June 29th, 1997. At the beginning of 2000, a new Skynet search engine research group was established, which was funded and developed by the National 973 Key Basic Research and Development Planning Project Fund, and included about 60 million web pages. Make use of the advantages of Education Network and have powerful ftp search function.

Baidu June 5438 +2000 10, Li Yanhong, the patent inventor of Hyperchain Analysis and former Infoseek senior engineer, and his friend Xu Yong (Dr. Berkeley, California) founded Baidu Company in Zhongguancun, Beijing. Baidu.com search engine beta was released in August, 20001year (Baidu only provides search engines from Sohu, Sina, Tom and other portals), and Baidu search engine was officially released on October 22nd, 2000 10/0. Although Baidu only provides Chinese search, it currently contains more than 90 million Chinese web pages, which may be the largest Chinese database. Other functions of Baidu search engine include: webpage snapshot, webpage preview/all webpage preview, related search words, typo correction tips, news search, Flash search and information express search. After the Blitzen project was launched in March 2002, the technical upgrade was obviously accelerated.