Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
MetaCrawler
open-in-new
<h2 id="history">History</h2> <p>MetaCrawler was the first metasearch engine, originally developed in 1994 at the <a href="/facts/University_of_Washington/36PG6qkG">University of Washington</a> by graduate student Erik Selberg and Professor <a href="/facts/Oren_Etzioni/k0DRfkMb">Oren Etzioni</a> as Erik Selberg's Ph.D. qualifying project.<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a> Originally, it was created in order to provide a reliable abstraction layer to web search engine programs in order to study semantic structure on the World Wide Web. However, it was a useful service in its own right, and had a number of research challenges. </p><p>MetaCrawler was originally operating on four <a href="/facts/Digital_Equipment_Corporation/ztxs6keA">Digital Equipment Corporation</a> <a href="/facts/AlphaStation/LRCUKK8L">AlphaStations</a><a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a> and processing several hundred thousand queries per day. This was starting to create significant bandwidth load at UW. It became clear that MetaCrawler needed to have some method of paying for the queries it was forwarding to the primary search engines. Some time after the search engine launched, <a href="/facts/NetBot/eY8nwt6Q">NetBot</a>, Inc., which was cofounded by Etzioni,<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> was initiated to commercialize MetaCrawler<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> and three other UW programs: Ahoy! The HomePage Finder, Occam, and ShopBot. Ahoy! and Occam were never actually commercialized. NetBot then combined the core of MetaCrawler with ShopBot to create a meta-shopping website, Jango.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> </p><p>MetaCrawler launched on July 7, 1995.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> </p> <p>As of late 1995, MetaCrawler logged over 7,000 search queries per week, and accessed six services: Galaxy, <a href="/facts/InfoSeek/URcvrdzN">InfoSeek</a>, <a href="/facts/Lycos/36hwgx19">Lycos</a>, <a href="/facts/Open_Text/5mWYx5fM">Open Text</a>, <a href="/facts/WebCrawler/ra7D1xyb">WebCrawler</a> and <a href="/facts/Yahoo/nKhTaV3M">Yahoo</a>.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> By late 1996, there were over 150,000 queries per day.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> </p><p>MetaCrawler's owners were unable to determine a reasonable business model, so in January 1997 they sold it to another Internet startup company, Go2Net,<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> in which Microsoft co-founder <a href="/facts/Paul_Allen/S0euHIKT">Paul Allen</a> later invested a 54 percent stake.<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> Go2Net went public in April that year, registering on <a href="/facts/Nasdaq/C6GlcQcB">Nasdaq</a>.<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> MetaCrawler had about 30,000 daily visitors at the start of 1997, but by mid 1998 jumped to 275,000.<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> </p> <p>NetBot would eventually be purchased by <a href="/facts/Excite_(web_portal)/EEd5j0Wu">Excite</a> in October 1997 for $35 million, where Jango became part of the Excite Network Shopping Channel.<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> Both Selberg and Etzioni resumed working for UW until 1999, when they joined Go2Net for a year, quitting just prior to Go2Net's acquisition by <a href="/facts/InfoSpace/FPX3p62L">InfoSpace</a>, Inc. in July 2000 for $4.2 billion.<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> By that time, Go2Net had purchased another metasearch engine, <a href="/facts/Dogpile/YTKoyX0q">Dogpile</a>.<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> </p><p>In 2014, MetaCrawler was merged into another one of InfoSpace's search engines, <a href="/facts/Zoo.com/4nUNRuUr">Zoo.com</a>,<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> which was originally launched in 2006.<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> The MetaCrawler domain at first redirected to Zoo.com,<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a><a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a> but was afterwards changed to redirect to msxml.excite.com, the search page for Excite, also operated by InfoSpace.<a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a><a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a> </p><p>In July 2016, InfoSpace was sold by parent company Blucora to OpenMail for $45 million, putting MetaCrawler under the ownership of OpenMail.<a class="footnote-ref" id="fnref:22" href="#fn:22"><sup>22</sup></a> OpenMail was later renamed System1.<a class="footnote-ref" id="fnref:23" href="#fn:23"><sup>23</sup></a> </p><p>In 2017, MetaCrawler relaunched as its own search engine.<a class="footnote-ref" id="fnref:24" href="#fn:24"><sup>24</sup></a> </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/List_of_search_engines/cznsla0a">List of search engines</a></li> <li><a href="/facts/WebCrawler/ra7D1xyb">WebCrawler</a></li> <li><a href="/facts/InfoSpace/FPX3p62L">InfoSpace</a></li> <li><a href="/facts/Dogpile/YTKoyX0q">Dogpile</a></li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="http://www.metacrawler.com/">Official website</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"Erik Selberg IoT Conference Bio". Retrieved 2019-09-25. <a href="https://tmt.knect365.com/iot-world-asia/speakers/erik-selberg" target="_blank">https://tmt.knect365.com/iot-world-asia/speakers/erik-selberg</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>"Federated Search". federatedsearchblog.com. Retrieved 2019-01-29. <a href="http://federatedsearchblog.com/2008/11/28/erik-selberg-federated-search-luminary-part-i/" target="_blank">http://federatedsearchblog.com/2008/11/28/erik-selberg-federated-search-luminary-part-i/</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>"Robots Mailing List Archive: The Metacrawler, Reborn". webdoc.gwdg.de. Retrieved 2019-01-29. <a href="http://webdoc.gwdg.de/ebook/aw/1999/webcrawler/mailing-lists/robots/2081.html" target="_blank">http://webdoc.gwdg.de/ebook/aw/1999/webcrawler/mailing-lists/robots/2081.html</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>"ASEE PRISM - Apr 1999 - Cover Story - Building Strategic Partnerships". www.prism-magazine.org. Archived from the original on 2015-09-09. Retrieved 2019-01-29. <a href="https://web.archive.org/web/20150909082557/http://www.prism-magazine.org/april/html/cover.html" target="_blank">https://web.archive.org/web/20150909082557/http://www.prism-magazine.org/april/html/cover.html</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>"Shopping Engine History - SingleFeed, Shopping Engines made easy". www.singlefeed.com. Retrieved 2019-01-29. <a href="https://www.singlefeed.com/resources/enginehistory.php" target="_blank">https://www.singlefeed.com/resources/enginehistory.php</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"Multi-Service Search and Comparison Using the MetaCrawler". www.w3.org. Retrieved 2019-02-12. <a href="https://www.w3.org/Conferences/WWW4/Papers/169/Overview.html" target="_blank">https://www.w3.org/Conferences/WWW4/Papers/169/Overview.html</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>"Multi-Service Search and Comparison Using the MetaCrawler". homes.cs.washington.edu. Retrieved 2019-01-29. <a href="https://homes.cs.washington.edu/~etzioni/papers/Overview.html" target="_blank">https://homes.cs.washington.edu/~etzioni/papers/Overview.html</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>"The MetaCrawler Architecture for Resource Aggregation on the Web". homes.cs.washington.edu. Retrieved 2019-01-29. <a href="https://homes.cs.washington.edu/~etzioni/papers/ieee-metacrawler.html" target="_blank">https://homes.cs.washington.edu/~etzioni/papers/ieee-metacrawler.html</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>"Robots Mailing List Archive: The Metacrawler, Reborn". webdoc.gwdg.de. Retrieved 2019-01-29. <a href="http://webdoc.gwdg.de/ebook/aw/1999/webcrawler/mailing-lists/robots/2081.html" target="_blank">http://webdoc.gwdg.de/ebook/aw/1999/webcrawler/mailing-lists/robots/2081.html</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"Paul Allen sets 54-percent stake in Go2Net - Mar. 15, 1999". money.cnn.com. Retrieved 2019-01-29. <a href="https://money.cnn.com/1999/03/15/technology/go2net/" target="_blank">https://money.cnn.com/1999/03/15/technology/go2net/</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Court, Randolph (1998-04-23). "Go2net's Low-Overhead Plan". Wired. ISSN 1059-1028. Retrieved 2019-01-29. <a href="https://www.wired.com/1998/04/go2nets-low-overhead-plan/" target="_blank">https://www.wired.com/1998/04/go2nets-low-overhead-plan/</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>King, Suzanne. "O.P. Web site developer is Seattle-bound". American City Business Journals. Retrieved 2019-01-29. <a href="https://www.bizjournals.com/kansascity/stories/1998/05/18/story4.html" target="_blank">https://www.bizjournals.com/kansascity/stories/1998/05/18/story4.html</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"Excite to buy NetBot". CNET. Retrieved 2019-01-29. <a href="https://www.cnet.com/news/excite-to-buy-netbot/" target="_blank">https://www.cnet.com/news/excite-to-buy-netbot/</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Francisco, August Cole, Bambi (27 July 2000). "InfoSpace to acquire Go2Net". MarketWatch. Retrieved 2019-01-29.{{cite web}}: CS1 maint: multiple names: authors list (link) <a href="https://www.marketwatch.com/story/infospace-to-acquire-go2net" target="_blank">https://www.marketwatch.com/story/infospace-to-acquire-go2net</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>"Google Added To Go2Net's MetaCrawler and Dogpile Metasearch Services – News announcements – News from Google – Google". googlepress.blogspot.com. Retrieved 2019-01-29. <a href="https://googlepress.blogspot.com/2000/01/google-added-to-go2nets-metacrawler-and.html" target="_blank">https://googlepress.blogspot.com/2000/01/google-added-to-go2nets-metacrawler-and.html</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>"MetaCrawler". 2014-01-01. Archived from the original on 2014-01-01. Retrieved 2019-01-29. <a href="https://web.archive.org/web/20140101001213/http://www.metacrawler.com/" target="_blank">https://web.archive.org/web/20140101001213/http://www.metacrawler.com/</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>"New Kid-Friendly Search Engine Zoo.com Offers Wealth of Information Without the Worry". www.businesswire.com. 2006-11-14. Retrieved 2019-01-29. <a href="https://www.businesswire.com/news/home/20061114005334/en/New-Kid-Friendly-Search-Engine-Zoo.com-Offers-Wealth" target="_blank">https://www.businesswire.com/news/home/20061114005334/en/New-Kid-Friendly-Search-Engine-Zoo.com-Offers-Wealth</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>"What is Zoo?". www.computerhope.com. Retrieved 2019-01-29. <a href="https://www.computerhope.com/jargon/z/zoo.htm" target="_blank">https://www.computerhope.com/jargon/z/zoo.htm</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>"Module 1". mason.gmu.edu. Archived from the original on 2019-01-26. Retrieved 2019-01-29. <a href="https://web.archive.org/web/20190126221503/http://mason.gmu.edu/~ebotts/507/module1.html" target="_blank">https://web.archive.org/web/20190126221503/http://mason.gmu.edu/~ebotts/507/module1.html</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> <li id="fn:20"><p>"Søgemaskiner - Search-on.dk JR Corp". www.search-on.dk. Retrieved 2019-01-29. <a href="http://www.search-on.dk/soegemaskiner.html" target="_blank">http://www.search-on.dk/soegemaskiner.html</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li> <li id="fn:21"><p>"MetaCrawler". Archived from the original on 2016-01-01. Retrieved 2019-01-29. <a href="https://web.archive.org/web/20160101002606/http://www.metacrawler.com/" target="_blank">https://web.archive.org/web/20160101002606/http://www.metacrawler.com/</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li> <li id="fn:22"><p>"Blucora to sell InfoSpace business for $45 million". Seattle Times. July 5, 2016. <a href="http://www.seattletimes.com/business/technology/blucora-to-sell-infospace-business-for-45-million/" target="_blank">http://www.seattletimes.com/business/technology/blucora-to-sell-infospace-business-for-45-million/</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></p></li> <li id="fn:23"><p>"System1 raises $270 million for 'consumer intent' advertising". L.A. Biz. Retrieved 2017-12-01. <a href="https://www.bizjournals.com/losangeles/news/2017/09/20/system1-raises-270-million-for-consumer-intent.html" target="_blank">https://www.bizjournals.com/losangeles/news/2017/09/20/system1-raises-270-million-for-consumer-intent.html</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></p></li> <li id="fn:24"><p>"MetaCrawler". 2017-04-03. Archived from the original on 2017-04-03. Retrieved 2019-01-29. <a href="https://web.archive.org/web/20170403063154/http://www.metacrawler.com/" target="_blank">https://web.archive.org/web/20170403063154/http://www.metacrawler.com/</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></p></li> </ol>