Group items tagged

Grub Next Generation is distributed web crawling system (clients/servers) which helps to build and maintain free (as in freedom) index of the Web.

Torrent search made easy uSniff.com is a fast, easy to use, cool looking, real-time torrent search engine that crawls 16 different torrent sites (for now) to give you the best results available online. Just type your search above and enjoy your download!

seekda™ is the free search engine for Web API (Web Services at the moment) and their providers. We run a focused crawled that gathers information about services available on the Web and we monitor these services daily. We also allow our users to edit certain data regarding providers or their services.

Deep Web Wiki is a volunteer-led search organization that maintains a database containing detailed descriptions of so-called "Deep Web" sites that are either ignored or poorly indexed by the major search engines; it is estimated that ~95 to 99% of Web-accessible content cannot be reached using conventional web crawling methods. The Deep Web Wiki also contains other useful information sites that often are not popular enough to rank on the first few pages of search engine's results--despite their value.The Deep Web Wiki depends on the efforts of Web users to build the wiki and improve the quality and accuracy of site descriptions. Contributions to the index can be made either anonymously or by signing up for a free membership that allows members to track their contributions and watch pages of interest.

"Riddler.io allows you to search in a high quality dataset with more than 400,326,949 hostnames. Unlike others, we do not rely on simple port scanning techniques - we crawl the web, ensuring an in-depth quality data set you will not find anywhere else."