Group items tagged

NutchWAX ("Nutch + Web Archive eXtensions" ) searches web archive collections. The Web Archive eXtensions (WAX) include adaptation of the Nutch fetcher step to go against web archives rather than crawl the open net -- adaptation currently does Internet Archive ARC files only -- and plugins to add extra fields to the index that return an Archive Records' location in the repository, its collection name, etc.

eritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

The Open Library is a project of the Internet Archive. Its goal is to create an online catalog that contains one web page for every book ever published. To do this, it accepts data from a variety of sources: libraries, publishers, book-sellers, and individuals.

Media Cloud is a system that lets you see the flow of the media. The Internet is fundamentally altering the way that news is produced and distributed, but there are few comprehensive approaches to understanding the nature of these changes. Media Cloud automatically builds an archive of news stories and blog posts from the web, applies language processing, and gives you ways to analyze and visualize the data.

Web 2.0 has brought many wonderful innovations and ideas to the Internet. We can no longer imagine the web without a social dimension, and we can no longer imagine an online world that is read-only - it is now a read/write web full of user-generated content. But there is another fairly recent innovation, which might have just as profound implications. We're speaking of the contextual user interface.