Visualizing the words used in the 2011-2012 Republican Primary debates. The method: We collected transcripts from the American Presidency Project at UCSB, categorized them by hand, then ranked lemmatized word-phrases (or n-grams) by their frequency of use. Word-phrases can be made of up to five words. Our ranking agorithm accounts for things such as exclusive word-phrases - meaning, it won't count "United States" twice if it's used in a higher n-gram such as "President of the United States." While still in beta, the mini-app is responsive and easy to use. The next challenge, I think, is to really show what everyone talked about. For example, click on education and you see Newt Gingrich, Ron Paul, and Rick Perry brought those up. Then roll over the names to see the words each candidate used related to that topic. You get some sense of content, but it's still hard to decipher what each actually said about education.

The Investigative Dashboard (ID) is a work in progress, that is designed to showcase the potential for collaboration and data-sharing between investigative reporters across the world. The initiative is spearheaded by the Organized Crime and Corruption Reporting Project, the Romanian Center for Investigative Journalism, the Forum for African Investigative Reporters and the International Center for Journalists, and will expand to include other institutional members of the Global Investigative Journalism Network. The project is coordinated by Paul Cristian Radu (of OCCRP and CRJI) and Justin Arenstein (of FAIR) and was developed while both were in residence at Stanford University as Knight fellows. The John S. Knight Fellowships for Professional Journalists made possible the ID by providing access to the know-how of co-fellow journalists and of experts at Stanford University and in Silicon Valley. This first iteration of the ID website shares detailed methodologies, resources, and links for journalists to track money, shareholders, and company ownership across international borders. It also shares video tutorials, and other tools, to help journalists navigate often rapidly evolving data-sources. Future versions of ID will offer more advanced collaborative workspaces, data-archives, and discounted (or, where possible, free) access to expensive or proprietary research services. But, perhaps most importantly, the ID will campaign for investigative centres across the world to collaborate with each other to improve the depth and impact of their reportage.

Goals and Topics Our goal with this workshop is to bring together data storytellers from diverse disciplines and continue the conversation of how these different fields utilize each other's techniques and articulate principles for telling data narratives. Our target participants are researchers, journalists, bloggers, and others who seek to understand how visualizations support narrative, stories, and other communicative goals. Participants may be designers of such visualizations or designers of tools that support the creation of narrative visualizations. Visualizations that serve as a "community mirror" and thus create opportunities for discussion, reflection and sharing within a social network are also suitable topics. While we are inspired by many visualizations that display personal histories and storylines, our focus is on visualization situated in storytelling contexts, not necessarily visualizations of stories. Specific topics of interest may include, but are not limited to: Media and genres Embedding visualizations in social media to tell stories Multimodal storytelling with visualization (e.g. narrated or acted visualization, such as Rosling's Gapminder presentations) Non-traditional narrative - games and other procedural narratives incorporating data Visualization in (data)journalism - how news stories and visualization can complement each other Visualizations that support specific types of stories: Personal stories ("Here's a history of my cancer treatment") Community and collaboration stories ("How has our Facebook group changed over the past year?") Public data sets and narrative ("What is your Senator doing with your taxes?") Fictional, semi-fictional, and non-fiction stories

"RegExr is an online tool for editing and testing Regular Expressions (RegExp / RegEx). It provides a simple interface to enter RegEx expressions, and visualize matches in real-time editable source text. It also provides a handy RegExp snippet sidebar with descriptions and usage examples to make it easier to learn Regular Expressions through trial and error. It isn't as powerful as a product like RegExBuddy, but it has the advantage of being online and free. I will be releasing a free desktop version for Mac OSX and Windows built with AIR in the next day or two. So far this has only taken a day of development, and the main app is only 150 lines of code. Flex 3 makes this kind of app so darn simple to put together."

Learn to code Codecademy is the easiest way to learn how to code. It's interactive, fun, and you can do it with your friends.

Site available in Spanish and English Medialab-Prado is a program of the Department of Arts of the City Council of Madrid, aimed at the production, research, and dissemination of digital culture and of the area where art, science, technology, and society intersect. Many workshops for the production of projects, conferences, seminars, encounters, project exhibition, concerts, presentations, etc. take place in its versatile space. All activities are free and open to the general public. Our primary objective is to create a structure where both research and production are processes permeable to user participation. To that end, Medialab-Prado offers: A permanent information, reception, and meeting space attended by cultural mediators. Open calls for the presentation of proposals and participation in the collaborative development of projects. We have several on-going programmes, which are as follows: Interactivos?: creative uses of electronics and programming Inclusiva.net: research and reflections on the network culture Visualizar: data visualization tools and strategies Commons Lab: trans-disciplinary discussion on the Commons AVLAB: audio-visual and sound creation http://medialab-prado.es/article/que_es

Eurostat was established in 1953 to meet the requirements of the Coal and Steel Community. Over the years its task has broadened and when the European Community was founded in 1958 it became a Directorate-General (DG) of the European Commission. Eurostat's key role is to supply statistics to other DGs and supply the Commission and other European Institutions with data so they can define, implement and analyse Community policies. The result: Eurostat offers a whole range of important and interesting data that governments, businesses, the education sector, journalists and the public can use for their work and daily life. With the development of Community policies, Eurostat's role has changed. Today, collecting data for EMU and developing statistical systems in candidate countries for EU membership are more important than ten years ago.

They Rule Overview They Rule aims to provide a glimpse of some of the relationships of the US ruling class. It takes as its focus the boards of some of the most powerful U.S. companies, which share many of the same directors. Some individuals sit on 5, 6 or 7 of the top 1000 companies. It allows users to browse through these interlocking directories and run searches on the boards and companies. A user can save a map of connections complete with their annotations and email links to these maps to others. They Rule is a starting point for research about these powerful individuals and corporations. Context A few companies control much of the economy and oligopolies exert control in nearly every sector of the economy. The people who head up these companies swap on and off the boards from one company to another, and in and out of government committees and positions. These people run the most powerful institutions on the planet, and we have almost no say in who they are. This is not a conspiracy, they are proud to rule, yet these connections of power are not always visible to the public eye. Karl Marx once called this ruling class a 'band of hostile brothers.' They stand against each other in the competitve struggle for the continued accumulation of their capital, but they stand together as a family supporting their interests in perpetuating the profit system as whole. Protecting this system can require the cover of a 'legitimate' force - and this is the role that is played by the state. An understanding of this system can not be gleaned from looking at the inter-personal relations of this class alone, but rather how they stand in relation to other classes in society. Hopefully They Rule will raise larger questions about the structure of our society and in whose benefit it is run. The Data We do not claim that this data is 100% accurate at all times. Corporate directors have a habit of dying, quitting boards, joining new ones and most frustratingly passing on their name

Welcome to Graphviz Available translations:Belorussian What is Graphviz? Graphviz is open source graph visualization software. Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. It has important applications in networking, bioinformatics, software engineering, database and web design, machine learning, and in visual interfaces for other technical domains. Features The Graphviz layout programs take descriptions of graphs in a simple text language, and make diagrams in useful formats, such as images and SVG for web pages, PDF or Postscript for inclusion in other documents; or display in an interactive graph browser. (Graphviz also supports GXL, an XML dialect.) Graphviz has many useful features for concrete diagrams, such as options for colors, fonts, tabular node layouts, line styles, hyperlinks, rolland custom shapes.

Matthew Ericson « Illustrator MultiExporter script: Now with JPG and EPS When Maps Shouldn't Be Maps View full interactive map on nytimes.com » Often, when you get data that is organized by geography - say, for example, food stamp rates in every county, high school graduation rates in every state, election results in every House district, racial and ethnic distributions in each census tract - the impulse is since the data CAN be mapped, the best way to present the data MUST be a map. You plug the data into ArcView, join it up with a shapefile, export to Illustrator, clean up the styles and voilà! Instant graphic ready to be published. And in many cases, that's the right call.

Open Data Cook BookMaking Open Data Accessible for EveryoneAbout the Cook BookThe open data cook book is collecting recipes for ways to find and use open data, particularly open data of social value - such as open government data, or open data for campaigners and charities. Working with data can seem scary. But it doesn't have to be. There are many different ways to make data useful - and lots of different gadgets to help you. Take a look at the growing list of cook book recipes to find simple step by step ideas for making use of open data. RecipesYou can find a list of the recipes so far here. Drafts, ideas and notesIn the cooks notebook you can find draft notes on using different datasets and sketches that might develop into recipes in future. Get InvolvedFind out how to get involved here or jump right in and create a recipe. Tweet with the #opendatacookbook tag, or bookmark content on del.icio.us 'opendatacookbook ' to share with the project. Join the mailing list to discuss developments. UpdateAfter a brief experiment with Drupal as a CMS for the cook book - we've switched to DokuWiki for a bit to make compiling a list of recipes a lot easier before we work out the best way to run the Cook Book.

ChangeDetection.com provides page change monitoring and notification services to internet users worldwide. Anyone can use our service to monitor any website page for changes. Just fill in the form below, we will create a change log for the page and alert you by email when we detect a change in the page text. We've been doing it since 1999. It's free.

Again, US-centric, but this might generate some ideas of what could be accomplish in your city/nation. Late yesterday we announced a bunch of new features for Influence Explorer: http://sunlightlabs.com/blog/2011/ie-corporate-accountability/ As the blog post explains, you can now find information about a corporation's EPA violations, federal advisory committee memberships, and participation in the rulemaking process -- all in one place. I wanted to highlight that last feature a bit more, though. To my knowledge, this is the first time that the full corpus of public comments submitted to regulations.gov has been available for bulk download and analysis. This isn't a coincidence: regulations.gov is built using technologies that make scraping it unusually difficult. This is unfortunate, since everyone seems to agree that federal rulemakings are gaining in importance -- both because of congressional gridlock that leaves the regulatory process as a second-best option, and because of calls to simplify the regulatory landscape as a pro-growth measure. It's an area where influence is certainly exerted -- rulemakers are obliged to review every comment -- but little attention is paid to who's flooding dockets with comments, and which directions rules are being pushed. It's taken us several months to develop a reliable solution and to obtain past rulemakings, but we now have the data in hand. We plan to do much more with this dataset, and we're hoping that others will want to dig in, too. You can find a link to the bulk download options in the post above -- the full compressed archive of extracted text and metadata is ~16GB, but we've provided options for grabbing individual agencies' or dockets' data. If anyone wants the original documents (PDFs, DOCs, etc) we can talk through how to make that happen, but as they clock in at 1.5TB we'll want to make sure folks know what they're getting into before we spend the time and bandwidth. Finally, note that we currently o

An Applied Demography Toolbox A collection of applied demography programs, scripts, spreadsheets and databases. If you have any questions (such as how to apply the tools to your own work), recommendations or additions, you can send a message to me (Eddie Hunsinger) at edynivn@gmail.com. If you would like to use, share or reproduce any information or ideas from the linked files, be sure to cite the respective source. Here is a neat article that gives this site some inspiration. Acknowledgments. Subscribe to new postings . Return to Eddie's homepage. http://www.demog.berkeley.edu/~eddieh/toolbox.html#MedianCalculator

These tend to be US-centric, but there are universal tools here for statistical analysis.

ITA Software is proud to introduce Needlebase™, a revolutionary platform for acquiring, integrating, cleansing, analyzing and publishing data on the web. Using Needlebase through a web browser, without programmers or DBAs, your data team can easily: acquire data from multiple sources: A simple tagging process quickly imports structured data from complex websites, XML feeds, and spreadsheets into a unified database of your design. merge, deduplicate and cleanse: Needlebase uses intelligent semantics to help you find and merge variant forms of the same record. Your merges, edits and deletions persist even after the original data is refreshed from its source. build and publish custom data views: Use Needlebase's visual UI and powerful query language to configure exactly your desired view of the data, whether as a list, table, grid, or map. Then, with one click, publish the data for others to see, or export a feed of the clean data to your own local database. Needlebase dramatically reduces the time, cost, and expertise needed to build and maintain comprehensive databases of practically anything. Read on to learn more about Needlebase's capabilities and our early adopters' success stories, or watch our tutorial videos. Then sign up to get started! http://needlebase.com

kdmc data visualization tutorials KDMC produces a wealth of digital media tutorials to support our training sessions and classes. While the focus of some tutorials is on technology and journalism, most are general enough to be of use to anyone.

A very good collection of dataviz tips and tools

Be sure to view the video tutorial: http://www.cc.gatech.edu/gvu/ii/jigsaw/Jigsaw-tutorial.movhttp://www.cc.gatech.edu/gvu/ii/jigsaw/Jigsaw-tutorial.mov http://www.cc.gatech.edu/gvu/ii/jigsaw/views.html Jigsaw: Visual Analytics for Exploring and Understanding Document Collections System Views Jigsaw presents the individual reports in a document collection and the entities within those reports through a series of visualizations. We call these visualizations the system views. Below, we illustrate each view provided by the system and briefly describe their characteristics. Click on the individual images to see a larger version of the view. Also, a tutorial video illustrates the different views as well and the interactive behavior for each view can be seen on the video tutorial page. -tj

Also see "The Information Interfaces Group, an HCI research group in the School of Interactive Computing at Georgia Tech, develops computing technologies that help people take advantage of information to enrich their lives. " http://www.cc.gatech.edu/gvu/ii/

Europe's One-Stop Shop on Public Sector Information (PSI) Re-use Working to Stimulate PSI Re-use The aim of the ePSIplatform is to strenghthen community, stimulate action, report developments and monitor progress towards a stronger and more transparent environment for the growth of national and European PSI re-use markets. Interactive One-Stop Shop News about European and International PSI re-use Developments Emerging Good Practices Examples of New PSI Re-use Products and Services Information about Legal cases on PSI Re-use Discussion, Communication and Information Sharing across the European PSI community Expert PSI Community Meetings (two meetings per year) Guest Bloggers from the European PSI Community PSI Topic Reports - Analytical Briefings on PSI Re-use (across sectors and issues) PSI Re-use Resources What is PSI re-use? - background information - the European PSI Re-use Directive translated into 27 EU languages and European Commission Communication on Re-use of PSI Directive (language variants) (May 2009) plus other key links and resources.

A good platform to understand what is happening in the Open Data - Europe movement.

How It Works SpeakerText combines artificial and human intelligence to offer low-cost, high-quality video transcription. Sign up for an account Import your video library (we currently support Ooyala, Brightcove, YouTube, Vimeo, SoundCloud, Wistia and Blip.tv), or add your videos one-by-one. Choose which videos you want to transcribe. Check out and pay. SpeakerText sends you an email when your transcripts are finished. Download your transcripts as text or XML files from SpeakerText OR install CaptionBox and download your transcripts as HTML code to place on your website. Guarantee We guarantee that your transcripts will get back to you in less than 72 hours and be of the highest quality. Give it a try now! http://speakertext.com

This is the first I've heard of a tool like this doing a creditable job. I suspect there is some machine transcription going on, but then the first pass is sent to India or Jamaica to be polished. Here's an example of how the NYTimes used this tool: http://www.nytimes.com/interactive/2009/01/20/us/politics/20090120_INAUGURAL_ANALYSIS.html

FusionTablesLayer Builder This wizard helps you create the HTML for a map with a FusionTablesLayer and search element (either text-based search or select menu). After creating your map, you can copy and paste the HTML code in the textarea below to display the map on your own website! Please submit bug reports here: Issue Tracker

Click on the "Add another feature" drop-down to add additional layer or search box