Group items tagged

Constructing the Open Data Landscape Posted on September 7, 2011 by Nicola Hughes In an article in today's Telegraph regarding Francis Maude's Public Data Corporation, Michael Cross asks: "What makes the state think it can be at the cutting edge of the knowledge economy". He writes in terms of market and business share, giving the example of the satnav market worth over $100bn a year yet it's based on free data from the US Government's GPS system. He credits the internet revolution for transforming public sector data into 'cashable proposition'. We, along with many other start-ups, foundations and civic coding groups, are part of this 'geeky world' of Open Data. So we'd like to add our piece concerning the Open Data movement. Michael has the right to ask this question because there is this constant custodial battle being fought every day, every scrape and every script on the web for the rights to data. So let me tell you about the geeks' take on Open Data.

Data journalism at the Guardian: what is it and how do we do it? Simon Rogers: Our 10 point guide to data journalism and how it's changing Share reddit this Comments (2) Data abstract Data journalism. What is it and how is it changing? Photograph: Alamy Here's an interesting thing: data journalism is becoming part of the establishment. Not in an Oxbridge elite kind of way (although here's some data on that) but in the way it is becoming the industry standard. Two years ago, when we launched the Datablog, all this was new. People still asked if getting stories from data was really journalism and not everyone had seen Adrian Holovaty's riposte. But once you've had MPs expenses and Wikileaks, the startling thing is that no-one asks those questions anymore. Instead, they want to know, "how do we do it?"

ITA Software is proud to introduce Needlebase™, a revolutionary platform for acquiring, integrating, cleansing, analyzing and publishing data on the web. Using Needlebase through a web browser, without programmers or DBAs, your data team can easily: acquire data from multiple sources: A simple tagging process quickly imports structured data from complex websites, XML feeds, and spreadsheets into a unified database of your design. merge, deduplicate and cleanse: Needlebase uses intelligent semantics to help you find and merge variant forms of the same record. Your merges, edits and deletions persist even after the original data is refreshed from its source. build and publish custom data views: Use Needlebase's visual UI and powerful query language to configure exactly your desired view of the data, whether as a list, table, grid, or map. Then, with one click, publish the data for others to see, or export a feed of the clean data to your own local database. Needlebase dramatically reduces the time, cost, and expertise needed to build and maintain comprehensive databases of practically anything. Read on to learn more about Needlebase's capabilities and our early adopters' success stories, or watch our tutorial videos. Then sign up to get started! http://needlebase.com

DIVA-GIS DIVA-GIS is a free computer program for mapping and geographic data analysis (a geographic information system (GIS). With DIVA-GIS you can make maps of the world, or of a very small area, using, for example, state boundaries, rivers, a satellite image, and the locations of sites where an animal species was observed. We also provide free spatial data for the whole world that you can use in DIVA-GIS or other programs. You can use the discussion forum to ask questions, report problems, or make suggestions. Or contact us, and read the blog entries for the latest news. But first download the program and read the documentation. DIVA-GIS is particularly useful for mapping and analyzing biodiversity data, such as the distribution of species, or other 'point-distributions'. It reads and write standard data formats such as ESRI shapefiles, so interoperability is not a problem. DIVA-GIS runs on Windows and (with minor effort) on Mac OSX (see instructions). You can use the program to analyze data, for example by making grid (raster) maps of the distribution of biological diversity, to find areas that have high, low, or complementary levels of diversity. And you can also map and query climate data. You can predict species distributions using the BIOCLIM or DOMAIN models.

DIVA-GIS DIVA-GIS is a free computer program for mapping and geographic data analysis (a geographic information system (GIS). With DIVA-GIS you can make maps of the world, or of a very small area, using, for example, state boundaries, rivers, a satellite image, and the locations of sites where an animal species was observed. We also provide free spatial data for the whole world that you can use in DIVA-GIS or other programs. You can use the discussion forum to ask questions, report problems, or make suggestions. Or contact us, and read the blog entries for the latest news. But first download the program and read the documentation. DIVA-GIS is particularly useful for mapping and analyzing biodiversity data, such as the distribution of species, or other 'point-distributions'. It reads and write standard data formats such as ESRI shapefiles, so interoperability is not a problem. DIVA-GIS runs on Windows and (with minor effort) on Mac OSX (see instructions). You can use the program to analyze data, for example by making grid (raster) maps of the distribution of biological diversity, to find areas that have high, low, or complementary levels of diversity. And you can also map and query climate data. You can predict species distributions using the BIOCLIM or DOMAIN models.

"The Open Data Handbook This handbook discusses the legal, social and technical aspects of open data. It can be used by anyone but is especially designed for those seeking to open up data. It discusses the why, what and how of open data - why to go open, what open is, and the how to 'open' data. To get started, you may wish to look at the Introduction. You can navigate through the report using the Table of Contents (see sidebar or below). We warmly welcome comments on the text and will incorporate feedback as we go forward. We also welcome contributions or suggestions for additional sections and areas to examine."

Data-Driven Journalism Workshop on EU Spending: Tools & Techniques. Utrecht, 8th-9th September. Posted on August 9, 2011 by Lucy Chambers The following post is by Liliana Bonegru, Project Coordinator at the European Journalism Centre (EJC), and Lucy Chambers, Community Coordinator at the Open Knowledge Foundation. The post announces a joint workshop between the EJC and OKF, focusing on how to get started with data-driven reporting on spending data. This workshop will focus particularly on EU spending data. Interested in data-driven journalism and EU spending? The European Journalism Centre together with the Open Knowledge Foundation is hosting a one and a half day data-driven journalism workshop on EU spending in Utrecht, the Netherlands on 8th-9th September.

Data Visualization Platform, Weave, Now Open Source Logo Open Source Initiative Image via Wikipedia Civic Commons, Contributors (Karl Fogel, Author) With more and more civic data becoming available and accessible, the challenge grows for policy makers and citizens to leverage that data for better decision-making. It is often difficult to understand context and perform analysis. "Weave", however, helps. A web-based data visualization tool, Weave enables users to explore, analyze, visualize and disseminate data online from any location at any time. We saw tremendous potential in the platform and have been helping open-source the software, advising on community engagement strategy and licensing. This week, we were excited to see the soft launch of the Weave 1.0 Beta, which went open-source on Wednesday, June 15. Weave is the result of a broad partnership: it was developed by the Institute for Visualization and Perception Research at the University of Massachussetts Lowell, with support from the Open Indicators Consortium, which is made up of over ten municipal, regional, and state member organizations. This consortium will probably expand now that Weave is open source, leading hopefully to greater collaboration, more development, and further innovation on this important platform. Early-adopter data geeks should give it a spin. One of Weave's key features is high-speed interactivity and responsiveness, which is somewhat unusual in web-based visualization software; try out the demo sites or watch the video below. Our congratulations and thanks to the Weave team! As city management is increasingly data-driven, so data analysis and visualization tools will continue to be an important part of every city manager's toolkit. We are excited to see this evolving toolkit enter the civic commons. http://govinthelab.com/data-visualization-platform-weave-now-open-source

Sharing data on public services could have serious consequences unless the material has been valued, maintained and protected and the original reasons for its collection have been taken into account, the Information Society Alliance (Eurim), has warned. In a report on the quality of public sector information, the group says that the drive to put central and local government data online, open to public scrutiny, has revealed the long standing problems with quality that lie behind the reluctance of some departments and agencies to trust one another's data. It adds that it is important that decisions on spending cuts are based on good quality information.

An important article. Please read.

City of Ottawa moves to formalize open data program http://www.itworldcanada.com/news/city-of-ottawa-moves-to-formalize-open-data-program/143323?sub=569517&utm_source=569517&utm_medium=top5&utm_campaign=TD+

Open Data Cook BookMaking Open Data Accessible for EveryoneAbout the Cook BookThe open data cook book is collecting recipes for ways to find and use open data, particularly open data of social value - such as open government data, or open data for campaigners and charities. Working with data can seem scary. But it doesn't have to be. There are many different ways to make data useful - and lots of different gadgets to help you. Take a look at the growing list of cook book recipes to find simple step by step ideas for making use of open data. RecipesYou can find a list of the recipes so far here. Drafts, ideas and notesIn the cooks notebook you can find draft notes on using different datasets and sketches that might develop into recipes in future. Get InvolvedFind out how to get involved here or jump right in and create a recipe. Tweet with the #opendatacookbook tag, or bookmark content on del.icio.us 'opendatacookbook ' to share with the project. Join the mailing list to discuss developments. UpdateAfter a brief experiment with Drupal as a CMS for the cook book - we've switched to DokuWiki for a bit to make compiling a list of recipes a lot easier before we work out the best way to run the Cook Book.

"Data docs is a video platform that allows filmmakers and journalist to combine elements from the web, such as interactive graphics, text and scraped information, with linear media, such as video and audio. Having worked in video both in long-form documentary and web video, we understand the power of visual media. Videos are powerful vehicles that we can use to tell personable or explanatory immersive stories. But one of the drawbacks of video as a medium is that they are finished products, which, after they have been published, become outdated fairly quickly. Advances in technology and data bases has allowed for data to be more flexible than video. Data visualizations and interactive infographics, for instance, can be up-to-date at any moment in time if they are hooked up to the right data bases. Think of charts of stock markets that updated every millisecond because APIs or other technological mechanisms feed them live data. We wanted to combine those two worlds - the world of immersive video storyelling and that of live and constantly updated data. This is why we created Data Docs. Through the Data Docs code library filmmakers and developers can 'hook up' their video to live data and other up-to-date information from the web. The library also allows you to integrate your own interactives with specific fonts and styles into your video. It enables you to project HTML, CSS and JavaScript-based graphics on your video. This helps you make videos that will never be out of date or, in other words, to make videos that are evergreen."

Part 2 of the Open Data, Open Society report is now available online Posted on September 1, 2011 by marco Open Data, Open Society is a research project about openness of public data in EU local administrations by for the Laboratory of Economics and Management of Scuola Superiore Sant'Anna, Pisa. The first report of the project, released in October 2010 under a Creative Commons cc-by license, can be downloaded from the website of the DIME project (PDF) or read online as one HTML file on the Sant'Anna School website (*). The conclusions of the project, a shorter report titled "Open Data: Emerging trends, issues and best practices" and finished in June 2011, are now available online under the same license at the following locations: single HTML file PDF format, Sant'Anna school PDF format, DIME website Another part of the project, the Open Data, Open Society survey has been extended until the end of 2011. Thank you in advance for announcing the survey to all the city and regional administrations of EU-15 and, if you want, to add further translations of its introduction!

Human rights groups collect data containing details of human right abuses from various sources, including medical records, newspaper articles, witness testimonies, letters, interviews, and official reports and documents. Analyzer can be used to capture this data for analysis. Data is coded according to the "Who did what to whom" model and entered into the capture set of Analyzer. Data about the source of the information is entered in the source tab, shown in Figure 1. (Note: the data in the figures shown here, unless otherwise indicated, are from a sample, not an actual, dataset.)

Understanding how to clean data is an important skill every reporter needs. Demographic, financial and other data is available on a city, county, state and national level in the United States. But understanding how to take a large data file and distill it into a usable form can be daunting. In this tutorial, you'll learn how spreadsheets work, basic data-cleaning workflow and how to use formulas and functions to clean data. This is a general tutorial and it doesn't delve deeply into one program. We'll use Microsoft Excel but most of the same techniques work in Google Spreadsheets and other programs.

Narrative + investigative: tips from IRE 2012, Part 1 At last month's Investigative Reporters & Editors conference, in Boston, hundreds of reporters attended dozens of sessions on everything from analyzing unstructured data to working with the coolest web tools and building a digital newsroom. The conference, which started in the 1970s, after a Phoenix reporter died in a car bomb while covering the mob, is usually considered an investigative-only playground, but narrative writers can learn a lot from these journalists' techniques and resources. When might a narrative writer need investigative skills? A few possible scenarios: * When developing a character's timeline and activities beyond the basic backgrounding * When navigating precarious relationships with sources * When organizing large and potentially complicated amounts of material * When gathering data and documents that might provide storytelling context - geopolitical, financial, etc. We asked This Land correspondent Kiera Feldman to cover the conference with an eye for material that might be particularly useful in narrative. She netted a range of ideas, tips and resources. Today, in Part 1, she covers areas including documents and data, online research and source relationships. Check back tomorrow for Part 2, "Writing the Investigative Story," with best practices from Ken Armstrong of the Seattle Times and Steve Fainaru of ESPN.

Make it easy for your organization to publish and manage public data You can achieve your organization's transparency goals, cost-effectively, by streamlining the data publishing process and automating maintenance and updates. Internal stakeholders, in any department or agency, with little or no technical assistance, become first-class data publishers. While administrators manage the organization's data in one central location, offer constituents a consistent and privately-branded online experience and get real-time data consumption and citizen engagement metrics.

http://www.zanran.com/ a search engine for data & statistics. Time to open your data, people! #opendata

Data Without Borders seeks to match non-profits in need of data analysis with freelance and pro bono data scientists who can work to help them with data collection, analysis, visualization, or decision support.

A good resource to extend the intellectual power and reach of your newsroom.

They Rule Overview They Rule aims to provide a glimpse of some of the relationships of the US ruling class. It takes as its focus the boards of some of the most powerful U.S. companies, which share many of the same directors. Some individuals sit on 5, 6 or 7 of the top 1000 companies. It allows users to browse through these interlocking directories and run searches on the boards and companies. A user can save a map of connections complete with their annotations and email links to these maps to others. They Rule is a starting point for research about these powerful individuals and corporations. Context A few companies control much of the economy and oligopolies exert control in nearly every sector of the economy. The people who head up these companies swap on and off the boards from one company to another, and in and out of government committees and positions. These people run the most powerful institutions on the planet, and we have almost no say in who they are. This is not a conspiracy, they are proud to rule, yet these connections of power are not always visible to the public eye. Karl Marx once called this ruling class a 'band of hostile brothers.' They stand against each other in the competitve struggle for the continued accumulation of their capital, but they stand together as a family supporting their interests in perpetuating the profit system as whole. Protecting this system can require the cover of a 'legitimate' force - and this is the role that is played by the state. An understanding of this system can not be gleaned from looking at the inter-personal relations of this class alone, but rather how they stand in relation to other classes in society. Hopefully They Rule will raise larger questions about the structure of our society and in whose benefit it is run. The Data We do not claim that this data is 100% accurate at all times. Corporate directors have a habit of dying, quitting boards, joining new ones and most frustratingly passing on their name

I think this data must be very useful to the people in Occupy Wall Street

AfDB Launches the Open Data for Africa Platform The African Development Bank Group (AfDB), in partnership with Knoema, has launched an Open Data for Africa platform aimed at significantly increasing access to quality data necessary for managing and monitoring development results in African countries, including the MDGs. The platform will also serve as a knowledge center for collecting, accessing… View all Introduction