Group items tagged

What is it? A social analytics platform which contains over 1,000 days of tweets (all 70 billion of them), Facebook activity and blog posts. How is it of use to journalists? "Journalists can easily develop real-time insights into any story from Playground," PeopleBrowsr UK CEO Andrew Grill explains. Complex keyword searches can be divided by user influence, geolocation, sentiment, and virtual communities of people with shared interests and affinities. These features - and many more - let reporters and researchers easily drill down to find the people and content driving the conversation on social networks on any subject. Playground lets you use the data the way you want to use it. You can either export the graphs and tables that the site produces automatically or export the results in a CSV file to create your own visualisations, which could potentially make it the next favourite tool of data journalists. Grill added: The recent launch of our fully transparent Kred influencer platform will make it faster and easier for journalists to find key influencers in a particular community. You can give Playground a try for the first 14 days before signing up for one of their subscriptions ($19 a month for students and journalists, $149 for organisations and companies).

Mining of Massive Datasets The book has now been published by Cambridge University Press. A hardcopy can be obtained Here. By agreement with the publisher, you can still download it free from this page. Cambridge Press does, however, retain copyright on the work, and we expect that you will acknowledge our authorship if you republish parts or all of it. We are sorry to have to mention this point, but we have evidence that other items we have published on the Web have been appropriated and republished under other names. It is easy to detect such misuse, by the way, as you will learn in Chapter 3. --- Anand Rajaraman (@anand_raj) and Jeff Ullman Downloads Download the Complete Book (340 pages, approximately 2MB) Download chapters of the book: Preface and Table of Contents Chapter 1 Data Mining Chapter 2 Large-Scale File Systems and Map-Reduce Chapter 3 Finding Similar Items Chapter 4 Mining Data Streams Chapter 5 Link Analysis Chapter 6 Frequent Itemsets Chapter 7 Clustering Chapter 8 Advertising on the Web Chapter 9 Recommendation Systems Index

When you go to a conference, there are typically several talks going on at the same time, and you can always tell there's a popular paper coming up when you see people leave a bunch of rooms at once and head straight into one. There's also the unfortunate case when someone speaks, and there's only a handful of people in the room, all in the back staring at their laptops. Open Data City visualized this activity during the German internet conference re: publica. Open Data City used MAC addresses and access point connections to keep track of where devices went. So a person might be in a room connected to the nearest access point, disconnects as he leaves, and then reconnects as he reenters another room, which provides the flow. It's fun to watch the conference play out even if you didn't attend. Each dot represents an attendee, and as the animation plays the dots migrate from room to room. Click and drag over the dots to select specific people. [Thanks, Michael]

How to make searchable, Web-based Google charts Michelle Minkoff by Michelle Minkoff Published June 3, 2011 12:01 am Updated June 2, 2011 10:22 pm A lot of data visualization requires the technical expertise of a programmer and skills that take time and resources to develop. A rise in free tools, however, has made it easier to make interactive graphs in charts, whether you're a designer, developer, Web producer or hobbyist. The Google Visualization API, for instance, gives you options without making the work too complicated. I've created a tutorial below to help you make simple, Web-based Google charts. (You can click on any of the screenshots to go to a larger version.) In the first example, we'll craft an interactive bar chart that compares the numbers of tornado-related deaths in the United States throughout the past four years. We'll use data from the National Oceanic and Atmospheric Administration (NOAA), which can be found here. (You can download a cleaned version of this data here, formatted as a comma-delimited file, CSV.) http://www.poynter.org/how-tos/newsgathering-storytelling/126595/how-to-make-simple-web-based-google-charts

Download Download your Esri Maps for Office. Esri Maps for Office requires Office 2010 or later. Esri Maps for Office Add-In (x86) 32-bit (70MB) Esri Maps for Office Add-In (x64) 64-bit (70MB) Download the version of Esri Maps for Office that matches the bit version of Microsoft Office 2010 you have installed, not the version of your operating system (OS). If you are not sure, open Excel (or other Office application), click the File tab, and select Help. In the About Microsoft Excel section on the right, the version information states whether the Microsoft Office 2010 installation is 32-bit or 64-bit. Once your download is complete, open Excel.

This looks to be a very valuable tool.

Solving the Pie Chris Gemignani December 14, 2006 By: Chris Gemignani Last week I challenged you to reproduce this alternative to pie charts in Excel. I promised a screencast to show how it's done. http://juiceanalytics.com/writing/2006/12/square-pie-screencast/ Square Pie Eighteen people answered the call with nearly three dozen different solutions. Click here to watch the screencast showing how to accomplish the two most popular solutions; filling cells with conditional formatting and pushing the column chart to extremes. If you want to look at the source,Clint Ivy produced an excellent version of the cell filling approach.

Listserv archives IRE and NICAR offer several opportunities for members and even non-members to exchange ideas, information, techniques and war stories. Joining is easy. If you are an IRE member, you may view the list archives: * Click an archive link and login with any e-mail address on record with the IRE office. Click "Get Password" if your first visit, to receive your LISTSERV password (separate from the IRE website password). Most users will login with the e-mail used for their IRE login account. Please e-mail listmaster@ire.org if you need help or have any questions. IRE-L archives. NICAR-L archives. IREPLUS-L archives. CENSUS-L archives. The following lists are less active: CFIC-L archives IRE-EDU-L archives IREBC-L archives

Tom Torok (NYT) writes: After years of looking down my nose at Excel because of its limitations, I have to say that I'm very impressed with Excel 2010 when used with a free Microsoft add-in called PowerPivot. http://office.microsoft.com/en-us/excel/download-powerpivot-HA101959985.aspx In a PowerPivot tutorial (link below), I imported eight tables  from several sources and joined them - yes, you can join relational data. It uses some magical data compression that allows for lightning fast sorts, filters and calculated fields. The largest table in the tutorial has about 2 million rows. A calculated field on that table took seconds. A did a pivot table on the table and the answers appeared as soon as I selected the fields. In one of  the training videos (http://www.powerpivot.com/) an MS guy works with a 101 million-record table on his laptop. It's really amazing. http://powerpivotsdr.codeplex.com/ If you install, be sure to read the prerequisites or you'll be installing and uninstalling both PowerPivot and Excel. I'm running it on a 32-bit XP machine (it won't run on a 64-bit XP but will work on Windows 7 64-bit). The tutorial is for a Windows 7 setup, but there are items in the menu bar that match the reference to the tutorial's ribbon. I noticed that if I call up an xlsx by double clicking on a file in Windows Explorer that PowerPivot is not enabled in the ribbon. If you call up a file from within Excel 2010 everything works as advertised.Regards, TT  

What is BuzzData? Data should be free-flowing, well-organized and easy to share. Wouldn't it be nice if there was a place where you could store, share and show off your data with just a couple of mouse clicks? BuzzData lets you publish your data in a smarter, easier way. Instead of juggling versions and overwriting files, use BuzzData and enjoy a social network designed for data."

A bold experiment in distributed education, "Introduction to Databases" will be offered free and online to students worldwide during the fall of 2011. Students will have access to lecture videos, receive regular feedback on progress, and receive answers to questions. When you successfully complete this class, you will also receive a statement of accomplishment. Taught by Professor Jennifer Widom, the curriculum draws from Stanford's popular Introduction to Databases course. A syllabus and more information is available here. Sign up below to receive additional information about participating in the online version when it becomes available.

FusionTablesLayer Builder This wizard helps you create the HTML for a map with a FusionTablesLayer and search element (either text-based search or select menu). After creating your map, you can copy and paste the HTML code in the textarea below to display the map on your own website! Please submit bug reports here: Issue Tracker

Click on the "Add another feature" drop-down to add additional layer or search box

An Applied Demography Toolbox A collection of applied demography programs, scripts, spreadsheets and databases. If you have any questions (such as how to apply the tools to your own work), recommendations or additions, you can send a message to me (Eddie Hunsinger) at edynivn@gmail.com. If you would like to use, share or reproduce any information or ideas from the linked files, be sure to cite the respective source. Here is a neat article that gives this site some inspiration. Acknowledgments. Subscribe to new postings . Return to Eddie's homepage. http://www.demog.berkeley.edu/~eddieh/toolbox.html#MedianCalculator

These tend to be US-centric, but there are universal tools here for statistical analysis.

Zanran helps you to find 'semi-structured' data on the web. This is the numerical data that people have presented as graphs and tables and charts. For example, the data could be a graph in a PDF report, or a table in an Excel spreadsheet, or a barchart shown as an image in an HTML page. This huge amount of information can be difficult to find using conventional search engines, which are focused primarily on finding text rather than graphs, tables and bar charts. Put more simply: Zanran is Google for data. Language. English only please... for now. Phrase search. You can use double quotes to make phrases (e.g. "mobile phones"). Booleans. You can use a plus '+' to make a word mandatory, or a minus '-' to exclude it (e.g. +gas -oil production) Vocabulary. We have only limited synonyms - please try different words in your query. And we don't spell-check ... yet.

OpenData Open Data

Get TargetMap Premium for 7 days! Happy Mapping! We have reached 10 Million visitors... and we keep evolving! Targetmap has released a new Territory Manager and we're giving you 7 days Premium use in Targetmap for FREE to celebrate it. Territory Manager will let you: - See on the Map how your clients and Sales Reps are distributed - Aling and optimize your sales force potentials by Postal Codes (improving the customer service and saving expenses) - See and assign your Franchise's coverage area and evaluate their potential. - Export the Map to your presentations and the final Data to your CRM or Call Center. Maybe you want to give us a bit of your time and watch this video (it lasts 15 minutes... But there are so many features!) http://targetmap.blogspot.com/2011/09/get-targetmap-premium-for-7-days.html

Part 2 of the Open Data, Open Society report is now available online Posted on September 1, 2011 by marco Open Data, Open Society is a research project about openness of public data in EU local administrations by for the Laboratory of Economics and Management of Scuola Superiore Sant'Anna, Pisa. The first report of the project, released in October 2010 under a Creative Commons cc-by license, can be downloaded from the website of the DIME project (PDF) or read online as one HTML file on the Sant'Anna School website (*). The conclusions of the project, a shorter report titled "Open Data: Emerging trends, issues and best practices" and finished in June 2011, are now available online under the same license at the following locations: single HTML file PDF format, Sant'Anna school PDF format, DIME website Another part of the project, the Open Data, Open Society survey has been extended until the end of 2011. Thank you in advance for announcing the survey to all the city and regional administrations of EU-15 and, if you want, to add further translations of its introduction!

Shape to Fusion Hi. This website lets you import a shapefile to Google Fusion Tables. This blog post has some details on how it was built. (http://porcupinealley.com/2010/dec/20/shape-escape/) To continue, you will need to authorize this site to access your Fusion Tables data on your behalf. Site by Josh Livni. Source code available at http://code.google.com/p/shpescape/

Again, US-centric, but this might generate some ideas of what could be accomplish in your city/nation. Late yesterday we announced a bunch of new features for Influence Explorer: http://sunlightlabs.com/blog/2011/ie-corporate-accountability/ As the blog post explains, you can now find information about a corporation's EPA violations, federal advisory committee memberships, and participation in the rulemaking process -- all in one place. I wanted to highlight that last feature a bit more, though. To my knowledge, this is the first time that the full corpus of public comments submitted to regulations.gov has been available for bulk download and analysis. This isn't a coincidence: regulations.gov is built using technologies that make scraping it unusually difficult. This is unfortunate, since everyone seems to agree that federal rulemakings are gaining in importance -- both because of congressional gridlock that leaves the regulatory process as a second-best option, and because of calls to simplify the regulatory landscape as a pro-growth measure. It's an area where influence is certainly exerted -- rulemakers are obliged to review every comment -- but little attention is paid to who's flooding dockets with comments, and which directions rules are being pushed. It's taken us several months to develop a reliable solution and to obtain past rulemakings, but we now have the data in hand. We plan to do much more with this dataset, and we're hoping that others will want to dig in, too. You can find a link to the bulk download options in the post above -- the full compressed archive of extracted text and metadata is ~16GB, but we've provided options for grabbing individual agencies' or dockets' data. If anyone wants the original documents (PDFs, DOCs, etc) we can talk through how to make that happen, but as they clock in at 1.5TB we'll want to make sure folks know what they're getting into before we spend the time and bandwidth. Finally, note that we currently o

Matthew Ericson « Illustrator MultiExporter script: Now with JPG and EPS When Maps Shouldn't Be Maps View full interactive map on nytimes.com » Often, when you get data that is organized by geography - say, for example, food stamp rates in every county, high school graduation rates in every state, election results in every House district, racial and ethnic distributions in each census tract - the impulse is since the data CAN be mapped, the best way to present the data MUST be a map. You plug the data into ArcView, join it up with a shapefile, export to Illustrator, clean up the styles and voilà! Instant graphic ready to be published. And in many cases, that's the right call.

"Data docs is a video platform that allows filmmakers and journalist to combine elements from the web, such as interactive graphics, text and scraped information, with linear media, such as video and audio. Having worked in video both in long-form documentary and web video, we understand the power of visual media. Videos are powerful vehicles that we can use to tell personable or explanatory immersive stories. But one of the drawbacks of video as a medium is that they are finished products, which, after they have been published, become outdated fairly quickly. Advances in technology and data bases has allowed for data to be more flexible than video. Data visualizations and interactive infographics, for instance, can be up-to-date at any moment in time if they are hooked up to the right data bases. Think of charts of stock markets that updated every millisecond because APIs or other technological mechanisms feed them live data. We wanted to combine those two worlds - the world of immersive video storyelling and that of live and constantly updated data. This is why we created Data Docs. Through the Data Docs code library filmmakers and developers can 'hook up' their video to live data and other up-to-date information from the web. The library also allows you to integrate your own interactives with specific fonts and styles into your video. It enables you to project HTML, CSS and JavaScript-based graphics on your video. This helps you make videos that will never be out of date or, in other words, to make videos that are evergreen."

"ReConstitution 2012, a fun experiment by Sosolimited, processes transcripts from the presidential debates, and recreates them with animated words and charts. Part data visualization, part experimental typography, ReConstitution 2012 is a live web app linked to the US Presidential Debates. During and after the three debates, language used by the candidates generates a live graphical map of the events. Algorithms track the psychological states of Romney and Obama and compare them to past candidates. The app allows the user to get beyond the punditry and discover the hidden meaning in the words chosen by the candidates. As you let the transcript run, numbers followed by their units (like "18 months") flash on the screen, and trigger words for emotions like positivity, negativity, and rage are highlighted yellow, blue, and red, respectively. You can also see the classifications in graph form. There are a handful of less straightforward text classifications for truthy and suicidal, which are based on linguistic studies, which in turn are based on word frequencies. These estimates are more fuzzy. So, as the creators suggest, it's best not to interpret the project as an analytical tool, and more of a fun way to look back at the debate, which it is. It's pretty fun to watch. Here's a short video from Sosolimited for more on how the application works: "