Group items tagged

Blog Learning to love…grep (let the computer search text for you) Posted by Michelle Minkoff on Aug 9, 2012 in Blog, Uncategorized | No Comments I’ve gotten into the habit of posting daily learnings on Twitter, but some things require a more in-depth reminder. I also haven’t done as much paying as forward as I’d like (but I’m having a TON of fun!  and dealing with health problems!  but mostly fun!) I’d like to try to start posting more helpful tips here, partially as a notebook for myself, and partially to help others with similar issues. Today’s problem: I needed to search for a few lines of text, which could be contained in any one of nine files with 100,000 lines each. Opening all of the files took a very long time on my computer, not to mention executing a search. Enter the “grep” command in Terminal, that allows you to quickly search files using the power of the computer.

INFOdocket Information Industry News + New Web Sites and Tools From Gary Price and Shirl Kennedy National Science Foundation Helps Fund scrible, A New Web Annotation Tool/Personal Web Cache + Video Demo Posted on May 12, 2011 by Gary D. Price scrible (pronounced scribble) launched about a week ago and you can learn more (free to register and use) here. The company has received a $500,000 grant from the National Science Foundation. From Venture Beat: The company lets users do three things: Save articles and pages so they’re available if the original goes offline; richly annotate online content using tools reminiscent of Word (highlighter, sticky note, etc.), and share annotated pages privately with others. scrible is free and will continue to be free to all users (125MB of storage space). A premium edition is also planned but features (aside from a larger storage quota) have not been announced. Robert Scoble has posted a video demo of scrible with the CEO of of the company, Victor Karkar, doing the “driving.” scrible sounds a lot like Diigo without the mobile access options. It also sounds similar (minus the markup features) to Pinboard. Pinboard does charge $9.97 for a lifetime membership with almost all features (there are many with new ones are debut regularly). For an extra $25/year all of the material you’ve bookmarked is cached by Pinboard. Cached pages look great INCLUDING PDF files. Pinboard is extremely fast and has a very low learning curve. Think Delicious and then add a ton of useful tools to it. Pinboard also provides mobile access to your saved bookmarks and cached documents. Finally, when used responsibly (aka abused) there are no storage space quotas. Which service do you prefer or does each service have a niche depending on the work you’re doing? What other tools to you use? Hat Tips and Thanks: @NspireD2 and @New Media Consortium Share this: Share Share Tagged: Annotation Tools, Diigo, Pinboard, scrible Posted in: Personal Archiving, Web To

INFOdocket Information Industry News + New Web Sites and Tools From Gary Price and Shirl Kennedy National Science Foundation Helps Fund scrible, A New Web Annotation Tool/Personal Web Cache + Video Demo Posted on May 12, 2011 by Gary D. Price scrible (pronounced scribble) launched about a week ago and you can learn more (free to register and use) here. The company has received a $500,000 grant from the National Science Foundation. From Venture Beat: The company lets users do three things: Save articles and pages so they’re available if the original goes offline; richly annotate online content using tools reminiscent of Word (highlighter, sticky note, etc.), and share annotated pages privately with others. scrible is free and will continue to be free to all users (125MB of storage space). A premium edition is also planned but features (aside from a larger storage quota) have not been announced. Robert Scoble has posted a video demo of scrible with the CEO of of the company, Victor Karkar, doing the “driving.” scrible sounds a lot like Diigo without the mobile access options. It also sounds similar (minus the markup features) to Pinboard. Pinboard does charge $9.97 for a lifetime membership with almost all features (there are many with new ones are debut regularly). For an extra $25/year all of the material you’ve bookmarked is cached by Pinboard. Cached pages look great INCLUDING PDF files. Pinboard is extremely fast and has a very low learning curve. Think Delicious and then add a ton of useful tools to it. Pinboard also provides mobile access to your saved bookmarks and cached documents. Finally, when used responsibly (aka abused) there are no storage space quotas. Which service do you prefer or does each service have a niche depending on the work you’re doing? What other tools to you use? Hat Tips and Thanks: @NspireD2 and @New Media Consortium Share this: Share Share Tagged: Annotation Tools, Diigo, Pinboard, scrible Posted in: Personal Archiving, Web Tools

TransparencyCamp '11 Recap Nicole Aro May 4, 2011, 11:28 a.m. Sunlight’s fourth TransparencyCamp was this past weekend, and I’d like to take this moment to say to all of our attendees: Thank you -- you guys rock. To everyone else, I’m sorry that you missed such an awesome weekend, but we hope to see you next time around! This weekend was made possible by the generosity of our sponsors: Microsoft, Google, O’Reilly, Governing, iStrategyLabs, Forum One, and Adobe. I’d like to say a special thank you to Patrick Svenburg of Microsoft who stayed late to make sure we could finish setup and even helped us carry supplies(!). The weekend brought together about 250 government workers, software developers, investigative journalists, bloggers, students and open government advocates of all stripes to share stories, build relationships, and plan together to take on the challenges of building more open government. This year, TransparencyCamp also went global, bringing in 22 amazing transparency advocates from around the world to teach, learn and share with us here in the states.

TransparencyCamp '11 Recap Nicole Aro May 4, 2011, 11:28 a.m. Sunlight’s fourth TransparencyCamp was this past weekend, and I’d like to take this moment to say to all of our attendees: Thank you -- you guys rock. To everyone else, I’m sorry that you missed such an awesome weekend, but we hope to see you next time around! This weekend was made possible by the generosity of our sponsors: Microsoft, Google, O’Reilly, Governing, iStrategyLabs, Forum One, and Adobe. I’d like to say a special thank you to Patrick Svenburg of Microsoft who stayed late to make sure we could finish setup and even helped us carry supplies(!). The weekend brought together about 250 government workers, software developers, investigative journalists, bloggers, students and open government advocates of all stripes to share stories, build relationships, and plan together to take on the challenges of building more open government. This year, TransparencyCamp also went global, bringing in 22 amazing transparency advocates from around the world to teach, learn and share with us here in the states.

MapKnitter View What links here Revisions balloon-mapping, kite-mapping, near-infrared-camera, Part of the Grassroots Mapping Curriculum series. Creating a map from a set of aerial photos Learn to turn aerial images into maps with the Public laboratory's open source MapKnitter.org tool. Upload an image, rotate, distort and stretch it onto a reference map, and use the transparency and outline modes to check how close the fit is. Then export to GeoTiff or OpenLayers/TMS.

Understanding how to clean  data is an important skill every reporter needs. Demographic, financial and other data is available on a city, county, state and national level in the United States. But understanding how to take a large data file and distill it into a usable form can be daunting. In this tutorial, you'll learn how spreadsheets work, basic data-cleaning workflow and how to use formulas and functions to clean data. This is a general tutorial and it doesn't delve deeply into one program. We'll use Microsoft Excel but most of the same techniques work in Google Spreadsheets and other programs.

In this piece, I’ll try to demystify JSON so that you can at least recognize it when you come across it. Again, it is just a data format. Reading and understanding JSON doesn’t require programming. But after you see how JSON is used, you’ll realize why it might be worth your while to learn some programming.

gExr is an online tool for editing and testing Regular Expressions (RegExp / RegEx). It provides a simple interface to enter RegEx expressions, and visualize matches in real-time editable source text. It also provides a handy RegExp snippet sidebar with descriptions and usage examples to make it easier to learn Regular Expressions through trial and error. It isn’t as powerful as a product like RegExBuddy, but it has the advantage of being online and free. I will be releasing a free desktop version for Mac OSX and Windows built with AIR in the next day or two. So far this has only taken a day of developmen

Narrative + investigative: tips from IRE 2012, Part 1 At last month’s Investigative Reporters & Editors conference, in Boston, hundreds of reporters attended dozens of sessions on everything from analyzing unstructured data to working with the coolest web tools and building a digital newsroom. The conference, which started in the 1970s, after a Phoenix reporter died in a car bomb while covering the mob, is usually considered an investigative-only playground, but narrative writers can learn a lot from these journalists’ techniques and resources. When might a narrative writer need investigative skills? A few possible scenarios: • When developing a character’s timeline and activities beyond the basic backgrounding • When navigating precarious relationships with sources • When organizing large and potentially complicated amounts of material • When gathering data and documents that might provide storytelling context – geopolitical, financial, etc. We asked This Land correspondent Kiera Feldman to cover the conference with an eye for material that might be particularly useful in narrative. She netted a range of ideas, tips and resources. Today, in Part 1, she covers areas including documents and data, online research and source relationships. Check back tomorrow for Part 2, “Writing the Investigative Story,” with best practices from Ken Armstrong of the Seattle Times and Steve Fainaru of ESPN.

ITA Software is proud to introduce Needlebase™, a revolutionary platform for acquiring, integrating, cleansing, analyzing and publishing data on the web.  Using Needlebase through a web browser, without programmers or DBAs, your data team can easily: acquire data from multiple sources:  A simple tagging process quickly imports structured data from complex websites, XML feeds, and spreadsheets into a unified database of your design. merge, deduplicate and cleanse: Needlebase uses intelligent semantics to help you find and merge variant forms of the same record.  Your merges, edits and deletions persist even after the original data is refreshed from its source. build and publish custom data views: Use Needlebase's visual UI and powerful query language to configure exactly your desired view of the data, whether as a list, table, grid, or map.  Then, with one click, publish the data for others to see, or export a feed of the clean data to your own local database. Needlebase dramatically reduces the time, cost, and expertise needed to build and maintain comprehensive databases of practically anything. Read on to learn more about Needlebase's capabilities and our early adopters' success stories, or watch our tutorial videos. Then sign up to get started!

Welcome to Graphviz  Available translations:Belorussian What is Graphviz? Graphviz is open source graph visualization software. Graph visualization is a way of representing structural information as diagrams of abstract graphs and networks. It has important applications in networking, bioinformatics,  software engineering, database and web design, machine learning, and in visual interfaces for other technical domains.   Features The Graphviz layout programs take descriptions of graphs in a simple text language, and make diagrams in useful formats, such as images and SVG for web pages, PDF or Postscript for inclusion in other documents; or display in an interactive graph browser. (Graphviz also supports GXL, an XML dialect.)  Graphviz has many useful features for concrete diagrams, such as options for colors, fonts, tabular node layouts, line styles, hyperlinks, rolland custom shapes.

Data-drivenjournalism:Whatistheretolearn?Apaperonthedata-drivenjournalismroundtableheldinAmsterdamon24August2010.Withadditionalmateri class=