Skip to main content

Home/ Open Web/ Group items tagged SQL

Rss Feed Group items tagged

Gary Edwards

Google Launches Cloud SQL API To Allow Developers To Manage Their Databases Programmati... - 0 views

  •  
    "Google's Cloud Platform has long featured Cloud SQL, a zero-maintenance MySQL database that's hosted on Google's cloud platform. What it didn't offer was an API to easily manage these databases without having to use Google's admin interface. Today, however, Google is launching the Cloud SQL API. This new REST API will allow developers to programmatically manage their database instances and open a number of new use cases for Cloud SQL. The API, which Google still deems to be experimental, will allow developers to create their own workflows to easily create and delete instances, restart them and restore them from backup. They will also be able to use it to important and export their databases to and from Google Cloud Storage. For developers, this means using Google's cloud database is now quite a bit easier, especially if they need to regularly manage multiple databases for their customers. Google's launch partner for this API is OrangeScape, which uses it to power parts of KiSSFLOW, its Google Apps workflow SaaS service. "
Gary Edwards

Matt On Stuff: Hadoop For The Rest Of Us - 0 views

  •  
    Excellent Hadoop/Hive explanation.  Hat tip to Matt Asay for the link.  I eft a comment on Matt's blog questioning the consequences of the Oracle vs. Google Android lawsuit, and the possible enforcement of the Java API copyright claim against Hadoop/Hive.  Based on this explanation of Hadoop/Hive, i'm wondering if Oracle is making a move to claim the entire era of Big Data Cloud Computing?  To understand why, it's first necessary to read Matt the Hadoople's explanation.   kill shot excerpt: "You've built your Hadoop job, and have successfully processed the data. You've generated some structured output, and that resides on HDFS. Naturally you want to run some reports, so you load your data into a MySQL or an Oracle database. Problem is, the data is large. In fact it's so large that when you try to run a query against the table you've just created, your database begins to cry. If you listen to its sobs, you'll probably hear "I was built to process Megabytes, maybe Gigabytes of data. Not Terabytes. Not Perabytes. That's not my job. I was built in the 80's and 90's, back when floppy drives were used. Just leave me alone". "This is where Hive comes to the rescue. Hive lets you run an SQL statement against structured data stored on HDFS. When you issue an SQL query, it parses it, and translates it into a Java Map/Reduce job, which is then executed on your data. Although Hive does some optimizations, in general it just goes record by record against all your data. This means that it's relatively slow - a typical Hive query takes 5 or 10 minutes to complete, depending on how much data you have. However, that's what makes it effective. Unlike a relational database, you don't waste time on query optimization, adding indexes, etc. Instead, what keeps the processing time down is the fact that the query is run on all machines in your Hadoop cluster, and the scalability is taken care of for you." "Hive is extremely useful in data-warehousing kind of scenarios. You would
Gary Edwards

Google Launches Dart Programming Language - Development - Web Development - Information... - 1 views

  •  
    Google releases JavaScript alternative Web application programming language.  Release includes Cloud SQL, a cloud computing database to write Web apps against - using either JavaScript or DART. excerpt: Google on Monday introduced a preview version of Dart, its new programming language for Web applications. The introduction was widely expected, not only because the announcement was listed on the GOTO developer conference schedule, but because a Google engineer described the language and its reason for being in a message sent to a developer mailing list late last year. "The goal of the Dash [Dart's former name] effort is ultimately to replace JavaScript as the lingua franca of Web development on the open Web platform," said Google engineer Mark S. Miller in his post last year. More Insights White Papers The Dodd-Frank Act: Impact on Derivatives Technology Infrastructure Simple is Better: Overcoming the complexity that robs financial data of its potential Analytics Mobility's Next Challenge: 8 Steps to a Secure Environment SaaS 2011: Adoption Soars, Yet Deployment Concerns Linger Webcasts Effective IT Inventory and Asset Management: From Quagmire to Quick Fix Outsourcing Security: What Every Potential Cloud Security Customer Should Know Videos In an interview at Interop New York, Cisco's Justin Griffin shows how their wireless products can physically map radio sources by analyzing the spectrum. This allows you to detect rogue devices and sources of interference. Lars Bak, a Google engineer who helped develop Chrome's V8 JavaScript engine and one of the creators of Dart, said in a phone interview that Google works regularly on large Web applications and that the company's engineers feel they need a new programming language to describe large, complex Web applications.
Paul Merrell

The People and Tech Behind the Panama Papers - Features - Source: An OpenNews project - 0 views

  • Then we put the data up, but the problem with Solr was it didn’t have a user interface, so we used Project Blacklight, which is open source software normally used by librarians. We used it for the journalists. It’s simple because it allows you to do faceted search—so, for example, you can facet by the folder structure of the leak, by years, by type of file. There were more complex things—it supports queries in regular expressions, so the more advanced users were able to search for documents with a certain pattern of numbers that, for example, passports use. You could also preview and download the documents. ICIJ open-sourced the code of our document processing chain, created by our web developer Matthew Caruana Galizia. We also developed a batch-searching feature. So say you were looking for politicians in your country—you just run it through the system, and you upload your list to Blacklight and you would get a CSV back saying yes, there are matches for these names—not only exact matches, but also matches based on proximity. So you would say “I want Mar Cabra proximity 2” and that would give you “Mar Cabra,” “Mar whatever Cabra,” “Cabra, Mar,”—so that was good, because very quickly journalists were able to see… I have this list of politicians and they are in the data!
  • Last Sunday, April 3, the first stories emerging from the leaked dataset known as the Panama Papers were published by a global partnership of news organizations working in coordination with the International Consortium of Investigative Journalists, or ICIJ. As we begin the second week of reporting on the leak, Iceland’s Prime Minister has been forced to resign, Germany has announced plans to end anonymous corporate ownership, governments around the world launched investigations into wealthy citizens’ participation in tax havens, the Russian government announced that the investigation was an anti-Putin propaganda operation, and the Chinese government banned mentions of the leak in Chinese media. As the ICIJ-led consortium prepares for its second major wave of reporting on the Panama Papers, we spoke with Mar Cabra, editor of ICIJ’s Data & Research unit and lead coordinator of the data analysis and infrastructure work behind the leak. In our conversation, Cabra reveals ICIJ’s years-long effort to build a series of secure communication and analysis platforms in support of genuinely global investigative reporting collaborations.
  • For communication, we have the Global I-Hub, which is a platform based on open source software called Oxwall. Oxwall is a social network, like Facebook, which has a wall when you log in with the latest in your network—it has forum topics, links, you can share files, and you can chat with people in real time.
  • ...3 more annotations...
  • We had the data in a relational database format in SQL, and thanks to ETL (Extract, Transform, and Load) software Talend, we were able to easily transform the data from SQL to Neo4j (the graph-database format we used). Once the data was transformed, it was just a matter of plugging it into Linkurious, and in a couple of minutes, you have it visualized—in a networked way, so anyone can log in from anywhere in the world. That was another reason we really liked Linkurious and Neo4j—they’re very quick when representing graph data, and the visualizations were easy to understand for everybody. The not-very-tech-savvy reporter could expand the docs like magic, and more technically expert reporters and programmers could use the Neo4j query language, Cypher, to do more complex queries, like show me everybody within two degrees of separation of this person, or show me all the connected dots…
  • We believe in open source technology and try to use it as much as possible. We used Apache Solr for the indexing and Apache Tika for document processing, and it’s great because it processes dozens of different formats and it’s very powerful. Tika interacts with Tesseract, so we did the OCRing on Tesseract. To OCR the images, we created an army of 30–40 temporary servers in Amazon that allowed us to process the documents in parallel and do parallel OCR-ing. If it was very slow, we’d increase the number of servers—if it was going fine, we would decrease because of course those servers have a cost.
  • For the visualization of the Mossack Fonseca internal database, we worked with another tool called Linkurious. It’s not open source, it’s licensed software, but we have an agreement with them, and they allowed us to work with it. It allows you to represent data in graphs. We had a version of Linkurious on our servers, so no one else had the data. It was pretty intuitive—journalists had to click on dots that expanded, basically, and could search the names.
Gary Edwards

How would you fix the Linux desktop? | ITworld - 0 views

  • VB integrates with COM
  • QL Server has a DCE/RPC interface. 
  • MS-Office?  all the components (Excel, Word etc.) have a COM and an OLE interface.
  •  
    Comment posted 1 week ago in reply to Zzgomes .....  by Ed Carp.  Finally someone who gets it! OBTW, i replaced Windows 7 with Linux Mint over a year ago and hope to never return.  The thing is though, i am not a member of a Windows productivity workgroup, nor do i need to connect to any Windows databases or servers.  Essentially i am not using any Windows business process or systems.  It's all Internet!!! 100% Web and Cloud Services systems.  And that's why i can dump Windows without a blink! While working for Sursen Corp, it was a very different story.  I had to have Windows XP and Windows 7, plus MSOffice 2003-2007, plus Internet Explorer with access to SharePoint, Skydrive/Live.com.  It's all about the business processes and systems you're part of, or must join.   And that's exactly why the Linux Desktop has failed.  Give Cloud Computing the time needed to re-engineer and re-invent those many Windows business processes, and the Linux Desktop might suceed.  The trick will be in advancing both the Linux Desktop and Application developer layers to target the same Cloud Computing services mobility targets.  ..... Windows will take of itself.   The real fight is in the great transition of business systems and processes moving from the Windows desktp/workgroup productivity model to the Cloud.  Linux Communities must fight to win the great transition. And yes, in the end this all about a massive platform shift.  The fourth wave of computing began with the Internet, and will finally close out the desktop client/server computing model as the Web evolves into the Cloud. excerpt: Most posters here have it completely wrong...the *real* reason Linux doesn't have a decent penetration into the desktop market is quite obvious if you look at the most successful desktop in history - Windows.  All this nonsense about binary driver compatibility, distro fragmentation, CORBA, and all the other red herrings that people are talking about are completely irrelevant
Gary Edwards

Increo Solutions: purchased by Box.net - 0 views

  •  
    Use Backboard to collect feedback on your projects, and facilitate communication and collaboration in your company. Use embedit.in to embed any file into your website or blog. Uses Adobe Air/Flash runtime.  Excellent feedback and comment interface.  Documents are uploaded and "viewed" through custom viewers.  Backboard can gather feedback on documents (Word (DOC/DOCX), Excel (XLS/XLSX), PowerPoint (PPT/PPTX), WPD, ODT, ODP, ODS, RTF, PDF, TXT, HTML, SQL and JS), images (GIF, JPEG, PNG, TIFF, BMP and PSD), vector graphics (AI, EPS, PS), web pages or other URLs, and free-form text. Backboard also works with iWork documents (Pages, Keynote, and Numbers) by using the Backboard Plug-in for iWork '09. I liked the viewing/comment interface, but have not tested the various documents for fidelity!  No direct interaction or collaborative editing of documents!!!!!!  Just a view and feedback.
Gary Edwards

Gray Matter : Open XML and the SharePoint Conference - 0 views

  •  
    excerpt: The trend in Office development is the migration of solutions away from in-application scripted processing toward more data-centric development. Of course this is a primary purpose of Open XML, and it is great to see the amount of activity in this area. We've seen customers scripting Word in a server environment to batch process / print documents or for other automation tasks. In reality Word isn't built to do that on a large scale, it is better to work directly against the document rather than via the application whenever possible. The Open XML SDK unlocks a "whole nuther" environment for document processing, and gets you out of the business of scripting client apps on servers to do the work of a true server application (not to mention the licensing problems created by installing Office on a server). comment:  Gray makes a very important point here.  The dominance of the desktop based MSOffice Productivity Environment was largely based the embedded logic driving "in-process" documents that was application and platform (Win32 API) specific.  Tear open any of these workgroup-workflow oriented compound documents and you find application specific scripts, macros, OLE, data bindings, security settings and other application specific settings.  These internal components are certain to break whenever these highly interactive and "live" compound documents are converted to another format, or application use.  This is how MSOffice documents and the business processes they represent become "bound" to the MSOffice Productivity Environment. What Gray is pointing to here is that Microsoft is moving the legacy Productivity Environment to an MSWeb based center where OpenXML, Silverlight, CAML, XAML and a number of other .NET-WPF technologies become the workgroup drivers.  The key applications for the MS WebStack are Exchange/SharePoint/SQL Server.  To make this move, documents had to be separated from the legacy desktop Productivity Environment settings. Note th
Gary Edwards

Review: Microsoft's Office's Slow Road to the Web - PC World - 0 views

  • The button to open a document in a local copy of Office is apparently IE-only, and some features will require the SilverLight plug-in.
    • Gary Edwards
       
      uh oh.  I'm not so worried about IE specific features or Silverlight only features as i am about MOSS 2010 specific features (MSOffice desktop and SharePoint-SQL Server).  Especially critical will be the OLE, VBA scripting, and data bindings feature sets. How will Microsoft move these stalwarts of the local MOPE (Microsoft Productivity Environment and Client/Server WorkGroup) to the Web?  The end game here is for Microsoft to successfully move the desktop MOPE "Point of Assembly" to a Web centered SharePoint-SQL Server MOPE.  And cut Oracle out in the process.
Paul Merrell

Microsoft Unveils 'Turnkey' Cloud Appliance -- InformationWeek - 0 views

  • Microsoft on Monday unveiled a preconfigured system designed to help businesses move to cloud computing quickly and efficiently without disrupting existing IT operations. The Windows Azure platform appliance consists of the Windows Azure cloud operating system, Microsoft SQL Azure, and, according to the company, "a Microsoft-specified configuration" of network, storage, and server hardware.
  • Hewlett-Packard, Dell, and Fujitsu have signed agreements to offer versions of the Windows Azure platform appliance based on their own gear, Microsoft said. eBay, meanwhile, has successfully tested the offering and is moving some of its Web pages to Microsoft's cloud.
Gary Edwards

NoSQL Pioneers Are Driving the Web's Manifest Destiny - 1 views

  •  
    Good Chart comparing four types of Data Stores: Key-Value, Tabular/Columnar, Document Store, Relational excerpt: The bottleneck is no longer around performance or the cost of computing - it's about quickly getting the information to thousands, or hundreds of thousands, of nodes trying to act as one computer delivering a service. Google and IBM both have written about the data center as a computer, and Facebook says it thinks of adding hardware at the rack level rather than at the server level. But the current means of storing and accessing data have not made this leap from a single server to a rack - let alone an entire data center. As programmers attempt this leap, they face several difficulties, which include working with existing software and programming languages and figuring out what problems and bottlenecks the new services built on these monolithic computer platforms will encounter. Plus, the IT world doesn't all move at once, which means plenty of jobs and workloads will continue with the old way of doing things - that is, relational databases such as Oracle's offerings and the open source MySQL, which Oracle now has a stake in thanks to its purchase of Sun. The result is not a steady movement to non-relational databases or other methods of storing data, but a back-and-forth as programmers and businesses figure out what kind of architecture they need and what problems they want to solve. For a closer look at the issue and a bunch of charts detailing how the landscape is currently laid out, analyst Matt Sarrel, has penned a report over at GigaOM Pro (sub. req'd.) on the NoSQL movement called "NoSQL Databases - Providing Extreme Scale and Flexibility."
Paul Merrell

Hewlett-Packard Traded WebOS for This: The Autonomy Gamble - 0 views

  • Content management systems today continue to be based on the types of structured database systems about one or two steps more evolved than dBASE. We've known they would be insufficient for the task, but we've put off the problem of composing a new architecture. It's already too late for major IT companies to start that new architecture from square one; if a company has any hope of addressing this colossal, underappreciated problem, it will need to acquire the architectural project in progress. This is what Hewlett-Packard announced yesterday that it intends to do: acquire a software firm whose core product aims to supplant everything we know about databases, both the SQL kind and the Google kind. In its place would come a clustered approach whose goal is no less than to be the central repository for meaning in the world.
  • As CEO Apotheker told analysts yesterday, HP intends to exploit the prospects for using Autonomy's technology as a foundation for a content management system. For now, that CMS would be a project for what, on the surface, seems an unlikely department: the Imaging and Printing Group (IPG). Autonomy describes this technology - which it calls Intelligent Data Operating Layer (IDOL) - as nothing less than a replacement for, a complete substitute for, a revolutionary disruption of, Google.
  • Elsewhere in Autonomy's literature is a monkey wrench it hurls directly at Google, with hopes of messing up its gears. Here, the company attacks the value of Google's page ranking technology in the enterprise: "in many cases, the most popular information is also the most relevant. The importance or popularity of a Web page is approximated by counting the number of other pages that are linked to it, and by how frequently those pages are viewed by other users. This works quite well on the Internet but in the enterprise it is doomed to failure. Firstly, there are no native links between information in the enterprise. Secondly, if a user happens to be an expert, perhaps in the field of gallium arsenide laser diodes, there may be no one else interested in the subject, but it is still imperative that they find relevant information." This is what HP is buying: an opportunity to disrupt Google. If IDOL is every bit the next stage of database evolution that Autonomy makes it out to be, then HP (at least in its executives' own minds) is not surrendering to Google at all, as some consumer publications this morning are suggesting. As HP perceives it, rather than cutting off Google's left arm, it's targeting the gut.
1 - 11 of 11
Showing 20 items per page