Group items tagged

A freely accessible index of 5 billion web pages, their page rank, their link graphs and other metadata, hosted on Amazon EC2, was announced today by the Common Crawl Foundation. "It is crucial [in] our information-based society that Web crawl data be open and accessible to anyone who desires to utilize it," writes Foundation director Lisa Green on the organization's blog.

The Foundation is an organization dedicated to leveraging the falling costs of crawling and storage for the benefit of "individuals, academic groups, small start-ups, big companies, governments and nonprofits." It's lead by Gilad Elbaz, the forefather of Google AdSense and the CEO of data platform startup Factual. Joining Elbaz on the Foundation board is internet public domain champion Carl Malamud and semantic web serial entrepreneur Nova Spivack. Director Lisa Green came to the Foundation by way of Creative Commons.

The Foundation explains the scope of the project thusly. "Common Crawl is a Web Scale crawl, and as such, each version of our crawl contains billions of documents from the various sites that we are successfully able to crawl. This dataset can be tens of terabytes in size, making transfer of the crawl to interested third parties costly and impractical. In addition to this, performing data processing operations on a dataset this large requires parallel processing techniques, and a potentially large computer cluster. "Luckily for us, Amazon's EC2/S3 cloud computing infrastructure provides us with both a theoretically unlimited storage capacity coupled with localized access to an elastic compute cloud."

Shares of Amazon were down about 1% $202.52 by mid-morning Friday. The stock is up about 13% for the year to date. The Kindle Fire is estimated to cost a total of $201.70 to build each unit, a number that includes a bill of materials — or BOM — of $185.60, with another $16.10 in manufacturing costs for each unit, iSuppli estimates. The tear-down estimate does not include cost figures for software, licensing or royalties that Amazon may pay on the device.

Amazon began shipping the Kindle Fire on Monday, after first unveiling the device in late September. The tablet costs $199 — about 60% less than the cheapest iPad from Apple Inc. /quotes/zigman/68270/quotes/nls/aapl AAPL -0.16% , which currently dominates the tablet market. Analysts expect the Kindle Fire to make the largest dent in the tablet market next to the iPad, after other devices using Google’s /quotes/zigman/93888/quotes/nls/goog GOOG -0.89%  Android operating system have largely failed to generate strong sales. The low price of the device, along with Amazon’s large library of books and other digital content, are expected to contribute to the appeal of the Kindle Fire. But Amazon does not break out data on its device sales.

The first GSMA Mobile Observatory report to focus on Africa has come back with some fascinating conclusions. First among them, Africa has passed Latin America to become the world's second largest mobile market. The global mobile association examined the 25 African countries that account for 91% of mobile use (calling them the "A25"). Here are some of the most interesting of the report's conclusions.

The report "integrates data from a wide range of existing sources to provide a comprehensive picture of the African mobile industry. These include public sources such as the ITU, World Bank and research by National Regulatory Authorities as well as commercial providers such as Wireless Intelligence, Informa, Gartner, Buddecomm and ID." Africa now has 620 million mobile connections, passing Latin America Regional penetration has risen from 2% in 2000, to 57% this year Over the last decade, the number of mobile connections has grown an average of 30% per year, forecast to reach 735 million by the end of next year Mobile provides 3.5% of African GDP Highly competetive sector has resulted in price decreases of up to 60% in some areas, with an overall average of 18% decrease in the last year 96% of is pre-paid with voice services currently dominating, however data is increasing rapidly - Kenya data revenues, for instance, including SMS, have increased at a 67% annual growth over the last 4 years, representing 26% of total revenues in that country

Colorado, defending Yahoo against attempts by the federal government to obtain the contents of Yahoo Mail messages without first obtaining a warrant. One month earlier, the Justice Department filed a 17-page brief arguing that Yahoo Mail messages do not fall under current statutory protection because, once opened, those messages are not considered to be in “electronic storage.” The privacy coalition—which included Google—came to Yahoo’s defense, arguing that users with e-mail stored in the cloud have a reasonable expectation of privacy in the contents of that e-mail, and should thus be protected from warrantless searches by the government. (Hopefully the irony of Google opposing robust searches is not lost on Google’s attorneys.) Unfortunately, the protections afforded by the warrant requirement have not yet been fully extended to the digital “cloud.” This handy metaphor for the ethereal Internet as a storage and access hub is coming to have other implications: can we really conceal our data inside this cloud, shielding it from government intrusion? Read more at arstechnica.com

Sen. Patrick Leahy on Tuesday unveiled an overhaul to a 25-year-old digital privacy law that would require the government to obtain warrants before accesssing email and other cloud-based data. The update to the Electronic Communications Privacy Act (ECPA), would also extend to location-based data, and allow private companies to collaborate with the government in the event of a cyber attack. The ECPA was first enacted in 1986, well before the Internet, email, or smartphones. As a result, it is "significantly outdated and out-paced by rapid changes in technology and the changing mission of our law enforcement agencies after September 11," said Leahy, a Vermont Democrat. As a result, Leahy's updated 2011 version of the ECPA would apply to technologies like email, cloud services, and location data on smartphones. If the government wanted an ISP to hand over emails on a particular customer, for example, they would need to first obtain a warrant. At this point, the government abides by a rule that provides access to email after 180 days, depending on the circumstance.

"The Invisible Internet is associated closely with the concept of the “Internet of Things,” in which a multitude of everyday objects are connected wirelessly. In such a world, not every object will have the intelligence to make decisions for itself — your carton of milk doesn’t need an advanced processor, only the ability to communicate what it is and its expiration date — but collectively they’ll create a form of ambient intelligence, allowing them to self-organize as a group. If the Invisible Internet of Things does become a reality, the Web will cease to be merely a virtual space, where people interact with one another from behind a PC or phone’s screen, and become a real space — “meat space,” if you will — where thousands of objects, both personal and public, interact with one another.

The one element, besides a radio, all of those objects have in common is awareness. They have to be able to sense one another as well as their surroundings. Embedding devices and objects with that kind of sensitivity probably is the smallest challenge the Internet of Things faces right now, said Henry Tirri, head of the Nokia Research Center. The core sensors needed in the network of the future already are embedded in the average smartphone today: GPS and cellular triangulation sense location; accelerometers and digital compasses sense movement and direction; digital cameras can see for the devices. Some of those sensors need to be refined, but for the most part, devices already have access to enormous amounts of raw sensory data, Tirri said. The challenge for the industry is processing that data, interpreting it and combining it with data from other sensors to make it useful. Once the technology overcomes those problems, there’s no limit to what can be wirelessly enabled, he added.

“In today’s world of handsets, we talk in billions; in the future, we will talk about trillions of devices,” Tirri said. “Radios and sensors will be very small. They will be in everyday devices like coffeemakers and key chains, as well as all consumer devices, but also things you wouldn’t think you’d have wireless capabilities, like chairs, tables, even your bed.”

"Apple, Google and employers are already contravening new European Union rules that will require companies to get explicit permission from users before any geo-location data can be used to track them, whether for the purposes of targeted advertising or monitoring employee behavior. This could be the start of the next big privacy argument. The hopes of companies planning to use geo-location data to push products and services to mobile device users have taken a beating in the European Union, following a pronouncement from the European Data Protection Supervisor, Peter Hustinx."

Third-party apps are the weakest link in user privacy on smart phones. They often get access to large quantities of user data, and there are few rules covering how they must handle that data once they have it. Worse yet, few third-party apps have a privacy policy telling users what they intend to do. That was the message delivered at a hearing of the U.S. Senate committee on Commerce, Science, and Transportation held yesterday. Companies and regulators are struggling to find ways to ensure that user data is handled properly by apps installed on smart phones, but the way apps are designed makes this difficult.

This week, Facebook joined Google and Apple on the hot seat. But all three companies run platforms that support thousands of third-party developers, and how to make sure those apps respect users' privacy, and explain their rules, is a major question. Sen. Mark Pryor (D-Arkansas) said at the hearing, "It's not clear that Americans understand how their information may be shared or transferred."

In New York, at the Mobile Marketing Association forum, Google presented sponsored research on global smartphone user and marketer behavior. The data come from two related studies. The first is an “an online survey of thousands of mobile consumers in 30 countries.” The second is based on “a telephone survey of 1,000 marketing decision makers,” with a focus on US, UK, Germany, France and Japan.

More than Half of Smartphone Users Online Daily In the US smartphone penetration stands at about 36 percent according to the most recent Nielsen data. In Western Europe, on a percentage basis, the numbers are higher in several countries. The Google research showed that increasingly smartphone users go online daily and that many are on the mobile internet multiple times a day: US — 58 percent (online) 53 percent (multiple times) UK — 55 percent (online) 49 percent (multiple times) France — 59 percent (online) 47 percent (multiple times) Germany — 45 percent (online) 42 percent (multiple times) Japan — 78 percent (online) 68 percent (multiple times)

Almost All Local Info Seekers Take Action Here’s what the data showed about local-mobile information seekers and then the percentage who have “taken action” after a local search/lookup: US — 90 percent (search/lookup) 87 percent (took action) UK — 81 percent (search/lookup) 80 percent (took action) France — 83 percent (search/lookup) 83 percent (took action) Germany — 85 percent (search/lookup) 79 percent (took action) Japan — 90 percent (search/lookup) 80 percent (took action)

the Internet of Things will allow physical objects to transmit data about themselves and their surroundings, bringing more information about the real world into the online realm. Imagine getting precise, live traffic data from all the local roads; trains that tell your smartphone that they’re full before they arrive; flowers that email you when they need watering; maybe even implants in your body that give you real-time updates about your health that feed into a secure online ‘locker’ of your personal data. All this and more is possible with the Internet of Things, helping to transform what we expect from the Web and the Internet.

Sparks are a very simple taxonomy right now, but do have persistent URIs, which you can find by hovering over a Spark and looking in the left of the status bar at the bottom of your screen.

This warrants more dissecting and attention. Will they eventually use all or some of the hierarchy of Google Directory? Will they become hierarchical? Will the algorithm improve as we click on links that interest us? Can we add our own information? Are we creating new entities for Google as we search for and add Sparks to our items of interest – it seems that way. It’s not an ontology yet, but it’s a start. Lots of people creating persistent URIs for entities they’ve dreamed up – I hear that evil cackle again!

Google, by nature of its founding, is in a prime position to address the challenges that many enterprise technologists have when thinking about semantic data – how do we handle unstructured data? We have metadata: in schema, in taxonomies, in ontologies even. We have loads of content. With no metadata. How do we get them together? We can’t afford to hire a small army of indexers to apply the metadata to the content. The system metadata is insufficient and poor. We have a pretty good search tool, and have put some effort into data dictionaries, entity extraction and rules-based classification. We have tools that do latent semantic indexing and latent semantic analysis.  Make sense of unstructured information? Sure, Google can do that. Hopefully they will not reduce efforts in these areas too much to focus on other projects. Many of us can execute a search and return nothing useful; crowdsourcing tagging in G+ may re-vitalize  components of the search algorithm.

Cities are experimenting with releasing more public data, engaging with citizens on social networks, adopting open source software, and finding ways to use new technologies to work with their citizens. They've been doing it through the depth of the Great Recession, amidst aging infrastructure, spiraling costs and flat or falling budgets. In that context, using technology and the Internet to make government work better and cities smarter is no longer a "nice to have" ... it's become a must-have.

That's the kind of "citizensourcing" smarter government that Tolva is looking to tap into in Chicago.

"This is as much about citizens talking to the infrastructure of the city as infrastructure talking to itself," he said. "It's where urban informatics and smarter cities cross over to Gov 2.0. There are efficiencies to be gained by having both approaches. You get the best of both worlds by getting an Internet of things to grow."

Internet traffic is booming, and something has got to give. Cisco reported this June that global IP traffic increased eightfold during the last five years, and is expected to jump by a factor of four, as we reach the rather ominously named "zettabyte threshold" by 2015. With the proliferation of millions of networked devices, and the popularity of Internet video, none of this data demand is expected to slacken.

Very few of those devices are going to require a cable. But Wi-Fi is only one (rather limited) option of getting Internet signals through the air to you. In the future, the Internet might come from the "white space" in your television spectrum, unused satellite signals, or the LED office lights overhead. Perhaps all of them. For the immediate future, your new lightbulb is a leading contender.

A German physicist has come up with a wireless Internet solution to send data through an LED lightbulb fluctuating in intensity faster than the human eye can detect. The invention, dubbed D-Light, can send data faster than 10 megabits per second--faster than the average broadband connection--simply by altering the frequency of the ambient light in the room. It has new applications in hospitals, airplanes, military, and even underwater.

If mobile video sharing is to follow in the footsteps of its more desirable mobile photo-sharing cousin, which application will users want to use to shoot, share and discover video clips? It’s too soon to tell, but startup Klip joins the fray and is now vying for your video attention. The startup released its application for iPhone on Monday with a focus on letting users share super-short 1-minute video clips — on Klip or with Facebook, Twitter and Youtube — and helping users discover clips from friends or other users based on topics of interests. “Klip re-invents the way consumers experience the world by organizing mobile videos in real time and by connecting consumers with the people and the topics that interest them,” the company says.