Group items tagged

The Common Crawl Foundation’s repository of openly and freely accessible web crawl data is about to go live as a Public Data Set on Amazon Web Services.

Elbaz’ goal in developing the repository: “You can’t access, let alone download, the Google or the Bing crawl data. So certainly we’re differentiated in being very open and transparent about what we’re crawling and actually making it available to developers,” he says. “You might ask why is it going to be revolutionary to allow many more engineers and researchers and developers and students access to this data, whereas historically you have to work for one of the big search engines…. The question is, the world has the largest-ever corpus of knowledge out there on the web, and is there more that one can do with it than Google and Microsoft and a handful of other search engines are already doing? And the answer is unquestionably yes. ”

Common Crawl’s data already is stored on Amazon’s S3 service, but now Amazon will be providing the storage space for free through the Public Data Set program. Not only does that remove from Common Crawl the storage burden and costs for hosting its crawl of 5 billion web pages – some 50 or 60 terabytes large – but it should make it easier for users to access the data, and remove the bandwidth-related costs they might incur for downloads. Users won’t have to deal with setting up accounts, being responsible for bandwidth bills incurred, and more complex authentication processes.

IBM announced today that it would throw another billion at Linux, the open-source operating system, to run its Power System servers. The first time it had thrown a billion at Linux was in 2001, when Linux was a crazy, untested, even ludicrous proposition for the corporate world. So the moolah back then didn’t go to Linux itself, which was free, but to related technologies across hardware, software, and service, including things like sales and advertising – and into IBM’s partnership with Red Hat which was developing its enterprise operating system, Red Hat Enterprise Linux. “It helped start a flurry of innovation that has never slowed,” said Jim Zemlin, executive director of the Linux Foundation. IBM claims that the investment would “help clients capitalize on big data and cloud computing with modern systems built to handle the new wave of applications coming to the data center in the post-PC era.” Some of the moolah will be plowed into the Power Systems Linux Center in Montpellier, France, which opened today. IBM’s first Power Systems Linux Center opened in Beijing in May. IBM may be trying to make hay of the ongoing revelations that have shown that the NSA and other intelligence organizations in the US and elsewhere have roped in American tech companies of all stripes with huge contracts to perfect a seamless spy network. They even include physical aspects of surveillance, such as license plate scanners and cameras, which are everywhere [read.... Surveillance Society: If You Drive, You Get Tracked].

Then another boon for IBM. Experts at the German Federal Office for Security in Information Technology (BIS) determined that Windows 8 is dangerous for data security. It allows Microsoft to control the computer remotely through a “special surveillance chip,” the wonderfully named Trusted Platform Module (TPM), and a backdoor in the software – with keys likely accessible to the NSA and possibly other third parties, such as the Chinese. Risks: “Loss of control over the operating system and the hardware” [read.... LEAKED: German Government Warns Key Entities Not To Use Windows 8 – Links The NSA.

It would be an enormous competitive advantage for an IBM salesperson to walk into a government or corporate IT department and sell Big Data servers that don’t run on Windows, but on Linux. With the Windows 8 debacle now in public view, IBM salespeople don’t even have to mention it. In the hope of stemming the pernicious revenue decline their employer has been suffering from, they can politely and professionally hype the security benefits of IBM’s systems and mention in passing the comforting fact that some of it would be developed in the Power Systems Linux Centers in Montpellier and Beijing. Alas, Linux too is tarnished. The backdoors are there, though the code can be inspected, unlike Windows code. And then there is Security-Enhanced Linux (SELinux), which was integrated into the Linux kernel in 2003. It provides a mechanism for supporting “access control” (a backdoor) and “security policies.” Who developed SELinux? Um, the NSA – which helpfully discloses some details on its own website (emphasis mine): The results of several previous research projects in this area have yielded a strong, flexible mandatory access control architecture called Flask. A reference implementation of this architecture was first integrated into a security-enhanced Linux® prototype system in order to demonstrate the value of flexible mandatory access controls and how such controls could be added to an operating system. The architecture has been subsequently mainstreamed into Linux and ported to several other systems, including the Solaris™ operating system, the FreeBSD® operating system, and the Darwin kernel, spawning a wide range of related work.

Online hacktivist collective Anonymous has announced that it is working on a new tool called Airchat which could allow people to communicate without the need for a phone or an internet connection - using radio waves instead. Anonymous, the amorphous group best known for attacking high profile targets like Sony and the CIA in recent years, said on the Lulz Labs project's Github page: "Airchat is a free communication tool [that] doesn't need internet infrastructure [or] a cell phone network. Instead it relies on any available radio link or device capable of transmitting audio." The idea is that people all over the world, including those in rural areas and developing countries, will one day be able to communicate for free without the need for a mobile phone network, phone line or internet access.

The fair use doctrine permits the unauthorized digitization of copyrighted works in order to create a full-text searchable database, the U.S. Court of Appeals for the Second Circuit ruled June 10.Affirming summary judgment in favor of a consortium of university libraries, the court also ruled that the fair use doctrine permits the unauthorized conversion of those works into accessible formats for use by persons with disabilities, such as the blind.

The dispute is connected to the long-running conflict between Google Inc. and various authors of books that Google included in a mass digitization program. In 2004, Google began soliciting the participation of publishers in its Google Print for Publishers service, part of what was then called the Google Print project, aimed at making information available for free over the Internet.Subsequently, Google announced a new project, Google Print for Libraries. In 2005, Google Print was renamed Google Book Search and it is now known simply as Google Books. Under this program, Google made arrangements with several of the world's largest libraries to digitize the entire contents of their collections to create an online full-text searchable database.The announcement of this program triggered a copyright infringement action by the Authors Guild that continues to this day.

Part of the deal between Google and the libraries included an offer by Google to hand over to the libraries their own copies of the digitized versions of their collections.In 2011, a group of those libraries announced the establishment of a new service, called the HathiTrust digital library, to which the libraries would contribute their digitized collections. This database of copies is to be made available for full-text searching and preservation activities. Additionally, it is intended to offer free access to works to individuals who have “print disabilities.” For works under copyright protection, the search function would return only a list of page numbers that a search term appeared on and the frequency of such appearance.

Huge volumes of private emails, phone calls, and internet chats are being intercepted by the National Security Agency with the secret cooperation of more foreign governments than previously known, according to newly disclosed documents from whistleblower Edward Snowden. The classified files, revealed today by the Danish newspaper Dagbladet Information in a reporting collaboration with The Intercept, shed light on how the NSA’s surveillance of global communications has expanded under a clandestine program, known as RAMPART-A, that depends on the participation of a growing network of intelligence agencies.

It has already been widely reported that the NSA works closely with eavesdropping agencies in the United Kingdom, Canada, New Zealand, and Australia as part of the so-called Five Eyes surveillance alliance. But the latest Snowden documents show that a number of other countries, described by the NSA as “third-party partners,” are playing an increasingly important role – by secretly allowing the NSA to install surveillance equipment on their fiber-optic cables. The NSA documents state that under RAMPART-A, foreign partners “provide access to cables and host U.S. equipment.” This allows the agency to covertly tap into “congestion points around the world” where it says it can intercept the content of phone calls, faxes, e-mails, internet chats, data from virtual private networks, and calls made using Voice over IP software like Skype.

The secret documents reveal that the NSA has set up at least 13 RAMPART-A sites, nine of which were active in 2013. Three of the largest – codenamed AZUREPHOENIX, SPINNERET and MOONLIGHTPATH – mine data from some 70 different cables or networks. The precise geographic locations of the sites and the countries cooperating with the program are among the most carefully guarded of the NSA’s secrets, and these details are not contained in the Snowden files. However, the documents point towards some of the countries involved – Denmark and Germany among them. An NSA memo prepared for a 2012 meeting between the then-NSA director, Gen. Keith Alexander, and his Danish counterpart noted that the NSA had a longstanding partnership with the country’s intelligence service on a special “cable access” program. Another document, dated from 2013 and first published by Der Spiegel on Wednesday, describes a German cable access point under a program that was operated by the NSA, the German intelligence service BND, and an unnamed third partner.

Zuckerberg's Internet.org will control what billions do online People in countries like India,1,2,3 Zimbabwe,4 Brazil,5 and Paraguay6 are speaking out about Facebook's so-called Internet.org platform and its ability to control what billions of Internet users can do online.7,8   Zuckerberg's partnership with telecom giants, Internet.org, provides access to a fake Internet where selected services are prioritized over others.9 This scheme threatens innovation,10 free expression,11 and privacy online12   It blocks many of the websites, apps, and services the world loves from being made available on equal terms.13   The fake Internet will also restrict access to local service providers struggling to get a foothold online.14   We all deserve access to the real open Internet. Stand with people around the world demanding Zuckerberg stops restricting access to the open Internet.

Before Chinese President Xi Jinping visits President Obama, he and Chinese executives have some business in Seattle: pressing U.S. tech companies, hungry for the Chinese market, to comply with the country’s new stringent and suppressive Internet policies. The New York Times reported last week that Chinese authorities sent a letter to some U.S. tech firms seeking a promise they would not harm China’s national security. That might require such things as forcing users to register with their real names, storing Chinese citizens’ data locally where the government can access it, and building government “back doors” into encrypted communication products for better surveillance. China’s new national security law calls for systems that are “secure and controllable”, which industry groups told the Times in July means companies will have to hand over encryption keys or even source code to their products. Among the big names joining Xi at Wednesday’s U.S.-China Internet Industry Forum: Apple, Google, Facebook, IBM, and Microsoft.

The meeting comes as U.S. law enforcement officials have been pressuring companies to give them a way to access encrypted communications. The technology community has responded by pointing out that any sort of hole for law enforcement weakens the entire system to attack from outside bad actors—such as China, which has been tied to many instances of state-sponsored hacking into U.S systems. In fact, one argument privacy advocates have repeatedly made is that back doors for law enforcement would set a dangerous precedent when countries like China want the same kind of access to pursue their own domestic political goals. But here, potentially, the situation has been reversed, with China using its massive economic leverage to demand that sort of access right now. Human rights groups are urging U.S. companies not to give in.

After months of deliberation, the Obama administration has made a long-awaited decision on the thorny issue of how to deal with encrypted communications: It will not — for now — call for legislation requiring companies to decode messages for law enforcement. Rather, the administration will continue trying to persuade companies that have moved to encrypt their customers’ data to create a way for the government to still peer into people’s data when needed for criminal or terrorism investigations. “The administration has decided not to seek a legislative remedy now, but it makes sense to continue the conversations with industry,” FBI Director James B. Comey said at a Senate hearing Thursday of the Homeland Security and Governmental Affairs Committee.

The decision, which essentially maintains the status quo, underscores the bind the administration is in — balancing competing pressures to help law enforcement and protect consumer privacy. The FBI says it is facing an increasing challenge posed by the encryption of communications of criminals, terrorists and spies. A growing number of companies have begun to offer encryption in which the only people who can read a message, for instance, are the person who sent it and the person who received it. Or, in the case of a device, only the device owner has access to the data. In such cases, the companies themselves lack “backdoors” or keys to decrypt the data for government investigators, even when served with search warrants or intercept orders.

The decision was made at a Cabinet meeting Oct. 1. “As the president has said, the United States will work to ensure that malicious actors can be held to account — without weakening our commitment to strong encryption,” National Security Council spokesman Mark Stroh said. “As part of those efforts, we are actively engaged with private companies to ensure they understand the public safety and national security risks that result from malicious actors’ use of their encrypted products and services.” But privacy advocates are concerned that the administration’s definition of strong encryption also could include a system in which a company holds a decryption key or can retrieve unencrypted communications from its servers for law enforcement. “The government should not erode the security of our devices or applications, pressure companies to keep and allow government access to our data, mandate implementation of vulnerabilities or backdoors into products, or have disproportionate access to the keys to private data,” said Savecrypto.org, a coalition of industry and privacy groups that has launched a campaign to petition the Obama administration.

Examples of authoring tools: ATAG 2.0 applies to a wide variety of web content generating applications, including, but not limited to: web page authoring tools (e.g., WYSIWYG HTML editors) software for directly editing source code (see note below) software for converting to web content technologies (e.g., "Save as HTML" features in office suites) integrated development environments (e.g., for web application development) software that generates web content on the basis of templates, scripts, command-line input or "wizard"-type processes software for rapidly updating portions of web pages (e.g., blogging, wikis, online forums) software for generating/managing entire web sites (e.g., content management systems, courseware tools, content aggregators) email clients that send messages in web content technologies multimedia authoring tools debugging tools for web content software for creating mobile web applications

Web-based and non-web-based: ATAG 2.0 applies equally to authoring tools of web content that are web-based, non-web-based or a combination (e.g., a non-web-based markup editor with a web-based help system, a web-based content management system with a non-web-based file uploader client). Real-time publishing: ATAG 2.0 applies to authoring tools with workflows that involve real-time publishing of web content (e.g., some collaborative tools). For these authoring tools, conformance to Part B of ATAG 2.0 may involve some combination of real-time accessibility supports and additional accessibility supports available after the real-time authoring session (e.g., the ability to add captions for audio that was initially published in real-time). For more information, see the Implementing ATAG 2.0 - Appendix E: Real-time content production. Text Editors: ATAG 2.0 is not intended to apply to simple text editors that can be used to edit source content, but that include no support for the production of any particular web content technology. In contrast, ATAG 2.0 can apply to more sophisticated source content editors that support the production of specific web content technologies (e.g., with syntax checking, markup prediction, etc.).

At decade's end, the trusty landline telephone could be nothing more than a memory. Telecom giants AT&T T +0.31% AT&T Inc. U.S.: NYSE $35.07 +0.11 +0.31% March 28, 2014 4:00 pm Volume (Delayed 15m) : 24.66M AFTER HOURS $35.03 -0.04 -0.11% March 28, 2014 7:31 pm Volume (Delayed 15m): 85,446 P/E Ratio 10.28 Market Cap $182.60 Billion Dividend Yield 5.25% Rev. per Employee $529,844 03/29/14 Prepare to Hang Up the Phone, ... 03/21/14 AT&T Criticizes Netflix's 'Arr... 03/21/14 Samsung's Galaxy S5 Smartphone... More quote details and news » T in Your Value Your Change Short position and Verizon Communications VZ -0.57% Verizon Communications Inc. U.S.: NYSE $47.42 -0.27 -0.57% March 28, 2014 4:01 pm Volume (Delayed 15m) : 24.13M AFTER HOURS $47.47 +0.05 +0.11% March 28, 2014 7:59 pm Volume (Delayed 15m): 1.57M

The two providers want to lay the crumbling POTS to rest and replace it with Internet Protocol-based systems that use the same wired and wireless broadband networks that bring Web access, cable programming and, yes, even your telephone service, into your homes. You may think you have a traditional landline because your home phone plugs into a jack, but if you have bundled your phone with Internet and cable services, you're making calls over an IP network, not twisted copper wires. California, Florida, Texas, Georgia, North Carolina, Wisconsin and Ohio are among states that agree telecom resources would be better redirected into modern telephone technologies and innovations, and will kill copper-based technologies in the next three years or so. Kentucky and Colorado are weighing similar laws, which force people to go wireless whether they want to or not. In Mantoloking, N.J., Verizon wants to replace the landline system, which Hurricane Sandy wiped out, with its wireless Voice Link. That would make it the first entire town to go landline-less, a move that isn't sitting well with all residents.

New Jersey's legislature, worried about losing data applications such as credit-card processing and alarm systems that wireless systems can't handle, wants a one-year moratorium to block that switch. It will vote on the measure this month. (Verizon tried a similar change in Fire Island, N.Y., when its copper lines were destroyed, but public opposition persuaded Verizon to install fiber-optic cable.) It's no surprise that landlines are unfashionable, considering many of us already have or are preparing to ditch them. More than 38% of adults and 45.5% of children live in households without a landline telephone, says the Centers for Disease Control and Prevention. That means two in every five U.S. homes, or 39%, are wireless, up from 26.6% three years ago. Moreover, a scant 8.5% of households relied only on a landline, while 2% were phoneless in 2013. Metropolitan residents have few worries about the end of landlines. High-speed wire and wireless services are abundant and work well, despite occasional dropped calls. Those living in rural areas, where cell towers are few and 4G capability limited, face different issues.

The United Nations’ top official for counter-terrorism and human rights (known as the “Special Rapporteur”) issued a formal report to the U.N. General Assembly today that condemns mass electronic surveillance as a clear violation of core privacy rights guaranteed by multiple treaties and conventions. “The hard truth is that the use of mass surveillance technology effectively does away with the right to privacy of communications on the Internet altogether,” the report concluded. Central to the Rapporteur’s findings is the distinction between “targeted surveillance” — which “depend[s] upon the existence of prior suspicion of the targeted individual or organization” — and “mass surveillance,” whereby “states with high levels of Internet penetration can [] gain access to the telephone and e-mail content of an effectively unlimited number of users and maintain an overview of Internet activity associated with particular websites.” In a system of “mass surveillance,” the report explained, “all of this is possible without any prior suspicion related to a specific individual or organization. The communications of literally every Internet user are potentially open for inspection by intelligence and law enforcement agencies in the States concerned.”

Mass surveillance thus “amounts to a systematic interference with the right to respect for the privacy of communications,” it declared. As a result, “it is incompatible with existing concepts of privacy for States to collect all communications or metadata all the time indiscriminately.” In concluding that mass surveillance impinges core privacy rights, the report was primarily focused on the International Covenant on Civil and Political Rights, a treaty enacted by the General Assembly in 1966, to which all of the members of the “Five Eyes” alliance are signatories. The U.S. ratified the treaty in 1992, albeit with various reservations that allowed for the continuation of the death penalty and which rendered its domestic law supreme. With the exception of the U.S.’s Persian Gulf allies (Saudi Arabia, UAE and Qatar), virtually every major country has signed the treaty. Article 17 of the Covenant guarantees the right of privacy, the defining protection of which, the report explained, is “that individuals have the right to share information and ideas with one another without interference by the State, secure in the knowledge that their communication will reach and be read by the intended recipients alone.”

The report’s key conclusion is that this core right is impinged by mass surveillance programs: “Bulk access technology is indiscriminately corrosive of online privacy and impinges on the very essence of the right guaranteed by article 17. In the absence of a formal derogation from States’ obligations under the Covenant, these programs pose a direct and ongoing challenge to an established norm of international law.” The report recognized that protecting citizens from terrorism attacks is a vital duty of every state, and that the right of privacy is not absolute, as it can be compromised when doing so is “necessary” to serve “compelling” purposes. It noted: “There may be a compelling counter-terrorism justification for the radical re-evaluation of Internet privacy rights that these practices necessitate. ” But the report was adamant that no such justifications have ever been demonstrated by any member state using mass surveillance: “The States engaging in mass surveillance have so far failed to provide a detailed and evidence-based public justification for its necessity, and almost no States have enacted explicit domestic legislation to authorize its use.”

In his first major policy speech as director of the F.B.I., James B. Comey on Thursday plans to wade deeper into the debate between law enforcement agencies and technology companies about new programs intended to protect personal information on communication devices.Mr. Comey will say that encryption technologies used on these devices, like the new iPhone, have become so sophisticated that crimes will go unsolved because law enforcement officers will not be able to get information from them, according to a senior F.B.I. official who provided a preview of the speech.The speech was prompted, in part, by the new encryption technology on the iPhone 6, which was released last month. The phone encrypts emails, photos and contacts, thwarting intelligence and law enforcement agencies, like the National Security Agency and F.B.I., from gaining access to it, even if they have court approval.

The F.B.I. has long had concerns about devices “going dark” — when technology becomes so sophisticated that the authorities cannot gain access. But now, Mr. Comey said he believes that the new encryption technology has evolved to the point that it will adversely affect crime solving.He will say in the speech that these new programs will most severely affect state and local law enforcement agencies, because they are the ones who most often investigate crimes like kidnappings and robberies in which getting information from electronic devices in a timely manner is essential to solving the crime.

They also do not have the resources that are available to the F.B.I. and other federal intelligence and law enforcement authorities in order to get around the programs.Mr. Comey will cite examples of crimes that the authorities were able to solve because they gained access to a phone.“He is going to call for a discussion on this issue and ask whether this is the path we want to go down,” said the senior F.B.I. official. “He is not going to accuse the companies of designing the technologies to prevent the F.B.I. from accessing them. But, he will say that this is a negative byproduct and we need to work together to fix it.”

Digitizing Print Collections with the Internet Archive Open and free online access, permanent storage, unlimited downloads and lifetime file management. We can help digitize your collections in 4 simple steps:

In addition to permanent hosting on archive.org, your books will be integrated with Open Library, openlibrary.org, a page on the web for every book.

Non-destructive color scanning using our Scribe system at one of our scanning centers across the globe. Complete MARC records, Dublin Core & XML, just 10c USD per image and a small set up charge per item.

David Mao, the Librarian of Congress, has issued new rules pertaining to exemptions to the Digital Millennium Copyright Act (DMCA) after a 3 year battle that was expedited in the wake of the Volkswagen scandal.

Opposition to this new decision is coming from the Environmental Protection Agency (EPA) and the auto industry because the DMCA prohibits “circumventing encryption or access controls to copy or modify copyrighted works.” For example, GM “claimed the exemption ‘could introduce safety and security issues as well as facilitate violation of various laws designed specifically to regulate the modern car, including emissions, fuel economy, and vehicle safety regulations’.” The exemption in question is in Section 1201 which forbids the unlocking of software access controls which has given the auto industry the unique ability to “threaten legal action against anyone who needs to get around those restrictions, no matter how legitimate the reason.” Journalist Nick Statt points out that this provision “made it illegal in the past to unlock your smartphone from its carrier or even to share your HBO Go password with a friend. It’s designed to let corporations protect copyrighted material, but it allows them to crackdown on circumventions even when they’re not infringing on those copyrights or trying to access or steal proprietary information.”

Kit Walsh, staff attorney for the Electronic Frontier Foundation (EFF), explained that the “‘access control’ rule is supposed to protect against unlawful copying. But as we’ve seen in the recent Volkswagen scandal—where VW was caught manipulating smog tests—it can be used instead to hide wrongdoing hidden in computer code.” Walsh continued: “We are pleased that analysts will now be able to examine the software in the cars we drive without facing legal threats from car manufacturers, and that the Librarian has acted to promote competition in the vehicle aftermarket and protect the long tradition of vehicle owners tinkering with their cars and tractors. The year-long delay in implementing the exemptions, though, is disappointing and unjustified. The VW smog tests and a long run of security vulnerabilities have shown researchers and drivers need the exemptions now.” As part of the new changes, gamers can “modify an old video game so it doesn’t perform a check with an authentication server that has since been shut down” and after the publisher cuts of support for the video game.