Group items tagged

From a military perspective, surveillance of the Internet is merely "Phase 0" in the US digital war strategy. Internal NSA documents indicate that it is the prerequisite for everything that follows. They show that the aim of the surveillance is to detect vulnerabilities in enemy systems. Once "stealthy implants" have been placed to infiltrate enemy systems, thus allowing "permanent accesses," then Phase Three has been achieved -- a phase headed by the word "dominate" in the documents. This enables them to "control/destroy critical systems & networks at will through pre-positioned accesses (laid in Phase 0)." Critical infrastructure is considered by the agency to be anything that is important in keeping a society running: energy, communications and transportation. The internal documents state that the ultimate goal is "real time controlled escalation". One NSA presentation proclaims that "the next major conflict will start in cyberspace." To that end, the US government is currently undertaking a massive effort to digitally arm itself for network warfare. For the 2013 secret intelligence budget, the NSA projected it would need around $1 billion in order to increase the strength of its computer network attack operations. The budget included an increase of some $32 million for "unconventional solutions" alone.

Part 2: How the NSA Reads Over Shoulders of Other Spies

NSA Docs on ExfiltrationExplanation of the APEX method of combining passive with active methods to exfiltrate data from networks attacked Explanation of APEX shaping to put exfiltrating network traffic into patterns that allow plausible deniability Presentation on the FASHIONCLEFT protocol that the NSA uses to exfiltrate data from trojans and implants to the NSA Methods to exfiltrate data even from devices which are supposed to be offline Document detailing SPINALTAP, an NSA project to combine data from active operations and passive signals intelligence Technical description of the FASHIONCLEFT protocol the NSA uses to exfiltrate data from Trojans and implants to the NSA

NSA Docs on Malware and ImplantsCSEC document about the recognition of trojans and other "network based anomaly" The formalized process through which analysts choose their data requirement and then get to know the tools that can do the job QUANTUMTHEORY is a set of technologies allowing man-on-the-side interference attacks on TCP/IP connections (includes STRAIGHTBIZARRE and DAREDEVIL) Sample code of a malware program from the Five Eyes alliance

According to top secret documents from the archive of NSA whistleblower Edward Snowden seen exclusively by SPIEGEL, they are planning for wars of the future in which the Internet will play a critical role, with the aim of being able to use the net to paralyze computer networks and, by doing so, potentially all the infrastructure they control, including power and water supplies, factories, airports or the flow of money. During the 20th century, scientists developed so-called ABC weapons -- atomic, biological and chemical. It took decades before their deployment could be regulated and, at least partly, outlawed. New digital weapons have now been developed for the war on the Internet. But there are almost no international conventions or supervisory authorities for these D weapons, and the only law that applies is the survival of the fittest. Canadian media theorist Marshall McLuhan foresaw these developments decades ago. In 1970, he wrote, "World War III is a guerrilla information war with no division between military and civilian participation." That's precisely the reality that spies are preparing for today.

As the gap has grown between Washington’s global reach and its shrinking mailed fist, as it struggles to maintain 40% of world armaments (the 2012 figure) with only 23% of global gross economic output, the U.S. will need to find new ways to exercise its power far more economically. As the Cold War took off, a heavy-metal U.S. military -- with 500 bases worldwide circa 1950 -- was sustainable because the country controlled some 50% of the global gross product. But as its share of world output falls -- to an estimated 17% by 2016 -- and its social welfare costs climb relentlessly from 4% of gross domestic product in 2010 to a projected 18% by 2050, cost-cutting becomes imperative if Washington is to survive as anything like the planet’s “sole superpower.” Compared to the $3 trillion cost of the U.S. invasion and occupation of Iraq, the NSA’s 2012 budget of just $11 billion for worldwide surveillance and cyberwarfare looks like cost saving the Pentagon can ill-afford to forego. Yet this seeming “bargain” comes at what turns out to be an almost incalculable cost. The sheer scale of such surveillance leaves it open to countless points of penetration, whether by a handful of anti-war activists breaking into an FBI field office in Media, Pennsylvania, back in 1971 or Edward Snowden downloading NSA documents at a Hawaiian outpost in 2012.

In October 2001, not satisfied with the sweeping and extraordinary powers of the newly passed Patriot Act, President Bush ordered the National Security Agency to commence covert monitoring of private communications through the nation's telephone companies without the requisite FISA warrants. Somewhat later, the agency began sweeping the Internet for emails, financial data, and voice messaging on the tenuous theory that such “metadata” was “not constitutionally protected.” In effect, by penetrating the Internet for text and the parallel Public Switched Telephone Network (PSTN) for voice, the NSA had gained access to much of the world’s telecommunications. By the end of Bush’s term in 2008, Congress had enacted laws that not only retrospectively legalized these illegal programs, but also prepared the way for NSA surveillance to grow unchecked. Rather than restrain the agency, President Obama oversaw the expansion of its operations in ways remarkable for both the sheer scale of the billions of messages collected globally and for the selective monitoring of world leaders.

By 2012, the centralization via digitization of all voice, video, textual, and financial communications into a worldwide network of fiber optic cables allowed the NSA to monitor the globe by penetrating just 190 data hubs -- an extraordinary economy of force for both political surveillance and cyberwarfare.

With a few hundred cable probes and computerized decryption, the NSA can now capture the kind of gritty details of private life that J. Edgar Hoover so treasured and provide the sort of comprehensive coverage of populations once epitomized by secret police like East Germany’s Stasi. And yet, such comparisons only go so far. After all, once FBI agents had tapped thousands of phones, stenographers had typed up countless transcripts, and clerks had stored this salacious paper harvest in floor-to-ceiling filing cabinets, J. Edgar Hoover still only knew about the inner-workings of the elite in one city: Washington, D.C.  To gain the same intimate detail for an entire country, the Stasi had to employ one police informer for every six East Germans -- an unsustainable allocation of human resources. By contrast, the marriage of the NSA’s technology to the Internet’s data hubs now allows the agency’s 37,000 employees a similarly close coverage of the entire globe with just one operative for every 200,000 people on the planet

Through the expenditure of $250 million annually under its Sigint Enabling Project, the NSA has stealthily penetrated all encryption designed to protect privacy. “In the future, superpowers will be made or broken based on the strength of their cryptanalytic programs,” reads a 2007 NSA document. “It is the price of admission for the U.S. to maintain unrestricted access to and use of cyberspace.” By collecting knowledge -- routine, intimate, or scandalous -- about foreign leaders, imperial proconsuls from ancient Rome to modern America have gained both the intelligence and aura of authority necessary for dominion over alien societies. The importance, and challenge, of controlling these local elites cannot be overstated. During its pacification of the Philippines after 1898, for instance, the U.S. colonial regime subdued contentious Filipino leaders via pervasive policing that swept up both political intelligence and personal scandal. And that, of course, was just what J. Edgar Hoover was doing in Washington during the 1950s and 1960s.

Indeed, the mighty British Empire, like all empires, was a global tapestry woven out of political ties to local leaders or “subordinate elites” -- from Malay sultans and Indian maharajas to Gulf sheiks and West African tribal chiefs. As historian Ronald Robinson once observed, the British Empire spread around the globe for two centuries through the collaboration of these local leaders and then unraveled, in just two decades, when that collaboration turned to “non-cooperation.” After rapid decolonization during the 1960s transformed half-a-dozen European empires into 100 new nations, their national leaders soon found themselves the subordinate elites of a spreading American global imperium. Washington suddenly needed the sort of private information that could keep such figures in line. Surveillance of foreign leaders provides world powers -- Britain then, America now -- with critical information for the exercise of global hegemony. Such spying gave special penetrating power to the imperial gaze, to that sense of superiority necessary for dominion over others.  It also provided operational information on dissidents who might need to be countered with covert action or military force; political and economic intelligence so useful for getting the jump on allies in negotiations of all sorts; and, perhaps most important of all, scurrilous information about the derelictions of leaders useful in coercing their compliance.

In late 2013, the New York Times reported that, when it came to spying on global elites, there were “more than 1,000 targets of American and British surveillance in recent years,” reaching down to mid-level political actors in the international arena. Revelations from Edward Snowden’s cache of leaked documents indicate that the NSA has monitored leaders in some 35 nations worldwide -- including Brazilian president Dilma Rousseff, Mexican presidents Felipe Calderón and Enrique Peña Nieto, German Chancellor Angela Merkel, and Indonesia’s president Susilo Bambang Yudhoyono.  Count in as well, among so many other operations, the monitoring of “French diplomatic interests” during the June 2010 U.N. vote on Iran sanctions and “widespread surveillance” of world leaders during the Group 20 summit meeting at Ottawa in June 2010. Apparently, only members of the historic “Five Eyes” signals-intelligence alliance (Australia, Canada, New Zealand, and Great Britain) remain exempt -- at least theoretically -- from NSA surveillance. Such secret intelligence about allies can obviously give Washington a significant diplomatic advantage. During U.N. wrangling over the U.S. invasion of Iraq in 2002-2003, for example, the NSA intercepted Secretary-General Kofi Anan’s conversations and monitored the “Middle Six” -- Third World nations on the Security Council -- offering what were, in essence, well-timed bribes to win votes. The NSA’s deputy chief for regional targets sent a memo to the agency’s Five Eyes allies asking “for insights as to how membership is reacting to on-going debate regarding Iraq, plans to vote on any related resolutions [..., and] the whole gamut of information that could give U.S. policymakers an edge in obtaining results favorable to U.S. goals.”

Indicating Washington’s need for incriminating information in bilateral negotiations, the State Department pressed its Bahrain embassy in 2009 for details, damaging in an Islamic society, on the crown princes, asking: “Is there any derogatory information on either prince? Does either prince drink alcohol? Does either one use drugs?” Indeed, in October 2012, an NSA official identified as “DIRNSA,” or Director General Keith Alexander, proposed the following for countering Muslim radicals: “[Their] vulnerabilities, if exposed, would likely call into question a radicalizer’s devotion to the jihadist cause, leading to the degradation or loss of his authority.” The agency suggested that such vulnerabilities could include “viewing sexually explicit material online” or “using a portion of the donations they are receiving… to defray personal expenses.” The NSA document identified one potential target as a “respected academic” whose “vulnerabilities” are “online promiscuity.”

Just as the Internet has centralized communications, so it has moved most commercial sex into cyberspace. With an estimated 25 million salacious sites worldwide and a combined 10.6 billion page views per month in 2013 at the five top sex sites, online pornography has become a global business; by 2006, in fact, it generated $97 billion in revenue. With countless Internet viewers visiting porn sites and almost nobody admitting it, the NSA has easy access to the embarrassing habits of targets worldwide, whether Muslim militants or European leaders. According to James Bamford, author of two authoritative books on the agency, “The NSA's operation is eerily similar to the FBI's operations under J. Edgar Hoover in the 1960s where the bureau used wiretapping to discover vulnerabilities, such as sexual activity, to ‘neutralize’ their targets.”

Indeed, whistleblower Edward Snowden has accused the NSA of actually conducting such surveillance.  In a December 2013 letter to the Brazilian people, he wrote, “They even keep track of who is having an affair or looking at pornography, in case they need to damage their target's reputation.” If Snowden is right, then one key goal of NSA surveillance of world leaders is not U.S. national security but political blackmail -- as it has been since 1898. Such digital surveillance has tremendous potential for scandal, as anyone who remembers New York Governor Eliot Spitzer’s forced resignation in 2008 after routine phone taps revealed his use of escort services; or, to take another obvious example, the ouster of France’s budget minister Jérôme Cahuzac in 2013 following wire taps that exposed his secret Swiss bank account. As always, the source of political scandal remains sex or money, both of which the NSA can track with remarkable ease.

By starting a swelling river of NSA documents flowing into public view, Edward Snowden has given us a glimpse of the changing architecture of U.S. global power. At the broadest level, Obama’s digital “pivot” complements his overall defense strategy, announced in 2012, of reducing conventional forces while expanding into the new, cost-effective domains of space and cyberspace. While cutting back modestly on costly armaments and the size of the military, President Obama has invested billions in the building of a new architecture for global information control. If we add the $791 billion expended to build the Department of Homeland Security bureaucracy to the $500 billion spent on an increasingly para-militarized version of global intelligence in the dozen years since 9/11, then Washington has made a $1.2 trillion investment in a new apparatus of world power.

So formidable is this security bureaucracy that Obama’s recent executive review recommended the regularization, not reform, of current NSA practices, allowing the agency to continue collecting American phone calls and monitoring foreign leaders into the foreseeable future. Cyberspace offers Washington an austerity-linked arena for the exercise of global power, albeit at the cost of trust by its closest allies -- a contradiction that will bedevil America’s global leadership for years to come. To update Henry Stimson: in the age of the Internet, gentlemen don't just read each other’s mail, they watch each other’s porn. Even if we think we have nothing to hide, all of us, whether world leaders or ordinary citizens, have good reason to be concerned.

He praised the revelations and bravery of former NSA contractor Edward Snowden and told me that he had indirect contact with a number of other NSA employees who felt disgusted with the agency’s work. They’re keen to speak out but fear retribution and exile, not unlike Snowden himself, who is likely to remain there for some time.

Binney recently told the German NSA inquiry committee that his former employer had a “totalitarian mentality” that was the "greatest threat" to US society since that country’s US Civil War in the 19th century. Despite this remarkable power, Binney still mocked the NSA’s failures, including missing this year’s Russian intervention in Ukraine and the Islamic State’s take-over of Iraq.The era of mass surveillance has gone from the fringes of public debate to the mainstream, where it belongs. The Pew Research Centre released a report this month, Digital Life in 2025, that predicted worsening state control and censorship, reduced public trust, and increased commercialisation of every aspect of web culture.It’s not just internet experts warning about the internet’s colonisation by state and corporate power. One of Europe’s leading web creators, Lena Thiele, presented her stunning series Netwars in London on the threat of cyber warfare. She showed how easy it is for governments and corporations to capture our personal information without us even realising.Thiele said that the US budget for cyber security was US$67 billion in 2013 and will double by 2016. Much of this money is wasted and doesn't protect online infrastructure. This fact doesn’t worry the multinationals making a killing from the gross exaggeration of fear that permeates the public domain.

Wikileaks understands this reality better than most. Founder Julian Assange and investigative editor Sarah Harrison both remain in legal limbo. I spent time with Assange in his current home at the Ecuadorian embassy in London last week, where he continues to work, release leaks, and fight various legal battles. He hopes to resolve his predicament soon.At the Centre for Investigative Journalism conference, Harrison stressed the importance of journalists who work with technologists to best report the NSA stories. “It’s no accident”, she said, “that some of the best stories on the NSA are in Germany, where there’s technical assistance from people like Jacob Appelbaum.” A core Wikileaks belief, she stressed, is releasing all documents in their entirety, something the group criticised the news site The Intercept for not doing on a recent story. “The full archive should always be published”, Harrison said.

With 8m documents on its website after years of leaking, the importance of publishing and maintaining source documents for the media, general public and court cases can’t be under-estimated. “I see Wikileaks as a library”, Assange said. “We’re the librarians who can’t say no.”With evidence that there could be a second NSA leaker, the time for more aggressive reporting is now. As Binney said: “I call people who are covering up NSA crimes traitors”.

The Defense Department, through its Defense Advanced Research Projects Agency (DARPA), started funding academic and commercial research into speech recognition in the early 1970s. What emerged were several systems to turn speech into text, all of which slowly but gradually improved as they were able to work with more data and at faster speeds. In a brief interview, Dan Kaufman, director of DARPA’s Information Innovation Office, indicated that the government’s ability to automate transcription is still limited. Kaufman says that automated transcription of phone conversation is “super hard,” because “there’s a lot of noise on the signal” and “it’s informal as hell.”

A 2008 document from the Snowden archive shows that  transcribing news broadcasts was already working well seven years ago, using a program called Enhanced Video Text and Audio Processing: (U//FOUO) EViTAP is a fully-automated news monitoring tool. The key feature of this Intelink-SBU-hosted tool is that it analyzes news in six languages, including Arabic, Mandarin Chinese, Russian, Spanish, English, and Farsi/Persian. “How does it work?” you may ask. It integrates Automatic Speech Recognition (ASR) which provides transcripts of the spoken audio. Next, machine translation of the ASR transcript translates the native language transcript to English. Voila! Technology is amazing. A version of the system the NSA uses is now even available commercially.

But even then, a newer, more sophisticated product was already being rolled out by the NSA’s Human Language Technology (HLT) program office. The new system, called VoiceRT, was first introduced in Baghdad, and “designed to index and tag 1 million cuts per day.” The goal, according to another 2006 memo, was to use voice processing technology to be able “index, tag and graph,” all intercepted communications. “Using HLT services, a single analyst will be able to sort through millions of cuts per day and focus on only the small percentage that is relevant,” the memo states. A 2009 memo from the NSA’s British partner, GCHQ, describes how “NSA have had the BBN speech-to-text system Byblos running at Fort Meade for at least 10 years. (Initially they also had Dragon.) During this period they have invested heavily in producing their own corpora of transcribed Sigint in both American English and an increasing range of other languages.” (GCHQ also noted that it had its own small corpora of transcribed voice communications, most of which happened to be “Northern Irish accented speech.”)

According to a 2011 memo, “How is Human Language Technology (HLT) Progressing?“, NSA that year deployed “HLT Labs” to Afghanistan, NSA facilities in Texas and Georgia, and listening posts in Latin America run by the Special Collection Service, a joint NSA/CIA unit that operates out of embassies and other locations. “Spanish is the most mature of our speech-to-text analytics,” the memo says, noting that the NSA and its Special Collections Service sites in Latin America, have had “great success searching for Spanish keywords.”

The Snowden archive, as searched and analyzed by The Intercept, documents extensive use of speech-to-text by the NSA to search through international voice intercepts — particularly in Iraq and Afghanistan, as well as Mexico and Latin America. For example, speech-to-text was a key but previously unheralded element of the sophisticated analytical program known as the Real Time Regional Gateway (RTRG), which started in 2005 when newly appointed NSA chief Keith B. Alexander, according to the Washington Post, “wanted everything: Every Iraqi text message, phone call and e-mail that could be vacuumed up by the agency’s powerful computers.” The Real Time Regional Gateway was credited with playing a role in “breaking up Iraqi insurgent networks and significantly reducing the monthly death toll from improvised explosive devices.” The indexing and searching of “voice cuts” was deployed to Iraq in 2006. By 2008, RTRG was operational in Afghanistan as well.

VoiceRT, in turn, was surpassed a few years after its launch. According to the intelligence community’s “Black Budget” for fiscal year 2013, VoiceRT was decommissioned and replaced in 2011 and 2012, so that by 2013, NSA could operationalize a new system. This system, apparently called SPIRITFIRE, could handle more data, faster. SPIRITFIRE would be “a more robust voice processing capability based on speech-to-text keyword search and paired dialogue transcription.”

What’s less clear from the archive is how extensively this capability is used to transcribe or otherwise index and search voice conversations that primarily involve what the NSA terms “U.S. persons.” The NSA did not answer a series of detailed questions about automated speech recognition, even though an NSA “classification guide” that is part of the Snowden archive explicitly states that “The fact that NSA/CSS has created HLT models” for speech-to-text processing as well as gender, language and voice recognition, is “UNCLASSIFIED.”

Also unclassified: The fact that the processing can sort and prioritize audio files for human linguists, and that the statistical models are regularly being improved and updated based on actual intercepts. By contrast, because they’ve been tuned using actual intercepts, the specific parameters of the systems are highly classified.

The presidentially appointed but independent Privacy and Civil Liberties Oversight Board (PCLOB) didn’t mention speech-to-text technology in its public reports. “I’m not going to get into whether any program does or does not have that capability,” PCLOB chairman David Medine told The Intercept. His board’s reports, he said, contained only information that the intelligence community agreed could be declassified.

Our goal is to highlight one of the most important ways that the government collection of telephone records is unconstitutional: it violates the First Amendment right of association. When the government gets access to the phone records of political and activist organizations and their members, it knows who is talking to whom, when, and for how long. This so-called “metadata,” especially when collected in bulk and aggregated, tracks the associations of these organizations. After all, if the government knows that you call the Unitarian Church or Calguns or People for the American Way or Students for Sensible Drug Policy regularly, it has a very good indication that you are a member and it certainly knows that you associate regularly. The law has long recognized that government access to associations can create a chilling effect—people are less likely to associate with organizations when they know the government is watching and when the government can track their associations. 

Twenty-two organizations including Unitarian church groups, gun ownership advocates, and a broad coalition of membership and political advocacy organizations filed suit against the National Security Agency for violating their First Amendment right of association by illegally collecting their call records. The coalition is represented by EFF. At the heart of First Unitarian Church of Los Angeles v. NSA is the bulk telephone records collection program that was confirmed by the publication of an order from the Foreign Intelligence Surveillance Court (FISC) in June of 2013. The Director of National Intelligence (DNI) further confirmed that this formerly secret document was authentic, and part of a broader program to collect all major telecommunications customers’ call history. The order demands wholesale collection of every call made, the location of the phone, the time of the call, the duration of the call, and other “identifying information” for every phone and call for all customers of Verizon for a period of three months. Government officials further confirmed that this was just one of series of orders issued on a rolling basis since at least 2006. First Unitarian v. NSA argues that this spying violates the First Amendment, which protects the freedom to associate and express political views as a group.

The First Amendment right of association is a well established doctrine that prevents the government “interfering with the right to peaceably assemble or prohibit the petition for a governmental redress of grievances.” The most famous case embracing it is a 1958 Supreme Court Case from the Civil Rights era called  NAACP v. Alabama. In that case the Supreme Court held that it would violate the First Amendment for the NAACP to have to turn over its membership lists in litigation. The right stems from the simple fact that the First Amendment protects the freedom to associate and express political views as a group. This constitutional protection is critical because, as the court noted “[e]ffective advocacy of both public and private points of view, particularly controversial ones, is undeniably enhanced by group association[.]” NAACP v. Alabama, 357 U.S. at 460. As another court noted: the Constitution protects freedom of association to encourage the “advancing ideas and airing grievances” Bates v. City of Little Rock, 361 U.S. 516, 522-23 (1960).

The collection and analysis of telephone records give the government a broad window into our associations. The First Amendment protects against this because, as the Supreme Court has recognized, “it may induce members to withdraw from the association and dissuade others from joining it because of fear of exposure of their beliefs shown through their associations and of the consequences of their exposure.” NAACP v. Alabama, 357 U.S. at 462-63. See also Bates, 361 U.S. at 523; Gibson v. Florida Legislative Investigation Comm., 372 U.S. 539 (1963).  Privacy in one’s associational ties is also closely linked to freedom of association: “Inviolability of privacy in group association may in many circumstances be indispensable to preservation of freedom of association, particularly where a group espouses dissident beliefs.” NAACP v. Alabama, 357 U.S. at 462. 

The Supreme Court has made clear that infringements on freedom of association may survive constitutional scrutiny only when they “serve compelling state interests, unrelated to the suppression of ideas, that cannot be achieved through means significantly less restrictive of associational freedoms.” Roberts v. United States Jaycees, 468 U.S. 609, 623 (1984); see also NAACP v. Button, 371 U.S. at 341; Knox v. SEIU, Local 1000, 132 S. Ct. 2277, 2291 (2012)  Here, the wholesale collection of telephone records of millions of innocent Americans’ communications records, and thereby collection of their associations, is massively overbroad, regardless of the government’s interest. Thus, the NSA spying program fails under the basic First Amendment tests that have been in place for over fifty years.

The NSA came under attack this week in a landmark report from a White House panel appointed to review U.S. surveillance policy. The panel noted that "encryption is an essential basis for trust on the Internet," and called for a halt to any NSA efforts to undermine it.

From RSA's earliest days, the U.S. intelligence establishment worried it would not be able to crack well-engineered public key cryptography. Martin Hellman, a former Stanford researcher who led the team that first invented the technique, said NSA experts tried to talk him and others into believing that the keys did not have to be as large as they planned.

Jim Lewis, a director and senior fellow with the Center for Strategic and International Studies, adds that the Chinese view state-sponsored espionage a little differently than the U.S. does. Both countries believe in espionage for national security purposes, but the Chinese argue that such spying might include the theft of commercial secrets. “The Chinese will tell you that stealing technology and business secrets is a way of building their economy, and that this is important for national security,” says Lewis, who has helped oversee meetings between the U.S. and the Chinese, including officers in the PLA. “I’ve been in the room when they’ve said that. The last time was when a PLA colonel said: ‘In the U.S., military espionage is heroic and economic espionage is a crime. In China, the line is not that clear.’” But here in the United States, we now know, the NSA may blur other lines in the name of national security. Segal says that although he, as an American, believes the U.S. government is on stronger ethical ground than the Chinese, other nations are beginning to question its motives. “The U.S has to convince other countries that our type of intelligence gathering is different,” he says. “I don’t think that the Brazils and the Indias and the Indonesias and the South Africas are convinced. That’s a big problem for us.”

The thing to realize, as the revelations of NSA snooping continue to pour out, is that everyone deserves scrutiny — the U.S government and its allies, as well as the Chinese and others you may be more likely to view with skepticism. “All big countries,” Lewis says, “are going to try and do this.”

The portion of the contract obtained by Muckrock where the cost of the subscription is listed has been redacted, but a Vupen hacker who spoke to Greenberg last year said deals in the five-figures wasn't uncommon. "People seem surprised to discover that major government agencies are acquiring Vupen's vulnerability intelligence," Bekrar wrote in an email to Information Week's Matthew Schwartz after the NSA contract with his signature was published. "There is no news here, governments need to leverage the most detailed and advanced vulnerability research to protect their infrastructures and citizens against adversaries." Critics of Vupen and its competitors see government-waged cyber-operations in a different light, however. Christopher Soghoian of the American Civil Liberties Union's Speech, Privacy and Technology Project has spoken outright against companies that sell exploits and have equated the computer codes being sold for big money as a new sort of underground arms trade fueling an international, online battle. To Greenberg last year, Soghoian described Vupen as  a “modern-day merchant of death” selling “the bullets for cyberwar," and upon publishing of the NSA contract called the company a “cyber weapon merchant.” The NSA is a customer of French 0-day cyber weapon merchant VUPEN, FOIA docs reveal: (via @ramdac & @MuckRockNews) https://t.co/OPJ82miK3c — Christopher Soghoian (@csoghoian) September 16, 2013

The secret documents do not reveal the precise targets of so much espionage by the NSA — and possibly its Canadian partner — during the Toronto summit. But both the U.S. and Canadian intelligence agencies have been implicated with their British counterpart in hacking the phone calls and emails of foreign politicians and diplomats attending the G20 summit in London in 2009 — a scant few months before the Toronto gathering of the same world leaders.

The spying at the Toronto summit in 2010 fits a pattern of economic and political espionage by the powerful U.S. intelligence agency and its partners such as Canada. That espionage was conducted to secure meeting sites and protect leaders against terrorist threats posed by al-Qaeda but also to forward the policy goals of the United States and Canada. The G20 summit in Toronto had a lot on its agenda that would have been of acute interest to the NSA and Canada.

The world was still struggling to climb out of the great recession of 2008. Leaders were debating a wide array of possible measures including a global tax on banks, an idea strongly opposed by both the U.S. and Canadian governments. That notion was eventually scotched. The secret NSA documents list all the main agenda items for the G20 in Toronto — international development, banking reform, countering trade protectionism, and so on — with the U.S. snooping agency promising to support "U.S. policy goals." Whatever the intelligence goals of the NSA during the Toronto summit, international security experts question whether the NSA spying operation at the G20 in Toronto was even legal.

"If CSEC tasked NSA to conduct spying activities on Canadians within Canada that CSEC itself was not authorized to take, then I am comfortable saying that would be an unlawful undertaking by CSEC," says Craig Forcese, an expert in national security at University of Ottawa's faculty of law. By law, CSEC cannot target anyone in Canada without a warrant, including world leaders and foreign diplomats at a G20 summit. But, the Canadian eavesdropping agency is also prohibited by international agreement from getting the NSA to do the spying or anything that would be illegal for CSEC.

The NSA warns the more likely security threat would come from "issue-based extremists" conducting acts of vandalism. They got that right. Protest marches by about 10,000 turned the Toronto G20 into an historic melee of arrests by more than 20,000 police in what would become one of the largest and most expensive security operations in Canadian history. By the time the tear gas had cleared and the investigations were complete, law enforcement agencies stood accused of mass-violations of civil rights. Add to that dubious legacy illegal spying by an American intelligence agency with the blessing of the Canadian government.

One of the president’s commission recommendations is that the NSA cease holding our phone records and demand that the private phone companies retain those records instead – for the NSA to access as it wishes. This is supposed to be reform?   The president will make a speech this Friday to tell the rest of us which of the suggestions made by his own commission he will decide to implement. Congress has no problem with that. Rep. Adam B. Schiff (D-Calif.) admitted last week that Congress has no intention of asserting itself in the process. “It’s my hope that [Obama will] do as much as he can through the executive process because the legislative process will be difficult, perilous and long.”

Senator Church famously said back in 1975: In the need to develop a capacity to know what potential enemies are doing, the United States government has perfected a technological capability that enables us to monitor the messages that go through the air… We must know, at the same time, that capability at any time could be turned around on the American people, and no American would have any privacy left... There would be no place to hide…. I know the capacity that is there to make tyranny total in America, and we must see to it that this agency and all agencies that possess this technology operate within the law and under proper supervision so that we never cross over that abyss. That is the abyss from which there is no return.” Have we reached that point? Let us hope not. Real reform begins with the repeal of the PATRIOT Act and of the 2001 Authorization for the use of military force. If we keep our eye on that goal and not allow ourselves to become distracted with the president’s phony commissions we might force Congress to listen.

The National Security Agency last year significantly expanded its cooperative relationship with the Saudi Ministry of Interior, one of the world’s most repressive and abusive government agencies. An April 2013 top secret memo provided by NSA whistleblower Edward Snowden details the agency’s plans “to provide direct analytic and technical support” to the Saudis on “internal security” matters. The Saudi Ministry of Interior—referred to in the document as MOI— has been condemned for years as one of the most brutal human rights violators in the world. In 2013, the U.S. State Department reported that “Ministry of Interior officials sometimes subjected prisoners and detainees to torture and other physical abuse,” specifically mentioning a 2011 episode in which MOI agents allegedly “poured an antiseptic cleaning liquid down [the] throat” of one human rights activist. The report also notes the MOI’s use of invasive surveillance targeted at political and religious dissidents. But as the State Department publicly catalogued those very abuses, the NSA worked to provide increased surveillance assistance to the ministry that perpetrated them. The move is part of the Obama Administration’s increasingly close ties with the Saudi regime; beyond the new cooperation with the MOI, the memo describes “a period of rejuvenation” for the NSA’s relationship with the Saudi Ministry of Defense. In general, U.S. support for the Saudi regime is long-standing. One secret 2007 NSA memo lists Saudi Arabia as one of four countries where the U.S. “has [an] interest in regime continuity.”

The nine Tor directory servers receive especially close monitoring from the NSA's spying software, which states the "goal is to find potential Tor clients connecting to the Tor directory servers." Tor clients linking into the directory servers are also logged."This shows that Tor is working well enough that Tor has become a target for the intelligence services," said Sebastian Hahn, who runs one of the key Tor servers. "For me this means that I will definitely go ahead with the project.”

While the German reporting team has published part of the XKeyscore scripting code, it doesn't say where it comes from. NSA whistleblower Edward Snowden would be a logical pick, but security experts are not so sure."I do not believe that this came from the Snowden documents," said security guru Bruce Schneier. "I also don't believe the TAO catalog came from the Snowden documents. I think there's a second leaker out there."If so, the NSA is in for much more scrutiny than it ever expected.

“For the past decade, N.S.A. has led an aggressive, multipronged effort to break widely used Internet encryption technologies,” said a 2010 memo describing a briefing about N.S.A. accomplishments for employees of its British counterpart, Government Communications Headquarters, or GCHQ. “Cryptanalytic capabilities are now coming online. Vast amounts of encrypted Internet data which have up till now been discarded are now exploitable.”

Some of the agency’s most intensive efforts have focused on the encryption in universal use in the United States, including Secure Sockets Layer, or SSL; virtual private networks, or VPNs; and the protection used on fourth-generation, or 4G, smartphones. Many Americans, often without realizing it, rely on such protection every time they send an e-mail, buy something online, consult with colleagues via their company’s computer network, or use a phone or a tablet on a 4G network.

For at least three years, one document says, GCHQ, almost certainly in collaboration with the N.S.A., has been looking for ways into protected traffic of popular Internet companies: Google, Yahoo, Facebook and Microsoft’s Hotmail. By 2012, GCHQ had developed “new access opportunities” into Google’s systems, according to the document. (Google denied giving any government access and said it had no evidence its systems had been breached).

Paul Kocher, a leading cryptographer who helped design the SSL protocol, recalled how the N.S.A. lost the heated national debate in the 1990s about inserting into all encryption a government back door called the Clipper Chip. “And they went and did it anyway, without telling anyone,” Mr. Kocher said. He said he understood the agency’s mission but was concerned about the danger of allowing it unbridled access to private information.

The documents are among more than 50,000 shared by The Guardian with The New York Times and ProPublica, the nonprofit news organization. They focus on GCHQ but include thousands from or about the N.S.A. Intelligence officials asked The Times and ProPublica not to publish this article, saying it might prompt foreign targets to switch to new forms of encryption or communications that would be harder to collect or read. The news organizations removed some specific facts but decided to publish the article because of the value of a public debate about government actions that weaken the most powerful privacy tools.

The files show that the agency is still stymied by some encryption, as Mr. Snowden suggested in a question-and-answer session on The Guardian’s Web site in June. “Properly implemented strong crypto systems are one of the few things that you can rely on,” he said, though cautioning that the N.S.A. often bypasses the encryption altogether by targeting the computers at one end or the other and grabbing text before it is encrypted or after it is decrypted.

Because strong encryption can be so effective, classified N.S.A. documents make clear, the agency’s success depends on working with Internet companies — by getting their voluntary collaboration, forcing their cooperation with court orders or surreptitiously stealing their encryption keys or altering their software or hardware.

At Microsoft, as The Guardian has reported, the N.S.A. worked with company officials to get pre-encryption access to Microsoft’s most popular services, including Outlook e-mail, Skype Internet phone calls and chats, and SkyDrive, the company’s cloud storage service.

Simultaneously, the N.S.A. has been deliberately weakening the international encryption standards adopted by developers. One goal in the agency’s 2013 budget request was to “influence policies, standards and specifications for commercial public key technologies,” the most common encryption method. Cryptographers have long suspected that the agency planted vulnerabilities in a standard adopted in 2006 by the National Institute of Standards and Technology and later by the International Organization for Standardization, which has 163 countries as members. Classified N.S.A. memos appear to confirm that the fatal weakness, discovered by two Microsoft cryptographers in 2007, was engineered by the agency. The N.S.A. wrote the standard and aggressively pushed it on the international group, privately calling the effort “a challenge in finesse.” “Eventually, N.S.A. became the sole editor,” the memo says.

But the agencies’ goal was to move away from decrypting targets’ tools one by one and instead decode, in real time, all of the information flying over the world’s fiber optic cables and through its Internet hubs, only afterward searching the decrypted material for valuable intelligence. A 2010 document calls for “a new approach for opportunistic decryption, rather than targeted.” By that year, a Bullrun briefing document claims that the agency had developed “groundbreaking capabilities” against encrypted Web chats and phone calls. Its successes against Secure Sockets Layer and virtual private networks were gaining momentum.

Ladar Levison, the founder of Lavabit, wrote a public letter to his disappointed customers, offering an ominous warning. “Without Congressional action or a strong judicial precedent,” he wrote, “I would strongly recommend against anyone trusting their private data to a company with physical ties to the United States.”