Group items tagged

That language could allow for surveillance of academics, journalists and human rights researchers. A Swiss academic who has information on the German government’s position in the run-up to an international trade negotiation, for instance, could be targeted if the government has determined there is a foreign-intelligence need for that information. If a U.S. college professor e-mails the Swiss professor’s e-mail address or phone number to a colleague, the American’s e-mail could be collected as well, under the program’s court-approved rules

Still, some lawmakers are concerned that the potential for intrusions on Americans’ privacy has grown under the 2008 law because the government is intercepting not just communications of its targets but communications about its targets as well. The expansiveness of the foreign-powers certification increases that concern.

In a 2011 FISA court opinion, a judge using an NSA-provided sample estimated that the agency could be collecting as many as 46,000 wholly domestic e-mails a year that mentioned a particular target’s e-mail address or phone number, in what is referred to as “about” collection. “When Congress passed Section 702 back in 2008, most members of Congress had no idea that the government was collecting Americans’ communications simply because they contained a particular individual’s contact information,” Sen. Ron Wyden (D-Ore.), who has co-sponsored ­legislation to narrow “about” collection authority, said in an e-mail to The Washington Post. “If ‘about the target’ collection were limited to genuine national security threats, there would be very little privacy impact. In fact, this collection is much broader than that, and it is scooping up huge amounts of Americans’ wholly domestic communications.”

The only reason the court has oversight of the NSA program is that Congress in 2008 gave the government a new authority to gather intelligence from U.S. companies that own the Internet cables running through the United States, former officials noted. Edgar, the former privacy officer at the Office of the Director of National Intelligence, said ultimately he believes the authority should be narrowed. “There are valid privacy concerns with leaving these collection decisions entirely in the executive branch,” he said. “There shouldn’t be broad collection, using this authority, of foreign government information without any meaningful judicial role that defines the limits of what can be collected.”

The Freedom Act ultimately sped to passage in the House on May 22 by a bipartisan 303-121 vote. NSA advocates who had blasted its earlier version as hazardous to national security dropped their objections – largely because they had no more reason.Accordingly, the compromise language caused civil libertarians and technology groups not just to abandon the Freedom Act that they had long championed, but to question whether it actually banned bulk data collection. The government could acquire call-records data up to two degrees of separation from any "reasonable articulable suspicion" of wrongdoing, potentially representing hundreds or thousands of people on a single judicial order." That was not all.

"As the bill stands today, it could still permit the collection of email records from everyone who uses a particular email service," warned a Google legislative action alert after the bill passed the House. In a recent statement, cloud-storage firm Tresorit lamented that "there still has been no real progress in achieving truly effective security for consumer and corporate information."No one familiar with the negotiations alleges the NSA or its allies broke the law by amending the bill during the technical-fix period. But it is unusual for substantive changes to be introduced secretly after a bill has cleared committee and before its open debate by the full Senate or House."It is not out of order, but major changes in substance are rare, and appropriately so," said Norman Ornstein, an expert on congressional procedure at the American Enterprise Institute.Steve Aftergood, an intelligence policy analyst at the Federation of American Scientists, said the rewrites to the bill were an "invitation to cynicism."

"There does seem to be a sort of gamesmanship to it. Why go through all the troubling of crafting legislation, enlisting support and co-sponsorship, and adopting compromises if the bill is just going to be rewritten behind closed doors anyway?" Aftergood said.

Civil libertarians and activists now hope to strengthen the bill in the Senate. Its chief sponsor, Patrick Leahy of Vermont, vowed to take it up this month, and to push for "meaningful reforms" he said he was "disappointed" the House excluded. Obama administration officials will testify in the Senate intelligence committee about the bill on Thursday afternoon, the first anniversary of the Guardian's disclosure of bulk domestic phone records collection. That same day, Reddit, Imgur and other large websites will stage an online "Reset The Net" protest of NSA bulk surveillance.But the way the bill "morphed behind the scenes," as Lofgren put it, points to the obstacles such efforts face. It also points to a continuing opportunity for the NSA to say that Congress has actually blessed widespread data collection – a claim made after the Snowden leaks, despite most members of Congress and the public not knowing that NSA and the Fisa court secretly reinterpreted the Patriot Act in order to collect all US phone records.

In America, Verizon and AT&T have published data, but only on their domestic operations. Deutsche Telekom in Germany and Telstra in Australia have also broken ground at home. Vodafone is the first to produce a global survey.

Snowden, the National Security Agency whistleblower, joined Google, Reddit, Mozilla and other tech firms and privacy groups on Thursday to call for a strengthening of privacy rights online in a "Reset the net" campaign.Twelve months after revelations about the scale of the US government's surveillance programs were first published in the Guardian and the Washington Post, Snowden said: "One year ago, we learned that the internet is under surveillance, and our activities are being monitored to create permanent records of our private lives – no matter how innocent or ordinary those lives might be. Today, we can begin the work of effectively shutting down the collection of our online communications, even if the US Congress fails to do the same."

“Collecting an inventory [like this] on world networks has big ramifications,” Nohl said, because it allows the NSA to track and circumvent upgrades in encryption technology used by cellphone companies to shield calls and texts from eavesdropping. Evidence that the agency has deliberately plotted to weaken the security of communication infrastructure, he added, was particularly alarming. “Even if you love the NSA and you say you have nothing to hide, you should be against a policy that introduces security vulnerabilities,” Nohl said, “because once NSA introduces a weakness, a vulnerability, it’s not only the NSA that can exploit it.”

The AURORAGOLD operation is carried out by specialist NSA surveillance units whose existence has not been publicly disclosed: the Wireless Portfolio Management Office, which defines and carries out the NSA’s strategy for exploiting wireless communications, and the Target Technology Trends Center, which monitors the development of new communication technology to ensure that the NSA isn’t blindsided by innovations that could evade its surveillance reach. The center’s logo is a picture of the Earth overshadowed by a large telescope; its motto is “Predict – Plan – Prevent.”

The NSA documents reveal that, as of May 2012, the agency had collected technical information on about 70 percent of cellphone networks worldwide—701 of an estimated 985—and was maintaining a list of 1,201 email “selectors” used to intercept internal company details from employees. (“Selector” is an agency term for a unique identifier like an email address or phone number.) From November 2011 to April 2012, between 363 and 1,354 selectors were “tasked” by the NSA for surveillance each month as part of AURORAGOLD, according to the documents. The secret operation appears to have been active since at least 2010.

By covertly monitoring GSMA working groups in a bid to identify and exploit security vulnerabilities, the NSA has placed itself into direct conflict with the mission of the National Institute for Standards and Technology, or NIST, the U.S. government agency responsible for recommending cybersecurity standards in the United States. NIST recently handed out a grant of more than $800,000 to GSMA so that the organization could research ways to address “security and privacy challenges” faced by users of mobile devices. The revelation that the trade group has been targeted for surveillance may reignite deep-seated tensions between NIST and NSA that came to the fore following earlier Snowden disclosures. Last year, NIST was forced to urge people not to use an encryption standard it had previously approved after it emerged NSA had apparently covertly worked to deliberately weaken it.

The NSA focuses on intercepting obscure but important technical documents circulated among the GSMA’s members known as “IR.21s.” Most cellphone network operators share IR.21 documents among each other as part of agreements that allow their customers to connect to foreign networks when they are “roaming” overseas on a vacation or a business trip. An IR.21, according to the NSA documents, contains information “necessary for targeting and exploitation.” The details in the IR.21s serve as a “warning mechanism” that flag new technology used by network operators, the NSA’s documents state. This allows the agency to identify security vulnerabilities in the latest communication systems that can be exploited, and helps efforts to introduce new vulnerabilities “where they do not yet exist.” The IR.21s also contain details about the encryption used by cellphone companies to protect the privacy of their customers’ communications as they are transmitted across networks. These details are highly sought after by the NSA, as they can aid its efforts to crack the encryption and eavesdrop on conversations.

One of the prime targets monitored under the AURORAGOLD program is the London-headquartered trade group, the GSM Association, or the GSMA, which represents the interests of more than 800 major cellphone, software, and internet companies from 220 countries. The GSMA’s members include U.S.-based companies such as Verizon, AT&T, Sprint, Microsoft, Facebook, Intel, Cisco, and Oracle, as well as large international firms including Sony, Nokia, Samsung, Ericsson, and Vodafone. The trade organization brings together its members for regular meetings at which new technologies and policies are discussed among various “working groups.” The Snowden files reveal that the NSA specifically targeted the GSMA’s working groups for surveillance.

Last year, the Washington Post reported that the NSA had already managed to break the most commonly used cellphone encryption algorithm in the world, known as A5/1. But the information collected under AURORAGOLD allows the agency to focus on circumventing newer and stronger versions of A5 cellphone encryption, such as A5/3. The documents note that the agency intercepts information from cellphone operators about “the type of A5 cipher algorithm version” they use, and monitors the development of new algorithms in order to find ways to bypass the encryption. In 2009, the British surveillance agency Government Communications Headquarters conducted a similar effort to subvert phone encryption under a project called OPULENT PUP, using powerful computers to perform a “crypt attack” to penetrate the A5/3 algorithm, secret memos reveal. By 2011, GCHQ was collaborating with the NSA on another operation, called WOLFRAMITE, to attack A5/3 encryption. (GCHQ declined to comment for this story, other than to say that it operates within legal parameters.)

The extensive attempts to attack cellphone encryption have been replicated across the Five Eyes surveillance alliance. Australia’s top spy agency, for instance, infiltrated an Indonesian cellphone company and stole nearly 1.8 million encryption keys used to protect communications, the New York Times reported in February.

The NSA’s documents show that it focuses on collecting details about virtually all technical standards used by cellphone operators, and the agency’s efforts to stay ahead of the technology curve occasionally yield significant results. In early 2010, for instance, its operatives had already found ways to penetrate a variant of the newest “fourth generation” smartphone-era technology for surveillance, years before it became widely adopted by millions of people in dozens of countries. The NSA says that its efforts are targeted at terrorists, weapons proliferators, and other foreign targets, not “ordinary people.” But the methods used by the agency and its partners to gain access to cellphone communications risk significant blowback. According to Mikko Hypponen, a security expert at Finland-based F-Secure, criminal hackers and foreign government adversaries could be among the inadvertent beneficiaries of any security vulnerabilities or encryption weaknesses inserted by the NSA into communication systems using data collected by the AURORAGOLD project.

Vines, the NSA spokeswoman, told The Intercept that the agency was committed to ensuring an “open, interoperable, and secure global internet.” “NSA deeply values these principles and takes great care to honor them in the performance of its lawful foreign-intelligence mission,” Vines said.

Documents published with this article: AURORAGOLD – Project Overview AURORAGOLD Working Group IR.21 – A Technology Warning Mechanism AURORAGOLD – Target Technology Trends Center support to WPMO NSA First-Ever Collect of High-Interest 4G Cellular Signal AURORAGOLD Working Aid WOLFRAMITE Encryption Attack OPULENT PUP Encryption Attack NSA/GCHQ/CSEC Network Tradecraft Advancement Team

The Obama administration has strained to reconcile the vast counter-terrorism bureaucracy with its policy declarations that the US is not at war with Islam and has attempted, with mixed results, to cultivate a less militarized and security-focused relationship with US Muslims, often preferring the term "countering violent extremism" over "counter-terrorism".In some anti-Islam circles, the removal of the instructional material is infamous and considered evidence of an administration capitulation to Islam – the exact opposite of the concern raised by the Intercept on Wednesday. The Intercept report, by former Guardian columnist Glenn Greenwald, Murtaza Hussain and Josh Meyer, suggested a persistent counter-terrorism atmosphere in which the mixture of Muslim heritage or faith and political activism attracted the scrutiny of US security agencies, despite first amendment protections. It presented the cases of five American activists and attorneys of Muslim heritage who appear to have been targeted for surveillance, at least between 2002 and 2008. None have been charged with a crime.

US Muslim leaders and civil rights groups reacted with fury to the Intercept report.A coalition of 44 civil rights organizations wrote Obama on Wednesday to request a meeting with him, attorney general Eric Holder and FBI director James Comey. "In short, the government’s domestic counterterrorism policies treat entire minority communities as suspect, and American Muslims have borne the brunt of government suspicion, stigma and abuse," reads an open letter issued by a coalition that includes the ACLU, the Presbyterian Church (USA), the Arab-American Institute, and dozens of others.In a statement, the law firm Muslim Advocates said the spying, apparently conducted between 2002 and 2008, "confirms the worst fears of American Muslims"."The federal government has targeted Americans, even those who have served their country in the military and government, simply because of their faith or religious heritage," the group said. "The report clearly documents how biased training by the FBI leads to biased surveillance."

Vincent Warren, executive director of the Center for Constitutional Rights, issued a statement comparing the surveillance of Nihad Awad of the Council on American-Islamic Relations, named by the Intercept as one of the five surveillance targets, to the infamous surveillance of the Civil Rights Movement."The NSA’s surveillance of Nihad Awad and CAIR fits the same pattern as the FBI surveillance of Martin Luther King Jr, Ella Baker, Jesse Jackson, Malcolm X, and other leaders of the civil rights movement. Then it was based on manufactured suspicions of associations with the Communist party. Now it is seemingly based on unproven claims of tangential associations with Hamas," Warren said.

Ever since the Snowden reporting began and public opinion (in both the U.S. and globally) began radically changing, the White House’s strategy has been obvious. It’s vintage Obama: Enact something that is called “reform”—so that he can give a pretty speech telling the world that he heard and responded to their concerns—but that in actuality changes almost nothing, thus strengthening the very system he can pretend he “changed.” That’s the same tactic as Silicon Valley, which also supported this bill: Be able to point to something called “reform” so they can trick hundreds of millions of current and future users around the world into believing that their communications are now safe if they use Facebook, Google, Skype and the rest. In pretty much every interview I’ve done over the last year, I’ve been asked why there haven’t been significant changes from all the disclosures. I vehemently disagree with the premise of the question, which equates “U.S. legislative changes” with “meaningful changes.” But it has been clear from the start that U.S. legislation is not going to impose meaningful limitations on the NSA’s powers of mass surveillance, at least not fundamentally. Those limitations are going to come from—are now coming from —very different places:

All of that illustrates what is, to me, the most important point from all of this: the last place one should look to impose limits on the powers of the U.S. government is . . . the U.S. government. Governments don’t walk around trying to figure out how to limit their own power, and that’s particularly true of empires. The entire system in D.C. is designed at its core to prevent real reform. This Congress is not going to enact anything resembling fundamental limits on the NSA’s powers of mass surveillance. Even if it somehow did, this White House would never sign it. Even if all that miraculously happened, the fact that the U.S. intelligence community and National Security State operates with no limits and no oversight means they’d easily co-opt the entire reform process. That’s what happened after the eavesdropping scandals of the mid-1970s led to the establishment of congressional intelligence committees and a special FISA “oversight” court—the committees were instantly captured by putting in charge supreme servants of the intelligence community like Senators Dianne Feinstein and Chambliss, and Congressmen Mike Rogers and “Dutch” Ruppersberger, while the court quickly became a rubber stamp with subservient judges who operate in total secrecy.

There is a real question about whether the defeat of this bill is good, bad, or irrelevant. To begin with, it sought to change only one small sliver of NSA mass surveillance (domestic bulk collection of phone records under section 215 of the Patriot Act) while leaving completely unchanged the primary means of NSA mass surveillance, which takes place under section 702 of the FISA Amendments Act, based on the lovely and quintessentially American theory that all that matters are the privacy rights of Americans (and not the 95 percent of the planet called “non-Americans”). There were some mildly positive provisions in the USA Freedom Act: the placement of “public advocates” at the FISA court to contest the claims of the government; the prohibition on the NSA holding Americans’ phone records, requiring instead that they obtain FISA court approval before seeking specific records from the telecoms (which already hold those records for at least 18 months); and reducing the agency’s “contact chaining” analysis from three hops to two. One could reasonably argue (as the ACLU and EFF did) that, though woefully inadequate, the bill was a net-positive as a first step toward real reform, but one could also reasonably argue, as Marcy Wheeler has with characteristic insight, that the bill is so larded with ambiguities and fundamental inadequacies that it would forestall better options and advocates for real reform should thus root for its defeat.

1) Individuals refusing to use internet services that compromise their privacy.

2) Other countries taking action against U.S. hegemony over the internet.

4) Greater individual demand for, and use of, encryption.

3) U.S. court proceedings.

The “USA Freedom Act”—which its proponents were heralding as “NSA reform” despite its suffocatingly narrow scope—died in the august U.S. Senate last night when it attracted only 58 of the 60 votes needed to close debate and move on to an up-or-down vote. All Democratic and independent senators except one (Bill Nelson of Florida) voted in favor of the bill, as did three tea-party GOP Senators (Ted Cruz, Mike Lee, and Dean Heller). One GOP Senator, Rand Paul, voted against it on the ground that it did not go nearly far enough in reining in the NSA. On Monday, the White House had issued a statement “strongly supporting” the bill.

Snowden’s name surfaced as top intelligence officials in the Obama administration and Congress pushed back against the journalists responsible for revealing the existence of sensitive surveillance programs and called for an investigation into the leaks.Clapper, in an interview with NBC that aired Saturday night, condemned the leaker’s actions but also sought to spotlight the journalists who first reported the programs, calling their disclosures irresponsible and full of “hyperbole.” Earlier Saturday, he issued a statement accusing the media of a “rush to publish.”“For me, it is literally — not figuratively — literally gut-wrenching to see this happen because of the huge, grave damage it does to our intelligence capabilities,” Clapper said.

A chief critic of the efforts, Sen. Rand Paul (R-Ky.), said he is considering filing a lawsuit against the government and called on 10 million Americans to join in.“I’m going to be asking all the Internet providers and all of the phone companies, ask your customers to join me in a class-action lawsuit,” Paul said on “Fox News Sunday.”

Today’s report suggests that the moment the PAA was the law of the land, the NSA started using it to obtain unfettered access to the servers of the nation’s leading online services. To comply with the requirement that the government not target Americans, PRISM searches are reportedly “designed to produce at least 51 percent confidence in a target’s ‘foreignness’” — the lowest conceivable standard. PRISM training materials reportedly instruct users that if searches happen to turn up the private information of Americans, “it’s nothing to worry about.”

Retroactive immunity for telecom companies dominated the 2008 debate, overshadowing the more important issue of the sweeping new powers that Congress had just granted to the executive branch. When Congress finally passed the FISA Amendments Act in July 2008, it included both immunity and a four-year extension of the government’s warrantless spying powers. But few members of Congress realized the breadth of the surveillance powers they were effectively approving.

The senators said they were seeking public answers to the following questions in order to give the American people the information they need to conduct an informed public debate. The specific questions include:• How long has the NSA used Patriot Act authorities to engage in bulk collection of Americans' records? Was this collection underway when the law was reauthorized in 2006?• Has the NSA used USA Patriot Act authorities to conduct bulk collection of any other types of records pertaining to Americans, beyond phone records?• Has the NSA collected or made any plans to collect Americans' cell-site location data in bulk?• Have there been any violations of the court orders permitting this bulk collection, or of the rules governing access to these records? If so, please describe these violations.

They ask Clapper to publicly provide information about the duration and scope of the program and provide examples of its effectiveness in providing unique intelligence, if such examples exist.The senators also expressed their concern that the program itself has a significant impact on the privacy of law-abiding Americans and that the Patriot Act could be used for the bulk collection of records beyond phone metadata."The Patriot Act's 'business records' authority can be used to give the government access to private financial, medical, consumer and firearm sales records, among others," said a press statement.In addition to raising concerns about the law's scope, the senators noted that keeping the official interpretation of the law secret and the instances of misleading public statements from executive branch officials prevented the American people from having an informed public debate about national security and domestic surveillance.

A bipartisan group of 26 US senators has written to intelligence chiefs to complain that the administration is relying on a "secret body of law" to collect massive amounts of data on US citizens.The senators accuse officials of making misleading statements and demand that the director of national intelligence James Clapper answer a series of specific questions on the scale of domestic surveillance as well as the legal justification for it.In their strongly-worded letter to Clapper, the senators said they believed the government may be misinterpreting existing legislation to justify the sweeping collection of telephone and internet data revealed by the Guardian."We are concerned that by depending on secret interpretations of the Patriot Act that differed from an intuitive reading of the statute, this program essentially relied for years on a secret body of law," they say."This and misleading statements by intelligence officials have prevented our constituents from evaluating the decisions that their government was making, and will unfortunately undermine trust in government more broadly."

Then, there's warrantless wiretapping. Many media reports describe that Comey's defiant stand at Ashcroft's bedside was in opposition to the warrantless wiretapping of Americans international communications. But we simply do not know exactly what Comey opposed, or why or what reforms he believed brought the secret program within the rule of law. We do, however, know that Comey was read into the program in January 2004.While, to his credit, he immediately began raising concerns, the program was still in existence when the New York Times exposed it in December 2005. This was a year and a half after Comey's hospital showdown with Gonzales and Card. In fact, the warrantless wiretapping program was supported by a May 2004 legal opinion (pdf) produced by the Justice Department's Office of Legal Counsel and signed off by Comey, which replaced the 2001 legal opinion Comey had problems with.This, of course, raises the question: just what illegal surveillance program did Comey oppose so much he would resign over it? Last weekend, the Washington Post provided a new theory: the Marina program, which collects internet metadata. Now, the Senate has an opportunity to end the theorizing and find out what exactly Comey objected to. It's a line of questioning that senators should focus doggedly on, in light of the recent revelations in the Post and the Guardian.

The final stain on Comey's record was his full-throated defense of the indefinite military detention of an American citizen arrested on American soil. In a June 2004 press conference, Comey told of Jose Padilla, an alleged al-Qaida member accused of plotting to detonate a dirty bomb as well as blow up apartment buildings in an American city. By working for al-Qaida, Padilla, Comey argued, could be deprived of a lawyer and indefinitely detained as an enemy combatant on a military brig off the South Carolina coast for the purpose of extracting intelligence out of him. It turned out that Padilla was never charged with the list of crimes and criminal associations pinned on him by Comey that day. When Padilla was finally convicted – in a federal court – in August 2007, it wasn't for plotting dirty bomb attacks or blowing up apartment buildings. Rather, he was convicted of material support of terrorism overseas. During his indefinite military detention, Padilla was tortured.

Everyone has a backstory, and the confirmation process should ensure the American public hears all relevant background information, both good and bad, when Comey appears before the Senate. Senators should insist that Comey explain his role during the Bush era and repudiate policies he endorsed on torture, indefinite detention, and illegal surveillance.The new FBI director will be around for the next decade. We need one who will respect the constitution and the rule of law; not one who will use discredited and illegal activities in the name of justice and safety.

When it came to judging the necessity and proportionality of what they were allowed to look for, would-be American users were told it was "your call".The Guardian understands that a total of 850,000 NSA employees and US private contractors with top secret clearance had access to GCHQ databases.

For the 2 billion users of the world wide web, Tempora represents a window on to their everyday lives, sucking up every form of communication from the fibre-optic cables that ring the world.The NSA has meanwhile opened a second window, in the form of the Prism operation, revealed earlier this month by the Guardian, from which it secured access to the internal systems of global companies that service the internet.The GCHQ mass tapping operation has been built up over five years by attaching intercept probes to transatlantic fibre-optic cables where they land on British shores carrying data to western Europe from telephone exchanges and internet servers in north America.This was done under secret agreements with commercial companies, described in one document as "intercept partners".The papers seen by the Guardian suggest some companies have been paid for the cost of their co-operation and GCHQ went to great lengths to keep their names secret. They were assigned "sensitive relationship teams" and staff were urged in one internal guidance paper to disguise the origin of "special source" material in their reports for fear that the role of the companies as intercept partners would cause "high-level political fallout".

The GCHQ documents that the Guardian has seen illustrate a constant effort to build up storage capacity at the stations at Cheltenham, Bude and at one overseas location, as well a search for ways to maintain the agency's comparative advantage as the world's leading communications companies increasingly route their cables through Asia to cut costs. Meanwhile, technical work is ongoing to expand GCHQ's capacity to ingest data from new super cables carrying data at 100 gigabits a second. As one training slide told new users: "You are in an enviable position – have fun and make the most of it."

The categories of material have included fraud, drug trafficking and terrorism, but the criteria at any one time are secret and are not subject to any public debate. GCHQ's compliance with the certificates is audited by the agency itself, but the results of those audits are also secret.An indication of how broad the dragnet can be was laid bare in advice from GCHQ's lawyers, who said it would be impossible to list the total number of people targeted because "this would be an infinite list which we couldn't manage".There is an investigatory powers tribunal to look into complaints that the data gathered by GCHQ has been improperly used, but the agency reassured NSA analysts in the early days of the programme, in 2009: "So far they have always found in our favour".

Historically, the spy agencies have intercepted international communications by focusing on microwave towers and satellites. The NSA's intercept station at Menwith Hill in North Yorkshire played a leading role in this. One internal document quotes the head of the NSA, Lieutenant General Keith Alexander, on a visit to Menwith Hill in June 2008, asking: "Why can't we collect all the signals all the time? Sounds like a good summer project for Menwith."By then, however, satellite interception accounted for only a small part of the network traffic. Most of it now travels on fibre-optic cables, and the UK's position on the western edge of Europe gave it natural access to cables emerging from the Atlantic.

The processing centres apply a series of sophisticated computer programmes in order to filter the material through what is known as MVR – massive volume reduction. The first filter immediately rejects high-volume, low-value traffic, such as peer-to-peer downloads, which reduces the volume by about 30%. Others pull out packets of information relating to "selectors" – search terms including subjects, phone numbers and email addresses of interest. Some 40,000 of these were chosen by GCHQ and 31,000 by the NSA. Most of the information extracted is "content", such as recordings of phone calls or the substance of email messages. The rest is metadata.

"The criteria are security, terror, organised crime. And economic well-being. There's an auditing process to go back through the logs and see if it was justified or not. The vast majority of the data is discarded without being looked at … we simply don't have the resources."However, the legitimacy of the operation is in doubt. According to GCHQ's legal advice, it was given the go-ahead by applying old law to new technology. The 2000 Regulation of Investigatory Powers Act (Ripa) requires the tapping of defined targets to be authorised by a warrant signed by the home secretary or foreign secretary.However, an obscure clause allows the foreign secretary to sign a certificate for the interception of broad categories of material, as long as one end of the monitored communications is abroad. But the nature of modern fibre-optic communications means that a proportion of internal UK traffic is relayed abroad and then returns through the cables.

British spy agency collects and stores vast quantities of global email messages, Facebook posts, internet histories and calls, and shares them with NSA, latest documents from Edward Snowden reveal

The N.S.A.’s protections for attorney-client conversations are narrowly crafted, said Stephen Gillers, an expert on legal ethics at New York University’s School of Law. The agency is barred from sharing with prosecutors intercepted attorney-client communications involving someone under indictment in the United States, according to previously disclosed N.S.A. rules. But the agency may still use or share the information for intelligence purposes. Andrew M. Perlman, a Suffolk University law professor who specializes in legal ethics and technology issues, said the growth of surveillance was troubling for lawyers. He helped create the bar association’s ethics code revisions that require lawyers to try to avoid being overheard by eavesdroppers. “You run out of options very quickly to communicate with someone overseas,” he said. “Given the difficulty of finding anything that is 100 percent secure, lawyers are in a difficult spot to ensure that all of the information remains in confidence.” 

Still, the N.S.A. can intercept the communications of Americans if they are in contact with a foreign intelligence target abroad, such as Indonesian officials. The N.S.A. is then required to follow so-called minimization rules to protect their privacy, such as deleting the identity of Americans or information that is not deemed necessary to understand or assess the foreign intelligence, before sharing it with other agencies. An N.S.A. spokeswoman said the agency’s Office of the General Counsel was consulted when issues of potential attorney-client privilege arose and could recommend steps to protect such information. “Such steps could include requesting that collection or reporting by a foreign partner be limited, that intelligence reports be written so as to limit the inclusion of privileged material and to exclude U.S. identities, and that dissemination of such reports be limited and subject to appropriate warnings or restrictions on their use,” said Vanee M. Vines, the spokeswoman.

In justifying the agency’s sweeping powers, the Obama administration often emphasizes the N.S.A.’s role in fighting terrorism and cyberattacks, but disclosures in recent months from the documents leaked by Mr. Snowden show the agency routinely spies on trade negotiations, communications of economic officials in other countries and even foreign corporations.

Other documents obtained from Mr. Snowden reveal that the N.S.A. shares reports from its surveillance widely among civilian agencies. A 2004 N.S.A. document, for example, describes how the agency’s intelligence gathering was critical to the Agriculture Department in international trade negotiations. “The U.S.D.A. is involved in trade operations to protect and secure a large segment of the U.S. economy,” that document states. Top agency officials “often rely on SIGINT” — short for the signals intelligence that the N.S.A. eavesdropping collects — “to support their negotiations.”

The new disclosures come amid a debate over whether the surveillance court, which hears arguments only from the Justice Department, should be restructured for its evolving role. Proposals include overhauling how judges are selected to serve on it and creating a public advocate to provide adversarial arguments when the government offers complex legal analysis for expanding its powers.

The Raw Take order, back in 2002, also relaxed limits on sharing private information about Americans with foreign governments. The bar was higher for sharing with outsiders: Raw information was not provided, and even information deemed relevant about a terrorism issue required special approval. Under procedures described in a 1984 report, only the attorney general could authorize such dissemination. But on Aug. 20, 2002, Attorney General John Ashcroft, citing the recent order, secretly issued new procedures allowing the N.S.A. to provide information to foreign governments without his clearance. “If the proposed recipient(s) of the dissemination have a history of human rights abuses, that history should be considered in assessing the potential for economic injury, physical harm, or other restriction of movement, and whether the dissemination should be made,” he wrote.

The State Department list of extended closings comprised embassies and consulates in Jordan, Egypt, Saudi Arabia, Qatar, the United Arab Emirates, Kuwait, Bahrain, Oman, Yemen, Libya, Djibouti, Sudan, Madagascar, Burundi, Rwanda and Mauritius, the last four of which hadn’t been among the first wave announced Friday. Embassies and consulates in Afghanistan, Algeria, Bangladesh, Iraq and Mauritania reopened Monday.

Critics, including some congressional leaders, say that the role of N.S.A. surveillance in thwarting terrorist attacks — often cited by the agency to justify expanded powers — has been exaggerated. In response to the controversy about its activities after Mr. Snowden’s disclosures, agency officials claimed that the N.S.A.’s sweeping domestic surveillance programs had helped in 54 “terrorist-related activities.” But under growing scrutiny, congressional staff members and other critics say that the use of such figures by defenders of the agency has drastically overstated the value of the domestic surveillance programs in counterterrorism. Agency leaders believe that the N.S.A. has never enjoyed such a target-rich environment as it does now because of the global explosion of digital information — and they want to make certain that they can dominate “the Sigint battle space” in the future, the document said. To be “optimally effective,” the paper said, “legal, policy and process authorities must be as adaptive and dynamic as the technological and operational advances we seek to exploit.” Intent on unlocking the secrets of adversaries, the paper underscores the agency’s long-term goal of being able to collect virtually everything available in the digital world. To achieve that objective, the paper suggests that the N.S.A. plans to gain greater access, in a variety of ways, to the infrastructure of the world’s telecommunications networks.

Yet the paper also shows how the agency believes it can influence and shape trends in high-tech industries in other ways to suit its needs. One of the agency’s goals is to “continue to invest in the industrial base and drive the state of the art for high performance computing to maintain pre-eminent cryptanalytic capability for the nation.” The paper added that the N.S.A. must seek to “identify new access, collection and exploitation methods by leveraging global business trends in data and communications services.” And it wants to find ways to combine all of its technical tools to enhance its surveillance powers. The N.S.A. will seek to integrate its “capabilities to reach previously inaccessible targets in support of exploitation, cyberdefense and cyberoperations,” the paper stated. The agency also intends to improve its access to encrypted communications used by individuals, businesses and foreign governments, the strategy document said. The N.S.A. has already had some success in defeating encryption, The New York Times has reported, but the document makes it clear that countering “ubiquitous, strong, commercial network encryption” is a top priority. The agency plans to fight back against the rise of encryption through relationships with companies that develop encryption tools and through espionage operations. In other countries, the document said, the N.S.A. must also “counter indigenous cryptographic programs by targeting their industrial bases with all available Sigint and Humint” — human intelligence, meaning spies.

Above all, the strategy paper suggests the N.S.A.’s vast view of its mission: nothing less than to “dramatically increase mastery of the global network.” Other N.S.A. documents offer hints of how the agency is trying to do just that. One program, code-named Treasure Map, provides what a secret N.S.A. PowerPoint presentation describes as “a near real-time, interactive map of the global Internet.” According to the undated PowerPoint presentation, disclosed by Mr. Snowden, Treasure Map gives the N.S.A. “a 300,000 foot view of the Internet.”  Relying on Internet routing data, commercial and Sigint information, Treasure Map is a sophisticated tool, one that the PowerPoint presentation describes as a “massive Internet mapping, analysis and exploration engine.” It collects Wi-Fi network and geolocation data, and between 30 million and 50 million unique Internet provider addresses — code that can reveal the location and owner of a computer, mobile device or router — are represented each day on Treasure Map, according to the document. It boasts that the program can map “any device, anywhere, all the time.”  The documents include addresses labeled as based in the “U.S.,” and because so much Internet traffic flows through the United States, it would be difficult to map much of the world without capturing such addresses.

The program takes advantage of the capabilities of other secret N.S.A. programs. To support Treasure Map, for example, the document states that another program, called Packaged Goods, tracks the “traceroutes” through which data flows around the Internet. Through Packaged Goods, the N.S.A. has gained access to “13 covered servers in unwitting data centers around the globe,” according to the PowerPoint. The document identifies a list of countries where the data centers are located, including Germany, Poland, Denmark, South Africa and Taiwan as well as Russia, China and Singapore.