Skip to main content

Home/ Future of the Web/ Group items matching "messages" in title, tags, annotations or url

Group items matching
in title, tags, annotations or url

Sort By: Relevance | Date Filter: All | Bookmarks | Topics Simple Middle
Paul Merrell

Operation Socialist: How GCHQ Spies Hacked Belgium's Largest Telco - 0 views

  • When the incoming emails stopped arriving, it seemed innocuous at first. But it would eventually become clear that this was no routine technical problem. Inside a row of gray office buildings in Brussels, a major hacking attack was in progress. And the perpetrators were British government spies. It was in the summer of 2012 that the anomalies were initially detected by employees at Belgium’s largest telecommunications provider, Belgacom. But it wasn’t until a year later, in June 2013, that the company’s security experts were able to figure out what was going on. The computer systems of Belgacom had been infected with a highly sophisticated malware, and it was disguising itself as legitimate Microsoft software while quietly stealing data. Last year, documents from National Security Agency whistleblower Edward Snowden confirmed that British surveillance agency Government Communications Headquarters was behind the attack, codenamed Operation Socialist. And in November, The Intercept revealed that the malware found on Belgacom’s systems was one of the most advanced spy tools ever identified by security researchers, who named it “Regin.”
  • The full story about GCHQ’s infiltration of Belgacom, however, has never been told. Key details about the attack have remained shrouded in mystery—and the scope of the attack unclear. Now, in partnership with Dutch and Belgian newspapers NRC Handelsblad and De Standaard, The Intercept has pieced together the first full reconstruction of events that took place before, during, and after the secret GCHQ hacking operation. Based on new documents from the Snowden archive and interviews with sources familiar with the malware investigation at Belgacom, The Intercept and its partners have established that the attack on Belgacom was more aggressive and far-reaching than previously thought. It occurred in stages between 2010 and 2011, each time penetrating deeper into Belgacom’s systems, eventually compromising the very core of the company’s networks.
  • Snowden told The Intercept that the latest revelations amounted to unprecedented “smoking-gun attribution for a governmental cyber attack against critical infrastructure.” The Belgacom hack, he said, is the “first documented example to show one EU member state mounting a cyber attack on another…a breathtaking example of the scale of the state-sponsored hacking problem.”
  • ...7 more annotations...
  • When the incoming emails stopped arriving, it seemed innocuous at first. But it would eventually become clear that this was no routine technical problem. Inside a row of gray office buildings in Brussels, a major hacking attack was in progress. And the perpetrators were British government spies. It was in the summer of 2012 that the anomalies were initially detected by employees at Belgium’s largest telecommunications provider, Belgacom. But it wasn’t until a year later, in June 2013, that the company’s security experts were able to figure out what was going on. The computer systems of Belgacom had been infected with a highly sophisticated malware, and it was disguising itself as legitimate Microsoft software while quietly stealing data. Last year, documents from National Security Agency whistleblower Edward Snowden confirmed that British surveillance agency Government Communications Headquarters was behind the attack, codenamed Operation Socialist. And in November, The Intercept revealed that the malware found on Belgacom’s systems was one of the most advanced spy tools ever identified by security researchers, who named it “Regin.”
  • Publicly, Belgacom has played down the extent of the compromise, insisting that only its internal systems were breached and that customers’ data was never found to have been at risk. But secret GCHQ documents show the agency gained access far beyond Belgacom’s internal employee computers and was able to grab encrypted and unencrypted streams of private communications handled by the company. Belgacom invested several million dollars in its efforts to clean-up its systems and beef-up its security after the attack. However, The Intercept has learned that sources familiar with the malware investigation at the company are uncomfortable with how the clean-up operation was handled—and they believe parts of the GCHQ malware were never fully removed.
  • The revelations about the scope of the hacking operation will likely alarm Belgacom’s customers across the world. The company operates a large number of data links internationally (see interactive map below), and it serves millions of people across Europe as well as officials from top institutions including the European Commission, the European Parliament, and the European Council. The new details will also be closely scrutinized by a federal prosecutor in Belgium, who is currently carrying out a criminal investigation into the attack on the company. Sophia in ’t Veld, a Dutch politician who chaired the European Parliament’s recent inquiry into mass surveillance exposed by Snowden, told The Intercept that she believes the British government should face sanctions if the latest disclosures are proven.
  • What sets the secret British infiltration of Belgacom apart is that it was perpetrated against a close ally—and is backed up by a series of top-secret documents, which The Intercept is now publishing.
  • Between 2009 and 2011, GCHQ worked with its allies to develop sophisticated new tools and technologies it could use to scan global networks for weaknesses and then penetrate them. According to top-secret GCHQ documents, the agency wanted to adopt the aggressive new methods in part to counter the use of privacy-protecting encryption—what it described as the “encryption problem.” When communications are sent across networks in encrypted format, it makes it much harder for the spies to intercept and make sense of emails, phone calls, text messages, internet chats, and browsing sessions. For GCHQ, there was a simple solution. The agency decided that, where possible, it would find ways to hack into communication networks to grab traffic before it’s encrypted.
  • The Snowden documents show that GCHQ wanted to gain access to Belgacom so that it could spy on phones used by surveillance targets travelling in Europe. But the agency also had an ulterior motive. Once it had hacked into Belgacom’s systems, GCHQ planned to break into data links connecting Belgacom and its international partners, monitoring communications transmitted between Europe and the rest of the world. A map in the GCHQ documents, named “Belgacom_connections,” highlights the company’s reach across Europe, the Middle East, and North Africa, illustrating why British spies deemed it of such high value.
  • Documents published with this article: Automated NOC detection Mobile Networks in My NOC World Making network sense of the encryption problem Stargate CNE requirements NAC review – October to December 2011 GCHQ NAC review – January to March 2011 GCHQ NAC review – April to June 2011 GCHQ NAC review – July to September 2011 GCHQ NAC review – January to March 2012 GCHQ Hopscotch Belgacom connections
Gonzalo San Gil, PhD.

Thunderclap: Free Information from Space Outernet for Aug 11, 2014 - 0 views

  • Right now, only 40% of humanity can connect to the Internet. Even less than that have access to truly free, uncensored Internet. What this represents is an enormous gap in access to information. While the Internet is an amazing communication tool, it is also the largest library ever constructed. It grants access to anything from books, videos, courseware, news, and weather, to open source farm equipment or instructions on how to treat infection or prevent HIV from spreading. #ImagineIf everyone could have that information for free?On August 11, 2014, Outernet will make that library available from space for free for the first time. Help us tell the world.#ImagineIf everyone had any information they wanted - what would that world look like? What new inventions would be created or diseases cured? What would people read about if their governments no longer deprived them of their right to free information? Soon, we won't have to imagine.
  • Right now, only 40% of humanity can connect to the Internet. Even less than that have access to truly free, uncensored Internet. What this represents is an enormous gap in access to information. While the Internet is an amazing communication tool, it is also the largest library ever constructed. It grants access to anything from books, videos, courseware, news, and weather, to open source farm equipment or instructions on how to treat infection or prevent HIV from spreading. #ImagineIf everyone could have that information for free?On August 11, 2014, Outernet will make that library available from space for free for the first time. Help us tell the world.#ImagineIf everyone had any information they wanted - what would that world look like? What new inventions would be created or diseases cured? What would people read about if their governments no longer deprived them of their right to free information? 
  •  
    INFORMATION FOR THE WORLD FROM OUTER SPACE Unrestricted, globally accessible, broadcast data. Quality content from all over the Internet. Available to all of humanity. For free. Through satellite data broadcasting, Outernet is able to bypass censorship, ensure privacy, and offer a universally-accessible information service at no cost to global citizens. It's the modern version of shortwave radio, or BitTorrent from space.
  •  
    ""#ImagineIf every human had a free library at home... Information equality begins TODAY: Outernet is LIVE from space! http://thndr.it/1pazaP3" "
  •  
    ""#ImagineIf every human had a free library at home... Information equality begins TODAY: Outernet is LIVE from space! http://thndr.it/1pazaP3" "
Paul Merrell

An Important Kindle request - 0 views

  • A Message from the Amazon Books Team Dear Readers, Just ahead of World War II, there was a radical invention that shook the foundations of book publishing. It was the paperback book. This was a time when movie tickets cost 10 or 20 cents, and books cost $2.50. The new paperback cost 25 cents — it was ten times cheaper. Readers loved the paperback and millions of copies were sold in just the first year. With it being so inexpensive and with so many more people able to afford to buy and read books, you would think the literary establishment of the day would have celebrated the invention of the paperback, yes? Nope. Instead, they dug in and circled the wagons. They believed low cost paperbacks would destroy literary culture and harm the industry (not to mention their own bank accounts). Many bookstores refused to stock them, and the early paperback publishers had to use unconventional methods of distribution — places like newsstands and drugstores. The famous author George Orwell came out publicly and said about the new paperback format, if "publishers had any sense, they would combine against them and suppress them." Yes, George Orwell was suggesting collusion. Well… history doesn't repeat itself, but it does rhyme.
  • Fast forward to today, and it's the e-book's turn to be opposed by the literary establishment. Amazon and Hachette — a big US publisher and part of a $10 billion media conglomerate — are in the middle of a business dispute about e-books. We want lower e-book prices. Hachette does not. Many e-books are being released at $14.99 and even $19.99. That is unjustifiably high for an e-book. With an e-book, there's no printing, no over-printing, no need to forecast, no returns, no lost sales due to out of stock, no warehousing costs, no transportation costs, and there is no secondary market — e-books cannot be resold as used books. E-books can and should be less expensive. Perhaps channeling Orwell's decades old suggestion, Hachette has already been caught illegally colluding with its competitors to raise e-book prices. So far those parties have paid $166 million in penalties and restitution. Colluding with its competitors to raise prices wasn't only illegal, it was also highly disrespectful to Hachette's readers. The fact is many established incumbents in the industry have taken the position that lower e-book prices will "devalue books" and hurt "Arts and Letters." They're wrong. Just as paperbacks did not destroy book culture despite being ten times cheaper, neither will e-books. On the contrary, paperbacks ended up rejuvenating the book industry and making it stronger. The same will happen with e-books.
Paul Merrell

What the Hack! 56 Suspected Hackers arrested in the UK | nsnbc international - 0 views

  • The UK National Crime Agency arrested 56 suspected hackers, including one 23-year-old male who allegedly attempted to hack his way into the U.S.’ Department of Defense in 2014. Not attempting to minimize the potential risks of hacking but how much does cyber-crime actually cost, what are the risks and what about those who hack the data of billions of internet users per day to, allegedly, “keep all of us safe?”
  • Besides the 23-year-old who allegedly attempted to hack his way into the a U.S. Department of Defense site, the other detainees allegedly were members of the hacking collectives Lizard Squad and D33DS which are being accused of fraud, money laundering and Denial of Service and Distributed Denial of Service (DOS & DDOS) attacks.  D33DS stands accused of having stolen data of some 450,000 Yahoo users. The arrests followed the recent announcement about the so-called FREAK security vulnerability that was leaving thousands of SSL sites unprotected. The arrest of the 56 hackers in the UK was reported as the National Crime Agency’s way of “sending a clear message” to the hacker community.
  • The U.S. DoD’s cyber-security functioned, obviously. A recent article by Benjamin Dean entitled “Hard Evidence: How much is cybercrime really costing us” suggests that the money spent on cyber-security per year is disproportional to the harm that is being caused by cyber-crime. Dean, who is a Fellow for Internet Governance and Cyber-security at the School of International and Public Affairs at Columbia University would conclude that: There are numerous competing budgetary priorities at any one time and limited funds to spend on meeting all these needs. How much money does it make sense to invest in bolstering cybersecurity, relative to the losses? …In the hysteria created in the wake of the hacks of 2014, we risk making the wrong choice simply because we don’t know what the current sums of money are being spent on.
  • ...1 more annotation...
  • Meanwhile, NSA whitleblower Edward Snowden (think about him what you want), would reveal that the NSA and the GCHQ hacked themselves into the possession of the encryption codes of the world’s largest SIM card manufacturer Gemalto. Snowden’s revelations about the NSA’s PRISM surveillance program wouldn’t come as a surprise to those who have known about the United States’ and allies mutual spying network Echelon for decades.
Paul Merrell

Internet privacy, funded by spooks: A brief history of the BBG | PandoDaily - 0 views

  • For the past few months I’ve been covering U.S. government funding of popular Internet privacy tools like Tor, CryptoCat and Open Whisper Systems. During my reporting, one agency in particular keeps popping up: An agency with one of those really bland names that masks its wild, bizarre history: the Broadcasting Board of Governors, or BBG. The BBG was formed in 1999 and runs on a $721 million annual budget. It reports directly to Secretary of State John Kerry and operates like a holding company for a host of Cold War-era CIA spinoffs and old school “psychological warfare” projects: Radio Free Europe, Radio Free Asia, Radio Martí, Voice of America, Radio Liberation from Bolshevism (since renamed “Radio Liberty”) and a dozen other government-funded radio stations and media outlets pumping out pro-American propaganda across the globe. Today, the Congressionally-funded federal agency is also one of the biggest backers of grassroots and open-source Internet privacy technology. These investments started in 2012, when the BBG launched the “Open Technology Fund” (OTF) — an initiative housed within and run by Radio Free Asia (RFA), a premier BBG property that broadcasts into communist countries like North Korea, Vietnam, Laos, China and Myanmar. The BBG endowed Radio Free Asia’s Open Technology Fund with a multimillion dollar budget and a single task: “to fulfill the U.S. Congressional global mandate for Internet freedom.”
  • Here’s a small sample of what the Broadcasting Board of Governors funded (through Radio Free Asia and then through the Open Technology Fund) between 2012 and 2014: Open Whisper Systems, maker of free encrypted text and voice mobile apps like TextSecure and Signal/RedPhone, got a generous $1.35-million infusion. (Facebook recently started using Open Whisper Systems to secure its WhatsApp messages.) CryptoCat, an encrypted chat app made by Nadim Kobeissi and promoted by EFF, received $184,000. LEAP, an email encryption startup, got just over $1 million. LEAP is currently being used to run secure VPN services at RiseUp.net, the radical anarchist communication collective. A Wikileaks alternative called GlobaLeaks (which was endorsed by the folks at Tor, including Jacob Appelbaum) received just under $350,000. The Guardian Project — which makes an encrypted chat app called ChatSecure, as well a mobile version of Tor called Orbot — got $388,500. The Tor Project received over $1 million from OTF to pay for security audits, traffic analysis tools and set up fast Tor exit nodes in the Middle East and South East Asia.
  •  
    But can we trust them?
Paul Merrell

What's Scarier: Terrorism, or Governments Blocking Websites in its Name? - The Intercept - 0 views

  • Forcibly taking down websites deemed to be supportive of terrorism, or criminalizing speech deemed to “advocate” terrorism, is a major trend in both Europe and the West generally. Last month in Brussels, the European Union’s counter-terrorism coordinator issued a memo proclaiming that “Europe is facing an unprecedented, diverse and serious terrorist threat,” and argued that increased state control over the Internet is crucial to combating it. The memo noted that “the EU and its Member States have developed several initiatives related to countering radicalisation and terrorism on the Internet,” yet argued that more must be done. It argued that the focus should be on “working with the main players in the Internet industry [a]s the best way to limit the circulation of terrorist material online.” It specifically hailed the tactics of the U.K. Counter-Terrorism Internet Referral Unit (CTIRU), which has succeeded in causing the removal of large amounts of material it deems “extremist”:
  • In addition to recommending the dissemination of “counter-narratives” by governments, the memo also urged EU member states to “examine the legal and technical possibilities to remove illegal content.” Exploiting terrorism fears to control speech has been a common practice in the West since 9/11, but it is becoming increasingly popular even in countries that have experienced exceedingly few attacks. A new extremist bill advocated by the right-wing Harper government in Canada (also supported by Liberal Party leader Justin Trudeau even as he recognizes its dangers) would create new crimes for “advocating terrorism”; specifically: “every person who, by communicating statements, knowingly advocates or promotes the commission of terrorism offences in general” would be a guilty and can be sent to prison for five years for each offense. In justifying the new proposal, the Canadian government admits that “under the current criminal law, it is [already] a crime to counsel or actively encourage others to commit a specific terrorism offence.” This new proposal is about criminalizing ideas and opinions. In the government’s words, it “prohibits the intentional advocacy or promotion of terrorism, knowing or reckless as to whether it would result in terrorism.”
  • If someone argues that continuous Western violence and interference in the Muslim world for decades justifies violence being returned to the West, or even advocates that governments arm various insurgents considered by some to be “terrorists,” such speech could easily be viewed as constituting a crime. To calm concerns, Canadian authorities point out that “the proposed new offence is similar to one recently enacted by Australia, that prohibits advocating a terrorist act or the commission of a terrorism offence-all while being reckless as to whether another person will engage in this kind of activity.” Indeed, Australia enacted a new law late last year that indisputably targets political speech and ideas, as well as criminalizing journalism considered threatening by the government. Punishing people for their speech deemed extremist or dangerous has been a vibrant practice in both the U.K. and U.S. for some time now, as I detailed (coincidentally) just a couple days before free speech marches broke out in the West after the Charlie Hebdo attacks. Those criminalization-of-speech attacks overwhelmingly target Muslims, and have resulted in the punishment of such classic free speech activities as posting anti-war commentary on Facebook, tweeting links to “extremist” videos, translating and posting “radicalizing” videos to the Internet, writing scholarly articles in defense of Palestinian groups and expressing harsh criticism of Israel, and even including a Hezbollah channel in a cable package.
  • ...2 more annotations...
  • Beyond the technical issues, trying to legislate ideas out of existence is a fool’s game: those sufficiently determined will always find ways to make themselves heard. Indeed, as U.S. pop star Barbra Streisand famously learned, attempts to suppress ideas usually result in the greatest publicity possible for their advocates and/or elevate them by turning fringe ideas into martyrs for free speech (I have zero doubt that all five of the targeted sites enjoyed among their highest traffic dates ever today as a result of the French targeting). But the comical futility of these efforts is exceeded by their profound dangers. Who wants governments to be able to unilaterally block websites? Isn’t the exercise of this website-blocking power what has long been cited as reasons we should regard the Bad Countries — such as China and Iran — as tyrannies (which also usually cite “counterterrorism” to justify their censorship efforts)?
  • s those and countless other examples prove, the concepts of “extremism” and “radicalizing” (like “terrorism” itself) are incredibly vague and elastic, and in the hands of those who wield power, almost always expand far beyond what you think it should mean (plotting to blow up innocent people) to mean: anyone who disseminates ideas that are threatening to the exercise of our power. That’s why powers justified in the name of combating “radicalism” or “extremism” are invariably — not often or usually, but invariably — applied to activists, dissidents, protesters and those who challenge prevailing orthodoxies and power centers. My arguments for distrusting governments to exercise powers of censorship are set forth here (in the context of a prior attempt by a different French minister to control the content of Twitter). In sum, far more damage has been inflicted historically by efforts to censor and criminalize political ideas than by the kind of “terrorism” these governments are invoking to justify these censorship powers. And whatever else may be true, few things are more inimical to, or threatening of, Internet freedom than allowing functionaries inside governments to unilaterally block websites from functioning on the ground that the ideas those sites advocate are objectionable or “dangerous.” That’s every bit as true when the censors are in Paris, London, and Ottawa, and Washington as when they are in Tehran, Moscow or Beijing.
Paul Merrell

How to Encrypt the Entire Web for Free - The Intercept - 0 views

  • If we’ve learned one thing from the Snowden revelations, it’s that what can be spied on will be spied on. Since the advent of what used to be known as the World Wide Web, it has been a relatively simple matter for network attackers—whether it’s the NSA, Chinese intelligence, your employer, your university, abusive partners, or teenage hackers on the same public WiFi as you—to spy on almost everything you do online. HTTPS, the technology that encrypts traffic between browsers and websites, fixes this problem—anyone listening in on that stream of data between you and, say, your Gmail window or bank’s web site would get nothing but useless random characters—but is woefully under-used. The ambitious new non-profit Let’s Encrypt aims to make the process of deploying HTTPS not only fast, simple, and free, but completely automatic. If it succeeds, the project will render vast regions of the internet invisible to prying eyes.
  • Encryption also prevents attackers from tampering with or impersonating legitimate websites. For example, the Chinese government censors specific pages on Wikipedia, the FBI impersonated The Seattle Times to get a suspect to click on a malicious link, and Verizon and AT&T injected tracking tokens into mobile traffic without user consent. HTTPS goes a long way in preventing these sorts of attacks. And of course there’s the NSA, which relies on the limited adoption of HTTPS to continue to spy on the entire internet with impunity. If companies want to do one thing to meaningfully protect their customers from surveillance, it should be enabling encryption on their websites by default.
  • Let’s Encrypt, which was announced this week but won’t be ready to use until the second quarter of 2015, describes itself as “a free, automated, and open certificate authority (CA), run for the public’s benefit.” It’s the product of years of work from engineers at Mozilla, Cisco, Akamai, Electronic Frontier Foundation, IdenTrust, and researchers at the University of Michigan. (Disclosure: I used to work for the Electronic Frontier Foundation, and I was aware of Let’s Encrypt while it was being developed.) If Let’s Encrypt works as advertised, deploying HTTPS correctly and using all of the best practices will be one of the simplest parts of running a website. All it will take is running a command. Currently, HTTPS requires jumping through a variety of complicated hoops that certificate authorities insist on in order prove ownership of domain names. Let’s Encrypt automates this task in seconds, without requiring any human intervention, and at no cost.
  • ...2 more annotations...
  • The benefits of using HTTPS are obvious when you think about protecting secret information you send over the internet, like passwords and credit card numbers. It also helps protect information like what you search for in Google, what articles you read, what prescription medicine you take, and messages you send to colleagues, friends, and family from being monitored by hackers or authorities. But there are less obvious benefits as well. Websites that don’t use HTTPS are vulnerable to “session hijacking,” where attackers can take over your account even if they don’t know your password. When you download software without encryption, sophisticated attackers can secretly replace the download with malware that hacks your computer as soon as you try installing it.
  • The transition to a fully encrypted web won’t be immediate. After Let’s Encrypt is available to the public in 2015, each website will have to actually use it to switch over. And major web hosting companies also need to hop on board for their customers to be able to take advantage of it. If hosting companies start work now to integrate Let’s Encrypt into their services, they could offer HTTPS hosting by default at no extra cost to all their customers by the time it launches.
  •  
    Don't miss the video. And if you have a web site, urge your host service to begin preparing for Let's Encrypt. (See video on why it's good for them.)
Paul Merrell

Guest Post: NSA Reform - The Consequences of Failure | Just Security - 0 views

  • In the absence of real reform, people and institutions at home and abroad are taking matters into their own hands. In America, the NSA’s overreach is changing the way we communicate with and relate to each other. In order to evade government surveillance, more and more Americans are employing encryption technology.  The veritable explosion of new secure messaging apps like Surespot, OpenWhisper’s collaboration with WhatsApp, the development and deployment of open source anti-surveillance tools like Detekt, the creation of organizationally-sponsored “surveillance self-defense” guides, the push to universalize the https protocol, anti-surveillance book events featuring free encryption workshops— are manifestations of the rise of the personal encryption and pro-privacy digital resistance movement. Its political implications are clear: Americans, along with people around the world, increasingly see the United States government’s overreaching surveillance activities as a threat to be blocked.
  • The federal government’s vacuum-cleaner approach to surveillance—manifested in Title II of the PATRIOT Act, the FISA Amendments Act, and EO 12333—has backfired in these respects, and the emergence of this digital resistance movement is one result. Indeed, the existence and proliferation of social networks hold the potential to help this movement spread faster and to more of the general public than would have been possible in decades past. This is evidenced by the growing concern worldwide about governments’ ability to access reams of information about people’s lives with relative ease. As one measure, compared to a year ago, 41% of online users in North America now avoid certain Internet sites and applications, 16% change who they communicate with, and 24% censor what they say online. Those numbers, if anywhere close to accurate, are a major concern for democratic society.
  • Even if commercially available privacy technology proves capable of providing a genuine shield against warrantless or otherwise illegal surveillance by the United States government, it will remain a treatment for the symptom, not a cure for the underlying legal and constitutional malady. In April 2014, a Harris poll of US adults showed that in response to the Snowden revelations, “Almost half of respondents (47%) said that they have changed their online behavior and think more carefully about where they go, what they say, and what they do online.” Set aside for a moment that just the federal government’s collection of the data of innocent Americans is itself likely a violation of the Fourth Amendment. The Harris poll is just one of numerous studies highlighting the collateral damage to American society and politics from NSA’s excesses: segments of our population are now fearful of even associating with individuals or organizations executive branch officials deem controversial or suspicious. Nearly half of Americans say they have changed their online behavior out of a fear of what the federal government might do with their personal information. The Constitution’s free association guarantee has been damaged by the Surveillance State’s very operation.
  • ...1 more annotation...
  • The failure of the Congress and the courts to end the surveillance state, despite the repeated efforts by a huge range of political and public interest actors to effect that change through the political process, is only fueling the growing resistance movement. Federal officials understand this, which is why they are trying—desperately and in the view of some, underhandedly—to shut down this digital resistance movement. This action/reaction cycle is exactly what it appears to be: an escalating conflict between the American public and its government. Without comprehensive surveillance authority reforms (including a journalist “shield law” and ironclad whistleblower protections for Intelligence Community contractors) that are verifiable and enforceable, that conflict will only continue.
Paul Merrell

BBC News - GCHQ's Robert Hannigan says tech firms 'in denial' on extremism - 0 views

  • Web giants such as Twitter, Facebook and WhatsApp have become "command-and-control networks... for terrorists and criminals", GCHQ's new head has said. Islamic State extremists had "embraced" the web but some companies remained "in denial" over the problem, Robert Hannigan wrote in the Financial Times. He called for them to do more to co-operate with security services. However, civil liberties campaigners said the companies were already working with the intelligence agencies. None of the major tech firms has yet responded to Mr Hannigan's comments.
  • GCHQ, terrorists, and the internet: what are the issues? GCHQ v tech firms: Internet reacts Change at the top for Britain's
  • Mr Hannigan said IS had "embraced the web as a noisy channel in which to promote itself, intimidate people, and radicalise new recruits." The "security of its communications" added another challenge to agencies such as GCHQ, he said - adding that techniques for encrypting - or digitally scrambling - messages "which were once the preserve of the most sophisticated criminals or nation states now come as standard". GCHQ and its sister agencies, MI5 and the Secret Intelligence Service, could not tackle these challenges "at scale" without greater support from the private sector, including the largest US technology companies which dominate the web, he wrote.
  •  
    What I want to know is what we're going to do with that NSA data center at Bluffdale, Utah, after the NSA is abolished? Maybe give it to the Internet Archive?
Paul Merrell

Cy Vance's Proposal to Backdoor Encrypted Devices Is Riddled With Vulnerabilities | Just Security - 0 views

  • Less than a week after the attacks in Paris — while the public and policymakers were still reeling, and the investigation had barely gotten off the ground — Cy Vance, Manhattan’s District Attorney, released a policy paper calling for legislation requiring companies to provide the government with backdoor access to their smartphones and other mobile devices. This is the first concrete proposal of this type since September 2014, when FBI Director James Comey reignited the “Crypto Wars” in response to Apple’s and Google’s decisions to use default encryption on their smartphones. Though Comey seized on Apple’s and Google’s decisions to encrypt their devices by default, his concerns are primarily related to end-to-end encryption, which protects communications that are in transit. Vance’s proposal, on the other hand, is only concerned with device encryption, which protects data stored on phones. It is still unclear whether encryption played any role in the Paris attacks, though we do know that the attackers were using unencrypted SMS text messages on the night of the attack, and that some of them were even known to intelligence agencies and had previously been under surveillance. But regardless of whether encryption was used at some point during the planning of the attacks, as I lay out below, prohibiting companies from selling encrypted devices would not prevent criminals or terrorists from being able to access unbreakable encryption. Vance’s primary complaint is that Apple’s and Google’s decisions to provide their customers with more secure devices through encryption interferes with criminal investigations. He claims encryption prevents law enforcement from accessing stored data like imessages, photos and videos, Internet search histories, and third party app data. He makes several arguments to justify his proposal to build backdoors into encrypted smartphones, but none of them hold water.
  • Before addressing the major privacy, security, and implementation concerns that his proposal raises, it is worth noting that while an increase in use of fully encrypted devices could interfere with some law enforcement investigations, it will help prevent far more crimes — especially smartphone theft, and the consequent potential for identity theft. According to Consumer Reports, in 2014 there were more than two million victims of smartphone theft, and nearly two-thirds of all smartphone users either took no steps to secure their phones or their data or failed to implement passcode access for their phones. Default encryption could reduce instances of theft because perpetrators would no longer be able to break into the phone to steal the data.
  • Vance argues that creating a weakness in encryption to allow law enforcement to access data stored on devices does not raise serious concerns for security and privacy, since in order to exploit the vulnerability one would need access to the actual device. He considers this an acceptable risk, claiming it would not be the same as creating a widespread vulnerability in encryption protecting communications in transit (like emails), and that it would be cheap and easy for companies to implement. But Vance seems to be underestimating the risks involved with his plan. It is increasingly important that smartphones and other devices are protected by the strongest encryption possible. Our devices and the apps on them contain astonishing amounts of personal information, so much that an unprecedented level of harm could be caused if a smartphone or device with an exploitable vulnerability is stolen, not least in the forms of identity fraud and credit card theft. We bank on our phones, and have access to credit card payments with services like Apple Pay. Our contact lists are stored on our phones, including phone numbers, emails, social media accounts, and addresses. Passwords are often stored on people’s phones. And phones and apps are often full of personal details about their lives, from food diaries to logs of favorite places to personal photographs. Symantec conducted a study, where the company spread 50 “lost” phones in public to see what people who picked up the phones would do with them. The company found that 95 percent of those people tried to access the phone, and while nearly 90 percent tried to access private information stored on the phone or in other private accounts such as banking services and email, only 50 percent attempted contacting the owner.
  • ...8 more annotations...
  • Vance attempts to downplay this serious risk by asserting that anyone can use the “Find My Phone” or Android Device Manager services that allow owners to delete the data on their phones if stolen. However, this does not stand up to scrutiny. These services are effective only when an owner realizes their phone is missing and can take swift action on another computer or device. This delay ensures some period of vulnerability. Encryption, on the other hand, protects everyone immediately and always. Additionally, Vance argues that it is safer to build backdoors into encrypted devices than it is to do so for encrypted communications in transit. It is true that there is a difference in the threats posed by the two types of encryption backdoors that are being debated. However, some manner of widespread vulnerability will inevitably result from a backdoor to encrypted devices. Indeed, the NSA and GCHQ reportedly hacked into a database to obtain cell phone SIM card encryption keys in order defeat the security protecting users’ communications and activities and to conduct surveillance. Clearly, the reality is that the threat of such a breach, whether from a hacker or a nation state actor, is very real. Even if companies go the extra mile and create a different means of access for every phone, such as a separate access key for each phone, significant vulnerabilities will be created. It would still be possible for a malicious actor to gain access to the database containing those keys, which would enable them to defeat the encryption on any smartphone they took possession of. Additionally, the cost of implementation and maintenance of such a complex system could be high.
  • Privacy is another concern that Vance dismisses too easily. Despite Vance’s arguments otherwise, building backdoors into device encryption undermines privacy. Our government does not impose a similar requirement in any other context. Police can enter homes with warrants, but there is no requirement that people record their conversations and interactions just in case they someday become useful in an investigation. The conversations that we once had through disposable letters and in-person conversations now happen over the Internet and on phones. Just because the medium has changed does not mean our right to privacy has.
  • In addition to his weak reasoning for why it would be feasible to create backdoors to encrypted devices without creating undue security risks or harming privacy, Vance makes several flawed policy-based arguments in favor of his proposal. He argues that criminals benefit from devices that are protected by strong encryption. That may be true, but strong encryption is also a critical tool used by billions of average people around the world every day to protect their transactions, communications, and private information. Lawyers, doctors, and journalists rely on encryption to protect their clients, patients, and sources. Government officials, from the President to the directors of the NSA and FBI, and members of Congress, depend on strong encryption for cybersecurity and data security. There are far more innocent Americans who benefit from strong encryption than there are criminals who exploit it. Encryption is also essential to our economy. Device manufacturers could suffer major economic losses if they are prohibited from competing with foreign manufacturers who offer more secure devices. Encryption also protects major companies from corporate and nation-state espionage. As more daily business activities are done on smartphones and other devices, they may now hold highly proprietary or sensitive information. Those devices could be targeted even more than they are now if all that has to be done to access that information is to steal an employee’s smartphone and exploit a vulnerability the manufacturer was required to create.
  • Vance also suggests that the US would be justified in creating such a requirement since other Western nations are contemplating requiring encryption backdoors as well. Regardless of whether other countries are debating similar proposals, we cannot afford a race to the bottom on cybersecurity. Heads of the intelligence community regularly warn that cybersecurity is the top threat to our national security. Strong encryption is our best defense against cyber threats, and following in the footsteps of other countries by weakening that critical tool would do incalculable harm. Furthermore, even if the US or other countries did implement such a proposal, criminals could gain access to devices with strong encryption through the black market. Thus, only innocent people would be negatively affected, and some of those innocent people might even become criminals simply by trying to protect their privacy by securing their data and devices. Finally, Vance argues that David Kaye, UN Special Rapporteur for Freedom of Expression and Opinion, supported the idea that court-ordered decryption doesn’t violate human rights, provided certain criteria are met, in his report on the topic. However, in the context of Vance’s proposal, this seems to conflate the concepts of court-ordered decryption and of government-mandated encryption backdoors. The Kaye report was unequivocal about the importance of encryption for free speech and human rights. The report concluded that:
  • States should promote strong encryption and anonymity. National laws should recognize that individuals are free to protect the privacy of their digital communications by using encryption technology and tools that allow anonymity online. … States should not restrict encryption and anonymity, which facilitate and often enable the rights to freedom of opinion and expression. Blanket prohibitions fail to be necessary and proportionate. States should avoid all measures that weaken the security that individuals may enjoy online, such as backdoors, weak encryption standards and key escrows. Additionally, the group of intelligence experts that was hand-picked by the President to issue a report and recommendations on surveillance and technology, concluded that: [R]egarding encryption, the U.S. Government should: (1) fully support and not undermine efforts to create encryption standards; (2) not in any way subvert, undermine, weaken, or make vulnerable generally available commercial software; and (3) increase the use of encryption and urge US companies to do so, in order to better protect data in transit, at rest, in the cloud, and in other storage.
  • The clear consensus among human rights experts and several high-ranking intelligence experts, including the former directors of the NSA, Office of the Director of National Intelligence, and DHS, is that mandating encryption backdoors is dangerous. Unaddressed Concerns: Preventing Encrypted Devices from Entering the US and the Slippery Slope In addition to the significant faults in Vance’s arguments in favor of his proposal, he fails to address the question of how such a restriction would be effectively implemented. There is no effective mechanism for preventing code from becoming available for download online, even if it is illegal. One critical issue the Vance proposal fails to address is how the government would prevent, or even identify, encrypted smartphones when individuals bring them into the United States. DHS would have to train customs agents to search the contents of every person’s phone in order to identify whether it is encrypted, and then confiscate the phones that are. Legal and policy considerations aside, this kind of policy is, at the very least, impractical. Preventing strong encryption from entering the US is not like preventing guns or drugs from entering the country — encrypted phones aren’t immediately obvious as is contraband. Millions of people use encrypted devices, and tens of millions more devices are shipped to and sold in the US each year.
  • Finally, there is a real concern that if Vance’s proposal were accepted, it would be the first step down a slippery slope. Right now, his proposal only calls for access to smartphones and devices running mobile operating systems. While this policy in and of itself would cover a number of commonplace devices, it may eventually be expanded to cover laptop and desktop computers, as well as communications in transit. The expansion of this kind of policy is even more worrisome when taking into account the speed at which technology evolves and becomes widely adopted. Ten years ago, the iPhone did not even exist. Who is to say what technology will be commonplace in 10 or 20 years that is not even around today. There is a very real question about how far law enforcement will go to gain access to information. Things that once seemed like merely science fiction, such as wearable technology and artificial intelligence that could be implanted in and work with the human nervous system, are now available. If and when there comes a time when our “smart phone” is not really a device at all, but is rather an implant, surely we would not grant law enforcement access to our minds.
  • Policymakers should dismiss Vance’s proposal to prohibit the use of strong encryption to protect our smartphones and devices in order to ensure law enforcement access. Undermining encryption, regardless of whether it is protecting data in transit or at rest, would take us down a dangerous and harmful path. Instead, law enforcement and the intelligence community should be working to alter their skills and tactics in a fast-evolving technological world so that they are not so dependent on information that will increasingly be protected by encryption.
Paul Merrell

NSA Director Finally Admits Encryption Is Needed to Protect Public's Privacy - 0 views

  • NSA Director Finally Admits Encryption Is Needed to Protect Public’s Privacy The new stance denotes a growing awareness within the government that Americans are not comfortable with the State’s grip on their data. By Carey Wedler | AntiMedia | January 22, 2016 Share this article! https://mail.google.com/mail/?view=cm&fs=1&to&su=NSA%20Director%20Finally%20Admits%20Encryption%20Is%20Needed%20to%20Protect%20Public%E2%80%99s%20Privacy&body=http%3A%2F%2Fwww.mintpress
  • Rogers cited the recent Office of Personnel Management hack of over 20 million users as a reason to increase encryption rather than scale it back. “What you saw at OPM, you’re going to see a whole lot more of,” he said, referring to the massive hack that compromised the personal data about 20 million people who obtained background checks. Rogers’ comments, while forward-thinking, signify an about face in his stance on encryption. In February 2015, he said he “shares [FBI] Director [James] Comey’s concern” about cell phone companies’ decision to add encryption features to their products. Comey has been one loudest critics of encryption. However, Rogers’ comments on Thursday now directly conflict with Comey’s stated position. The FBI director has publicly chastised encryption, as well as the companies that provide it. In 2014, he claimed Apple’s then-new encryption feature could lead the world to “a very dark place.” At a Department of Justice hearing in November, Comey testified that “Increasingly, the shadow that is ‘going dark’ is falling across more and more of our work.” Though he claimed, “We support encryption,” he insisted “we have a problem that encryption is crashing into public safety and we have to figure out, as people who care about both, to resolve it. So, I think the conversation’s in a healthier place.”
  • At the same hearing, Comey and Attorney General Loretta Lynch declined to comment on whether they had proof the Paris attackers used encryption. Even so, Comey recently lobbied for tech companies to do away with end-to-end encryption. However, his crusade has fallen on unsympathetic ears, both from the private companies he seeks to control — and from the NSA. Prior to Rogers’ statements in support of encryption Thursday, former NSA chief Michael Hayden said, “I disagree with Jim Comey. I actually think end-to-end encryption is good for America.” Still another former NSA chair has criticized calls for backdoor access to information. In October, Mike McConnell told a panel at an encryption summit that the United States is “better served by stronger encryption, rather than baking in weaker encryption.” Former Department of Homeland Security chief, Michael Chertoff, has also spoken out against government being able to bypass encryption.
  • ...2 more annotations...
  • Regardless of these individual defenses of encryption, the Intercept explained why these statements may be irrelevant: “Left unsaid is the fact that the FBI and NSA have the ability to circumvent encryption and get to the content too — by hacking. Hacking allows law enforcement to plant malicious code on someone’s computer in order to gain access to the photos, messages, and text before they were ever encrypted in the first place, and after they’ve been decrypted. The NSA has an entire team of advanced hackers, possibly as many as 600, camped out at Fort Meade.”
  • Rogers statements, of course, are not a full-fledged endorsement of privacy, nor can the NSA be expected to make it a priority. Even so, his new stance denotes a growing awareness within the government that Americans are not comfortable with the State’s grip on their data. “So spending time arguing about ‘hey, encryption is bad and we ought to do away with it’ … that’s a waste of time to me,” Rogers said Thursday. “So what we’ve got to ask ourselves is, with that foundation, what’s the best way for us to deal with it? And how do we meet those very legitimate concerns from multiple perspectives?”
Paul Merrell

NAS Report: A New Light in the Debate over Government Access to Encrypted Content - Lawfare - 0 views

  • The encryption debate dates back to Clinton administration proposals for the “clipper chip” and mandatory deposit of decryption keys. But that debate reached new prominence in connection with the FBI’s efforts to compel Apple to decrypt the phone of a dead terrorist in the San Bernardino case. A new study by the National Academies of Sciences, Engineering, and Medicine tries to shed some light, and turn down the heat, in the debate over whether government agencies should be provided access to plaintext versions of encrypted communications and other data. FBI and other law enforcement officials, and some intelligence officials, have argued that in the face of widespread encryption provided by smart phones, messaging apps, and other devices and software, the internet is “going dark.” These officials warn that encryption is restricting their access to information needed for criminal and national security investigations, arguing that they need a reliable, timely and scalable way to access it. Critics have raised legal and practical objections that regulations to ensure government access would pose unacceptable risks to privacy and civil liberties and undermine computer security in the face of rising cyber threats, and may be less necessary given the wider availability of data and alternative means of obtaining access to encrypted data. As the encryption debate has become increasingly polarized with participants on all sides making sweeping, sometimes absolutist, assertions, the new National Academies’ report doesn’t purport to tell anyone what to do, but rather provides a primer on the relevant issues.
Paul Merrell

Zuckerberg set up fraudulent scheme to 'weaponise' data, court case alleges | Technology | The Guardian - 1 views

  • Mark Zuckerberg faces allegations that he developed a “malicious and fraudulent scheme” to exploit vast amounts of private data to earn Facebook billions and force rivals out of business. A company suing Facebook in a California court claims the social network’s chief executive “weaponised” the ability to access data from any user’s network of friends – the feature at the heart of the Cambridge Analytica scandal. A legal motion filed last week in the superior court of San Mateo draws upon extensive confidential emails and messages between Facebook senior executives including Mark Zuckerberg. He is named individually in the case and, it is claimed, had personal oversight of the scheme. Facebook rejects all claims, and has made a motion to have the case dismissed using a free speech defence.
  • It claims the first amendment protects its right to make “editorial decisions” as it sees fit. Zuckerberg and other senior executives have asserted that Facebook is a platform not a publisher, most recently in testimony to Congress.
  • Heather Whitney, a legal scholar who has written about social media companies for the Knight First Amendment Institute at Columbia University, said, in her opinion, this exposed a potential tension for Facebook. “Facebook’s claims in court that it is an editor for first amendment purposes and thus free to censor and alter the content available on its site is in tension with their, especially recent, claims before the public and US Congress to be neutral platforms.” The company that has filed the case, a former startup called Six4Three, is now trying to stop Facebook from having the case thrown out and has submitted legal arguments that draw on thousands of emails, the details of which are currently redacted. Facebook has until next Tuesday to file a motion requesting that the evidence remains sealed, otherwise the documents will be made public.
Paul Merrell

Facebook unveils cryptocurrency Libra | Time - 0 views

  • As it continues to explore new business models that may work in a world focused on privacy rather than broadly sharing data online, Facebook on Tuesday revealed plans for its own global digital currency, Libra, which aims to allow users to make purchases or send money with close to zero transaction fees. Facebook said it hopes Libra will make it easier for the estimated 1.7 billion unbanked adults worldwide to access banking services and transfer money electronically. “Just as people can use their phones to message friends anywhere in the world today, with Libra, the same can be done with money — instantly, securely and at a low cost,” Facebook said in a Libra white paper.
  • Users will be able to make transactions with Libra by 2020, Facebook says, both through a standalone app called Calibra as well as with Facebook’s own family of apps. Unlike bitcoin and some similar offerings, the price of Libra is tied to low-risk assets, which should prevent the speculative behavior and wild price swings plaguing other digital currencies. The currency will be overseen by the Libra Association, a Switzerland-based collective of more than a dozen companies, including Visa, Uber and Mastercard, each of whom have invested at least $10 million in the technology. Users of Libra will pay $1 to use Libra which will sit in a bank account and earn interest which will pay the Libra Association.
Paul Merrell

Banning end-to-end encryption being considered by Trump team- 9to5Mac - 0 views

  • The Trump administration is considering the possibility of banning end-to-end encryption, as used by services like Apple’s Messages and FaceTime, as well as competing platforms like WhatsApp and Signal. The topic was reportedly the main topic of a previously-unreported meeting of a National Security Council meeting on Wednesday … NordVPN Politico cites three sources for the story. Senior Trump administration officials met on Wednesday to discuss whether to seek legislation prohibiting tech companies from using forms of encryption that law enforcement can’t break — a provocative step that would reopen a long-running feud between federal authorities and Silicon Valley. The encryption challenge, which the government calls “going dark,” was the focus of a National Security Council meeting Wednesday morning that included the No. 2 officials from several key agencies, according to three people familiar with the matter. The meeting reportedly discussed two options. Senior officials debated whether to ask Congress to effectively outlaw end-to-end encryption, which scrambles data so that only its sender and recipient can read it […] “The two paths were to either put out a statement or a general position on encryption, and [say] that they would continue to work on a solution, or to ask Congress for legislation,” said one of the people. No decision was reached given strongly opposing views within the government.
Paul Merrell

Time to 'Break Facebook Up,' Sanders Says After Leaked Docs Show Social Media Giant 'Treated User Data as a Bargaining Chip' | Common Dreams News - 0 views

  • After NBC News on Wednesday published a trove of leaked documents that show how Facebook "treated user data as a bargaining chip with external app developers," White House hopeful Sen. Bernie Sanders declared that it is time "to break Facebook up."
  • When British investigative journalist Duncan Campbell first shared the trove of documents with a handful of media outlets including NBC News in April, journalists Olivia Solon and Cyrus Farivar reported that "Facebook CEO Mark Zuckerberg oversaw plans to consolidate the social network's power and control competitors by treating its users' data as a bargaining chip, while publicly proclaiming to be protecting that data." With the publication Wednesday of nearly 7,000 pages of records—which include internal Facebook emails, web chats, notes, presentations, and spreadsheets—journalists and the public can now have a closer look at exactly how the company was using the vast amount of data it collects when it came to bargaining with third parties.
  • The document dump comes as Facebook and Zuckerberg are facing widespread criticism over the company's political advertising policy, which allows candidates for elected office to lie in the ads they pay to circulate on the platform. It also comes as 47 state attorneys general, led by Letitia James of New York, are investigating the social media giant for antitrust violations.
  • ...2 more annotations...
  • According to Solon and Farivar of NBC: Taken together, they show how Zuckerberg, along with his board and management team, found ways to tap Facebook users' data—including information about friends, relationships, and photos—as leverage over the companies it partnered with. In some cases, Facebook would reward partners by giving them preferential access to certain types of user data while denying the same access to rival companies. For example, Facebook gave Amazon special access to user data because it was spending money on Facebook advertising. In another case the messaging app MessageMe was cut off from access to data because it had grown too popular and could compete with Facebook.
  • The call from Sanders (I-Vt.) Wednesday to break up Facebook follows similar but less definitive statements from the senator. One of Sanders' rivals in the 2020 Democratic presidential primary race, Sen. Elizabeth Warren (D-Mass.), released her plan to "Break Up Big Tech" in March. Zuckerberg is among the opponents of Warren's proposal, which also targets other major technology companies like Amazon and Google.
Paul Merrell

UK government is secretly planning to break encryption and spy on people's phones, reveals leaked document | The Independent - 0 views

  • The UK government is secretly planning to force technology companies to build backdoors into their products, to enable intelligence agencies to read people’s private messages. A draft document leaked by the Open Rights Group details extreme new surveillance proposals, which would enable government agencies to spy on one in 10,000 citizens – around 6,500 people – at any one time.  The document, which follows the controversial Investigatory Powers Act, reveals government plans to force mobile operators and internet service providers to provide real-time communications of customers to the government “in an intelligible form”, and within one working day.
  • This would effectively ban encryption, an important security measure used by a wide range of companies, including WhatsApp and major banks, to keep people’s private data private and to protect them from hackers and cyber criminals. 
Paul Merrell

WhatsApp sues Israel's NSO for allegedly helping spies hack phones around the world - Reuters - 0 views

  • WhatsApp sued Israeli surveillance firm NSO Group on Tuesday, accusing it of helping government spies break into the phones of roughly 1,400 users across four continents in a hacking spree whose targets included diplomats, political dissidents, journalists and senior government officials.
  • In a lawsuit filed in federal court in San Francisco, messaging service WhatsApp, which is owned by Facebook Inc (FB.O), accused NSO of facilitating government hacking sprees in 20 countries. Mexico, the United Arab Emirates and Bahrain were the only countries identified. WhatsApp said in a statement that 100 civil society members had been targeted, and called it “an unmistakable pattern of abuse.” NSO denied the allegations.
  • Citizen Lab, a cybersecurity research laboratory based at the University of Toronto that worked with WhatsApp to investigate the phone hacking, told Reuters that the targets included well-known television personalities, prominent women who had been subjected to online hate campaigns and people who had faced “assassination attempts and threats of violence.”
  • ...1 more annotation...
  • NSO came under particularly harsh scrutiny over the allegation that its spyware played a role in the death of Washington Post journalist Jamal Khashoggi, who was murdered at the Saudi Consulate in Istanbul a little over a year ago. Khashoggi’s friend Omar Abdulaziz is one of seven activists and journalists who have taken the spyware firm to court in Israel and Cyprus over allegations that their phones were compromised using NSO technology. Amnesty has also filed a lawsuit, demanding that the Israeli Ministry of Defense revoke NSO’s export license to “stop it profiting from state-sponsored repression.”
« First ‹ Previous 101 - 118 of 118
Showing 20 items per page