Group items tagged

It has recruited talent from Microsoft, Google, Amazon.com and Myspace and is using a powerful, multiplatform approach to predicting consumer behavior that could raise its standing among investors and clients.

Acxiom has its own classification system, PersonicX, which assigns consumers to one of 70 detailed socioeconomic clusters and markets to them accordingly. In this situation, it pegs Mr. Hughes as a “savvy single” — meaning he’s in a cluster of mobile, upper-middle-class people who do their banking online, attend pro sports events, are sensitive to prices — and respond to free-shipping offers.

Analysts say companies design these sophisticated ecosystems to prompt consumers to volunteer enough personal data — like their names, e-mail addresses and mobile numbers — so that marketers can offer them customized appeals any time, anywhere.

Acxiom maintains its own database on about 190 million individuals and 126 million households in the United States. Separately, it manages customer databases for or works with 47 of the Fortune 100 companies. It also worked with the government after the September 2001 terrorist attacks

This year, Advertising Age ranked Epsilon, another database marketing firm, as the biggest advertising agency in the United States, with Acxiom second.

race coding may be incorrect. And even if a data broker has correct information, a person may not want to be marketed to based on race.

if marketing algorithms judge certain people as not worthy of receiving promotions for higher education or health services, they could have a serious impact.

“Over time, that can really turn into a mountain of pathways not offered, not seen and not known about,”

Unlike consumer reporting agencies that sell sensitive financial information about people for credit or employment purposes, database marketers aren’t required by law to show consumers their own reports and allow them to correct errors.

ACXIOM’S Consumer Data Products Catalog offers hundreds of details — called “elements” — that corporate clients can buy about individuals or households, to augment their own marketing databases.

the catalog also offers delicate information that has set off alarm bells among some privacy advocates, who worry about the potential for misuse by third parties that could take aim at vulnerable groups. Such information includes consumers’ interests — derived, the catalog says, “from actual purchases and self-reported surveys” — like “Christian families,” “Dieting/Weight Loss,” “Gaming-Casino,” “Money Seekers” and “Smoking/Tobacco.” Acxiom also sells data about an individual’s race, ethnicity and country of origin. “Our Race model,” the catalog says, “provides information on the major racial category: Caucasians, Hispanics, African-Americans, or Asians.” Competing companies sell similar data.

“At the same time, this is ethnic profiling,” he says. “The people on this list, they are being sold based on their ethnic stereotypes. There is a very strong citizen’s right to have a veto over the commodification of their profile.”

it’s as if the ore of our data-driven lives were being mined, refined and sold to the highest bidder, usually without our knowledge — by companies that most people rarely even know exist.

In its system, a store clerk need only “capture the shopper’s name from a check or third-party credit card at the point of sale and then ask for the shopper’s ZIP code or telephone number.” With that data Acxiom can identify shoppers within a 10 percent margin of error, it says, enabling stores to reward their best customers with special offers. Other companies offer similar services. “This is a direct way of circumventing people’s concerns about privacy,” says Mr. Chester of the Center for Digital Democracy.

When you click to show the list of people who “liked” a post, you probably think you’re getting a list of people who expressed positive sentiment toward a particular combination of sentences. But computationally, what you’re getting is slightly different. “Show me a list of the people who liked a Facebook post” is actually a command more like “Display a list of FirstName and LastName for usernames where the flag LikedThisPost = True.” There are a lot of assumptions built in there.

There’s the assumption that the username corresponds to a single real person, which is not always true—people have multiple Facebook accounts, and some accounts have multiple people posting to them, and some accounts are fake

There’s the assumption that the agent that clicked the like button is the same person referred to by the username—not necessarily true. People often forget to log out of their Facebook accounts on desktops and laptops, or use other people’s phones to browse social media

A like button is merely a tool. Humans use tools in breathtakingly creative ways—this is one of the many exciting and inspiring things about social uses of technology. However, the meaning a person imparts to an action on a social-media platform does not always correspond to the actual intent. In real-life social interactions, nuance is everything. On social media, where that nuance is obscured, we ought to be hyper-critical about the ethical ramifications of using social media data for real-world judgments.

For the kid listed in a gang database, it can be unclear how to get out of it. In the world of human interaction, we accept change through behavior: the addict can redeem himself by getting clean, or the habitual interrupter can redeem himself by not interrupting. We accept behavior change. But in the database world, unless someone has permission to delete or amend a database record, no such change is possible

The National Gang Center, in its list of gang-related legislation, shows only 12 states with policies that specifically address gang databases. Most deny the public access to the information in these databases. Only a few of these twelve mention regular purging of information, and some specifically say that a person cannot even find out if they have a record in the database

. “The last time that the gang unit purged its files, however, was in 1993—approximately 4 years before this study was conducted,” he wrote. “One clerk who is responsible for data entry and dissemination estimated, ‘At a minimum, 400 to 500 gang members would be deleted off the gang list today if we went through the files.

n predict what book I want to buy next, hasn’t this problem already been solved?” The answer is inevitably no. When computer scientists were building the Internet in the late 1990s, there weren’t any widely adopted or established ethical guidelines because we were building these systems for the first time in human history.

We also need to acknowledge that computer systems are not a panacea. “Your program really does stink, and the sooner you get used to the idea, the better,” writes Nathan S. Borenstein in Programming as if People Mattered. “The inadequacies of your software are simply a reflection of your frail, shortsighted, and limited human nature. Every program ever built is doomed to eventual obsolescence.” We need to put people before programs, and if programs don’t reflect our human values, we need to change the code. And if programmers can’t write code that is fair and just, we should consider relying on people instead of programs.

If American law enforcement is going to go deeper into the brave new world of data-driven policing, we need to create systems that have human values embedded in them. If our technological systems are entrapping innocent citizens or tampering with the presumption of innocence, should they be used?

just as we saw in the summer with immigration writ large, the progression doesn't end with Trump. We've had three presidential elections since the 9/11 terror attacks and no presidential candidate has ever proposed shutting down mosques in the United States or creating a special registry and identification cards for Muslims living in the United States.

So yesterday Megyn Kelly asked Marco Rubio whether he'd shut down radical mosques like Trump. He tried to deflect the question by saying that it wasn't about mosques but closing down any facility that was promoting radicalism. In other words, Rubio, while clearly not eager to answer the question, pointedly refused to rule out following Trump's lead.

It is a very good example of how Trump is not only shaping the debate on the right but rapidly mainstreaming ideas that were as recently as a week ago considered entirely outside the realm of mainstream political discourse.

It's particularly effective with the less sophisticated and principled candidates like Rubio. Jeb Bush said flatly this morning that Trump's database proposal is "just wrong." But Ben Carson quickly took Trump's lead comparing Syrian refugees to "mad dogs." The difference is that Marco Rubio could very well be president in 18 months. Jeb Bush won't be.

this is no longer a matter of Trump yakking on about building a gilded 100-foot wall along the southern border and having Mexico agree to pay for it. Trump is now proposing things that sound like they put millions of American citizens and resident aliens on a road to something like the internment of Japanese Americans during World War II.

Lately, it has also become one of the most controversial. The algorithm has been found to be promoting conspiracy theories about the Las Vegas mass shooting and incentivising, through recommendations, a thriving subculture that targets children with disturbing content

One YouTube creator who was banned from making advertising revenues from his strange videos – which featured his children receiving flu shots, removing earwax, and crying over dead pets – told a reporter he had only been responding to the demands of Google’s algorithm. “That’s what got us out there and popular,” he said. “We learned to fuel it and do whatever it took to please the algorithm.”

academics have speculated that YouTube’s algorithms may have been instrumental in fuelling disinformation during the 2016 presidential election. “YouTube is the most overlooked story of 2016,” Zeynep Tufekci, a widely respected sociologist and technology critic, tweeted back in October. “Its search and recommender algorithms are misinformation engines.”

Those are not easy questions to answer. Like all big tech companies, YouTube does not allow us to see the algorithms that shape our lives. They are secret formulas, proprietary software, and only select engineers are entrusted to work on the algorithm

Guillaume Chaslot, a 36-year-old French computer programmer with a PhD in artificial intelligence, was one of those engineers.

The experience led him to conclude that the priorities YouTube gives its algorithms are dangerously skewed.

“YouTube is something that looks like reality, but it is distorted to make you spend more time online,” he tells me when we meet in Berkeley, California. “The recommendation algorithm is not optimising for what is truthful, or balanced, or healthy for democracy.”

Chaslot explains that the algorithm never stays the same. It is constantly changing the weight it gives to different signals: the viewing patterns of a user, for example, or the length of time a video is watched before someone clicks away.

The engineers he worked with were responsible for continuously experimenting with new formulas that would increase advertising revenues by extending the amount of time people watched videos. “Watch time was the priority,” he recalls. “Everything else was considered a distraction.”

Chaslot was fired by Google in 2013, ostensibly over performance issues. He insists he was let go after agitating for change within the company, using his personal time to team up with like-minded engineers to propose changes that could diversify the content people see.

He was especially worried about the distortions that might result from a simplistic focus on showing people videos they found irresistible, creating filter bubbles, for example, that only show people content that reinforces their existing view of the world.

Chaslot said none of his proposed fixes were taken up by his managers. “There are many ways YouTube can change its algorithms to suppress fake news and improve the quality and diversity of videos people see,” he says. “I tried to change YouTube from the inside but it didn’t work.”

YouTube told me that its recommendation system had evolved since Chaslot worked at the company and now “goes beyond optimising for watchtime”.

It did not say why Google, which acquired YouTube in 2006, waited over a decade to make those changes

Chaslot believes such changes are mostly cosmetic, and have failed to fundamentally alter some disturbing biases that have evolved in the algorithm

It finds videos through a word search, selecting a “seed” video to begin with, and recording several layers of videos that YouTube recommends in the “up next” column. It does so with no viewing history, ensuring the videos being detected are YouTube’s generic recommendations, rather than videos personalised to a user. And it repeats the process thousands of times, accumulating layers of data about YouTube recommendations to build up a picture of the algorithm’s preferences.

Each study finds something different, but the research suggests YouTube systematically amplifies videos that are divisive, sensational and conspiratorial.

When his program found a seed video by searching the query “who is Michelle Obama?” and then followed the chain of “up next” suggestions, for example, most of the recommended videos said she “is a man”

He believes one of the most shocking examples was detected by his program in the run-up to the 2016 presidential election. As he observed in a short, largely unnoticed blogpost published after Donald Trump was elected, the impact of YouTube’s recommendation algorithm was not neutral during the presidential race: it was pushing videos that were, in the main, helpful to Trump and damaging to Hillary Clinton.

“It was strange,” he explains to me. “Wherever you started, whether it was from a Trump search or a Clinton search, the recommendation algorithm was much more likely to push you in a pro-Trump direction.”

Trump won the electoral college as a result of 80,000 votes spread across three swing states. There were more than 150 million YouTube users in the US. The videos contained in Chaslot’s database of YouTube-recommended election videos were watched, in total, more than 3bn times before the vote in November 2016.

“Algorithms that shape the content we see can have a lot of impact, particularly on people who have not made up their mind,”

“Gentle, implicit, quiet nudging can over time edge us toward choices we might not have otherwise made.”

“This research captured the apparent direction of YouTube’s political ecosystem,” he says. “That has not been done before.”

I spent weeks watching, sorting and categorising the trove of videos with Erin McCormick, an investigative reporter and expert in database analysis. From the start, we were stunned by how many extreme and conspiratorial videos had been recommended, and the fact that almost all of them appeared to be directed against Clinton.

But what was most compelling was how often Chaslot’s software detected anti-Clinton conspiracy videos appearing “up next” beside other videos.

There were too many videos in the database for us to watch them all, so we focused on 1,000 of the top-recommended videos. We sifted through them one by one to determine whether the content was likely to have benefited Trump or Clinton. Just over a third of the videos were either unrelated to the election or contained content that was broadly neutral or even-handed. Of the remaining 643 videos, 551 were videos favouring Trump, while only only 92 favoured the Clinton campaign.

The sample we had looked at suggested Chaslot’s conclusion was correct: YouTube was six times more likely to recommend videos that aided Trump than his adversary.

The spokesperson added: “Our search and recommendation systems reflect what people search for, the number of videos available, and the videos people choose to watch on YouTube. That’s not a bias towards any particular candidate; that is a reflection of viewer interest.”

YouTube seemed to be saying that its algorithm was a neutral mirror of the desires of the people who use it – if we don’t like what it does, we have ourselves to blame. How does YouTube interpret “viewer interest” – and aren’t “the videos people choose to watch” influenced by what the company shows them?

Offered the choice, we may instinctively click on a video of a dead man in a Japanese forest, or a fake news clip claiming Bill Clinton raped a 13-year-old. But are those in-the-moment impulses really a reflect of the content we want to be fed?

YouTube’s recommendation system has probably figured out that edgy and hateful content is engaging. “This is a bit like an autopilot cafeteria in a school that has figured out children have sweet teeth, and also like fatty and salty foods,” she says. “So you make a line offering such food, automatically loading the next plate as soon as the bag of chips or candy in front of the young person has been consumed.”

Once that gets normalised, however, what is fractionally more edgy or bizarre becomes, Tufekci says, novel and interesting. “So the food gets higher and higher in sugar, fat and salt – natural human cravings – while the videos recommended and auto-played by YouTube get more and more bizarre or hateful.”

“This is important research because it seems to be the first systematic look into how YouTube may have been manipulated,” he says, raising the possibility that the algorithm was gamed as part of the same propaganda campaigns that flourished on Twitter and Facebook.

“We believe that the activity we found was limited because of various safeguards that we had in place in advance of the 2016 election, and the fact that Google’s products didn’t lend themselves to the kind of micro-targeting or viral dissemination that these actors seemed to prefer.”

Senator Mark Warner, the ranking Democrat on the intelligence committee, later wrote to the company about the algorithm, which he said seemed “particularly susceptible to foreign influence”. The senator demanded to know what the company was specifically doing to prevent a “malign incursion” of YouTube’s recommendation system. Walker, in his written reply, offered few specifics

Tristan Harris, a former Google insider turned tech whistleblower, likes to describe Facebook as a “living, breathing crime scene for what happened in the 2016 election” that federal investigators have no access to. The same might be said of YouTube. About half the videos Chaslot’s program detected being recommended during the election have now vanished from YouTube – many of them taken down by their creators. Chaslot has always thought this suspicious. These were videos with titles such as “Must Watch!! Hillary Clinton tried to ban this video”, watched millions of times before they disappeared. “Why would someone take down a video that has been viewed millions of times?” he asks

In every case, the largest source of traffic – the invisible force – came from the clips appearing in the “up next” column. William Ramsey, an occult investigator from southern California who made “Irrefutable Proof: Hillary Clinton Has a Seizure Disorder!”, shared screen grabs that showed the recommendation algorithm pushed his video even after YouTube had emailed him to say it violated its guidelines. Ramsey’s data showed the video was watched 2.4m times by US-based users before election day. “For a nobody like me, that’s a lot,” he says. “Enough to sway the election, right?”

“I don’t have smoking-gun proof of who logged in to control those accounts,” he says. “But judging from the history of what we’ve seen those accounts doing before, and the characteristics of how they tweet and interconnect, they are assembled and controlled by someone – someone whose job was to elect Trump.”

After the Senate’s correspondence with Google over possible Russian interference with YouTube’s recommendation algorithm was made public last week, YouTube sent me a new statement. It emphasised changes it made in 2017 to discourage the recommendation system from promoting some types of problematic content. “We appreciate the Guardian’s work to shine a spotlight on this challenging issue,” it added. “We know there is more to do here and we’re looking forward to making more announcements in the months ahead.”

In the months leading up to the election, the Next News Network turned into a factory of anti-Clinton news and opinion, producing dozens of videos a day and reaching an audience comparable to that of MSNBC’s YouTube channel. Chaslot’s research indicated Franchi’s success could largely be credited to YouTube’s algorithms, which consistently amplified his videos to be played “up next”. YouTube had sharply dismissed Chaslot’s research.

I contacted Franchi to see who was right. He sent me screen grabs of the private data given to people who upload YouTube videos, including a breakdown of how their audiences found their clips. The largest source of traffic to the Bill Clinton rape video, which was viewed 2.4m times in the month leading up to the election, was YouTube recommendations.

The same was true of all but one of the videos Franchi sent me data for. A typical example was a Next News Network video entitled “WHOA! HILLARY THINKS CAMERA’S OFF… SENDS SHOCK MESSAGE TO TRUMP” in which Franchi, pointing to a tiny movement of Clinton’s lips during a TV debate, claims she says “fuck you” to her presidential rival. The data Franchi shared revealed in the month leading up to the election, 73% of the traffic to the video – amounting to 1.2m of its views – was due to YouTube recommendations. External traffic accounted for only 3% of the views.

many of the other creators of anti-Clinton videos I spoke to were amateur sleuths or part-time conspiracy theorists. Typically, they might receive a few hundred views on their videos, so they were shocked when their anti-Clinton videos started to receive millions of views, as if they were being pushed by an invisible force.

I shared the entire database of 8,000 YouTube-recommended videos with John Kelly, the chief executive of the commercial analytics firm Graphika, which has been tracking political disinformation campaigns. He ran the list against his own database of Twitter accounts active during the election, and concluded many of the videos appeared to have been pushed by networks of Twitter sock puppets and bots controlled by pro-Trump digital consultants with “a presumably unsolicited assist” from Russia.

Daniel Alexander Cannon, a conspiracy theorist from South Carolina, tells me: “Every video I put out about the Clintons, YouTube would push it through the roof.” His best-performing clip was a video titled “Hillary and Bill Clinton ‘The 10 Photos You Must See’”, essentially a slideshow of appalling (and seemingly doctored) images of the Clintons with voiceover in which Cannon speculates on their health. It has been seen 3.7m times on YouTube, and 2.9m of those views, Cannon said, came from “up next” recommendations.

his research also does something more important: revealing how thoroughly our lives are now mediated by artificial intelligence.

Less than a generation ago, the way voters viewed their politicians was largely shaped by tens of thousands of newspaper editors, journalists and TV executives. Today, the invisible codes behind the big technology platforms have become the new kingmakers.

They pluck from obscurity people like Dave Todeschini, a retired IBM engineer who, “let off steam” during the election by recording himself opining on Clinton’s supposed involvement in paedophilia, child sacrifice and cannibalism. “It was crazy, it was nuts,” he said of the avalanche of traffic to his YouTube channel, which by election day had more than 2m views

it’s not just law enforcement: Clearview has also licensed the app to at least a handful of companies for security purposes.

“The weaponization possibilities of this are endless,” said Eric Goldman, co-director of the High Tech Law Institute at Santa Clara University. “Imagine a rogue law enforcement officer who wants to stalk potential romantic partners, or a foreign government using this to dig up secrets about people to blackmail them or throw them in jail.”

While the company was dodging me, it was also monitoring me. At my request, a number of police officers had run my photo through the Clearview app. They soon received phone calls from company representatives asking if they were talking to the media — a sign that Clearview has the ability and, in this case, the appetite to monitor whom law enforcement is searching for.

The company eventually started answering my questions, saying that its earlier silence was typical of an early-stage start-up in stealth mode. Mr. Ton-That acknowledged designing a prototype for use with augmented-reality glasses but said the company had no plans to release it.

In addition to Mr. Ton-That, Clearview was founded by Richard Schwartz — who was an aide to Rudolph W. Giuliani when he was mayor of New York — and backed financially by Peter Thiel, a venture capitalist behind Facebook and Palantir.

“I’ve come to the conclusion that because information constantly increases, there’s never going to be privacy,” Mr. Scalzo said. “Laws have to determine what’s legal, but you can’t ban technology. Sure, that might lead to a dystopian future or something, but you can’t ban it.”

“In 2017, Peter gave a talented young founder $200,000, which two years later converted to equity in Clearview AI,” said Jeremiah Hall, Mr. Thiel’s spokesman. “That was Peter’s only contribution; he is not involved in the company.”

He began in 2016 by recruiting a couple of engineers. One helped design a program that can automatically collect images of people’s faces from across the internet, such as employment sites, news sites, educational sites, and social networks including Facebook, YouTube, Twitter, Instagram and even Venmo

Representatives of those companies said their policies prohibit such scraping, and Twitter said it explicitly banned use of its data for facial recognition

Another engineer was hired to perfect a facial recognition algorithm that was derived from academic papers. The result: a system that uses what Mr. Ton-That described as a “state-of-the-art neural net” to convert all the images into mathematical formulas, or vectors, based on facial geometry — like how far apart a person’s eyes are

Clearview created a vast directory that clustered all the photos with similar vectors into “neighborhoods.”

When a user uploads a photo of a face into Clearview’s system, it converts the face into a vector and then shows all the scraped photos stored in that vector’s neighborhood — along with the links to the sites from which those images came.

Mr. Schwartz paid for server costs and basic expenses, but the operation was bare bones; everyone worked from home. “I was living on credit card debt,” Mr. Ton-That said. “Plus, I was a Bitcoin believer, so I had some of those.”

The company soon changed its name to Clearview AI and began marketing to law enforcement. That was when the company got its first round of funding from outside investors: Mr. Thiel and Kirenaga Partners

Mr. Schwartz and Mr. Ton-That met in 2016 at a book event at the Manhattan Institute, a conservative think tank. Mr. Schwartz, now 61, had amassed an impressive Rolodex working for Mr. Giuliani in the 1990s and serving as the editorial page editor of The New York Daily News in the early 2000s. The two soon decided to go into the facial recognition business together: Mr. Ton-That would build the app, and Mr. Schwartz would use his contacts to drum up commercial interest.

They immediately got a match: The man appeared in a video that someone had posted on social media, and his name was included in a caption on the video. “He did not have a driver’s license and hadn’t been arrested as an adult, so he wasn’t in government databases,”

The man was arrested and charged; Mr. Cohen said he probably wouldn’t have been identified without the ability to search social media for his face. The Indiana State Police became Clearview’s first paying customer, according to the company

“I don’t see a future where we harness the benefits of face recognition technology without the crippling abuse of the surveillance that comes with it. The only way to stop it is to ban it.”

The company’s most effective sales technique was offering 30-day free trials to officers, who then encouraged their acquisition departments to sign up and praised the tool to officers from other police departments at conferences and online, according to the company and documents provided by police departments in response to public-record requests. Mr. Ton-That finally had his viral hit.

Photos “could be covertly taken with telephoto lens and input into the software, without ‘burning’ the surveillance operation,” the detective wrote in the email, provided to The Times by two researchers,

Sergeant Ferrara found Clearview’s app superior, he said. Its nationwide database of images is much larger, and unlike FACES, Clearview’s algorithm doesn’t require photos of people looking straight at the camera.

“With Clearview, you can use photos that aren’t perfect,” Sergeant Ferrara said. “A person can be wearing a hat or glasses, or it can be a profile shot or partial view of their face.”

Mr. Ton-That said the tool does not always work. Most of the photos in Clearview’s database are taken at eye level. Much of the material that the police upload is from surveillance cameras mounted on ceilings or high on walls.

Despite that, the company said, its tool finds matches up to 75 percent of the time. But it is unclear how often the tool delivers false matches, because it has not been tested by an independent party

One reason that Clearview is catching on is that its service is unique. That’s because Facebook and other social media sites prohibit people from scraping users’ images — Clearview is violating the sites’ terms of service.

Some law enforcement officials said they didn’t realize the photos they uploaded were being sent to and stored on Clearview’s servers. Clearview tries to pre-empt concerns with an F.A.Q. document given to would-be clients that says its customer-support employees won’t look at the photos that the police upload.

Mr. Clement, now a partner at Kirkland & Ellis, wrote that the authorities don’t have to tell defendants that they were identified via Clearview, as long as it isn’t the sole basis for getting a warrant to arrest them.

Because the police upload photos of people they’re trying to identify, Clearview possesses a growing database of individuals who have attracted attention from law enforcement. The company also has the ability to manipulate the results that the police see.

After the company realized I was asking officers to run my photo through the app, my face was flagged by Clearview’s systems and for a while showed no matches. When asked about this, Mr. Ton-That laughed and called it a “software bug.”

“It’s creepy what they’re doing, but there will be many more of these companies. There is no monopoly on math,” said Al Gidari, a privacy professor at Stanford Law School. “Absent a very strong federal privacy law, we’re all screwed.”

But if your profile has already been scraped, it is too late. The company keeps all the images it has scraped even if they are later deleted or taken down, though Mr. Ton-That said the company was working on a tool that would let people request that images be removed if they had been taken down from the website of origin

Woodrow Hartzog, a professor of law and computer science at Northeastern University in Boston, sees Clearview as the latest proof that facial recognition should be banned in the United States.

We’ve relied on industry efforts to self-police and not embrace such a risky technology, but now those dams are breaking because there is so much money on the table,”

Clearview deployed current and former Republican officials to approach police forces, offering free trials and annual licenses for as little as $2,000. Mr. Schwartz tapped his political connections to help make government officials aware of the tool

Mr. Ton-That said he was reluctant. “There’s always going to be a community of bad people who will misuse it,” he said.

Even if Clearview doesn’t make its app publicly available, a copycat company might, now that the taboo is broken. Searching someone by face could become as easy as Googling a name

Someone walking down the street would be immediately identifiable — and his or her home address would be only a few clicks away. It would herald the end of public anonymity.

“That will give me an identity,” he said, gesturing at the computer station where he had just completed his enrollment. “It will show that I am a human being, that I am alive, that I live on this planet. It will prove I am an Indian.”

In War and Peace and War (2006), his most accessible book, he likens himself to Hari Seldon, the “maverick mathematician” of Isaac Asimov’s Foundation series, who can foretell the rise and fall of empires. In those 10,000 years’ worth of data, Turchin believes he has found iron laws that dictate the fates of human societies.

The fate of our own society, he says, is not going to be pretty, at least in the near term. “It’s too late,”

The problems are deep and structural—not the type that the tedious process of demo­cratic change can fix in time to forestall mayhem.

Turchin likens America to a huge ship headed directly for an iceberg: “If you have a discussion among the crew about which way to turn, you will not turn in time, and you hit the iceberg directly.” The past 10 years or so have been discussion. That sickening crunch you now hear—steel twisting, rivets popping—­­is the sound of the ship hitting the iceberg.

Of the three factors driving social violence, Turchin stresses most heavily “elite overproduction”—­the tendency of a society’s ruling classes to grow faster than the number of positions for their members to fill.

Don’t we want everyone to be rich and educated? The problems begin when money and Harvard degrees become like royal titles in Saudi Arabia. If lots of people have them, but only some have real power, the ones who don’t have power eventually turn on the ones who do.

“You have a situation now where there are many more elites fighting for the same position, and some portion of them will convert to counter-elites,” Turchin said.

Elite jobs do not multiply as fast as elites do. There are still only 100 Senate seats, but more people than ever have enough money or degrees to think they should be running the country

Turchin noted that a person can be part of an ideological elite rather than an economic one.

Elite overproduction creates counter-elites, and counter-elites look for allies among the commoners. If commoners’ living standards slip—not relative to the elites, but relative to what they had before—they accept the overtures of the counter-elites and start oiling the axles of their tumbrels

Commoners’ lives grow worse, and the few who try to pull themselves onto the elite lifeboat are pushed back into the water by those already aboard.

The final trigger of impending collapse, Turchin says, tends to be state insolvency. At some point rising in­security becomes expensive. The elites have to pacify unhappy citizens with handouts and freebies—and when these run out, they have to police dissent and oppress people.

Eventually the state exhausts all short-term solutions, and what was heretofore a coherent civilization disintegrates.

“I gave up because I solved the problem.” Turchin published one final monograph, Complex Population Dynamics: A Theoretical / Empirical Synthesis (2003), then broke the news to his UConn colleagues that he would be saying a permanent sayonara to the field

“Does population ecology have general laws?” Turchin asked. Most ecologists said no: Populations have their own dynamics, and each situation is different.

All sciences go through this transition to mathematization,” Turchin told me. “When I had my midlife crisis, I was looking for a subject where I could help with this transition to a mathematized science. There was only one left, and that was history.”

After its long adolescence of collecting and cataloging, ecology had enough data to describe these universal laws—and to stop pretending that every species had its own idiosyncrasies. “Ecologists know these laws and should call them laws,” he said.

The laws Turchin applied to ecology—and his insistence on calling them laws—­generated respectful controversy at the time. Now they are cited in textbooks.

Having left ecology, Turchin began similar research that attempted to formulate general laws for a different animal species: human beings.

Turchin suggested that “there are several very general law-like propositions” that could be applied to ecology.

If the historians weren’t going to usher in a mathematical revolution themselves, he would storm their departments and do it for them.

“There is a longstanding debate among scientists and philosophers as to whether history has general laws,” he and a co-author wrote in Secular Cycles (2009). “A basic premise of our study is that historical societies can be studied with the same methods physicists and biologists used to study natural systems.”

he likened historians reluctant to build general principles to his colleagues in biology “who care most for the private life of warblers.” “Let history continue to focus on the particular,” he wrote. Cliodynamics would be a new science

To seed the journal’s research, Turchin masterminded a digital archive of historical and archaeological data.

The coding of its records requires finesse, he told me, because (for example) the method of determining the size of the elite-aspirant class of medieval France might differ from the measure of the same class in the present-day United States. (For medieval France, a proxy is the membership in its noble class, which became glutted with second and third sons who had no castles or manors to rule over. One American proxy, Turchin says, is the number of lawyers.)

Historians of religion have long pondered the relationship between the rise of complex civilization and the belief in gods—especially “moralizing gods,” the kind who scold you for sinning. Last year, Turchin and a dozen co-authors mined the database (“records from 414 societies that span the past 10,000 years from 30 regions around the world, using 51 measures of social complexity and 4 measures of supernatural enforcement of morality”) to answer the question conclusively. They found that complex societies are more likely to have moralizing gods, but the gods tend to start their scolding after the societies get complex, not before

One of Turchin’s most unwelcome conclusions is that complex societies arise through war. The effect of war is to reward communities that organize themselves to fight and survive, and it tends to wipe out ones that are simple and small-scale

the data are clear: Darwinian processes select for complex socie­ties because they kill off simpler ones.

Certain aspects of this cyclical view require relearning portions of American history, with special attention paid to the numbers of elites. The industrialization of the North, starting in the mid-19th century, Turchin says, made huge numbers of people rich. The elite herd was culled during the Civil War, which killed off or impoverished the southern slaveholding class, and during Reconstruction, when America experienced a wave of assassinations of Republican politicians.

“There is a very close correlation between adopting democratic institutions and having to fight a war for survival.”

Also unwelcome: the conclusion that civil unrest might soon be upon us, and might reach the point of shattering the country. In 2012, Turchin published an analysis of political violence in the United States, again starting with a database. He classified 1,590 incidents—riots, lynchings, any political event that killed at least one person—from 1780 to 2010. Some periods were placid and others bloody, with peaks of brutality in 1870, 1920, and 1970, a 50-year cycle.

In 2010, when Nature surveyed scientists about their predictions for the coming decade, most took the survey as an invitation to self-promote and rhapsodize, dreamily, about coming advances in their fields. Turchin retorted with his prophecy of doom and said that nothing short of fundamental change would stop another violent turn.

It wasn’t until the Progressive reforms of the 1920s, and later the New Deal, that elite overproduction actually slowed, at least for a time.

This oscillation between violence and peace, with elite over­production as the first horseman of the recurring American apocalypse, inspired Turchin’s 2020 prediction

The notion that democracy finds its strength in its essential goodness and moral improvement over its rival systems is likewise fanciful. Instead, democratic societies flourish because they have a memory of being nearly obliterated by an external enemy. They avoided extinction only through collective action, and the memory of that collective action makes democratic politics easier to conduct in the present,

Some sound like ideas that might have come from Senator Elizabeth Warren—tax the elites until there are fewer of them

others, such as a call to reduce immigration to keep wages high for American workers, resemble Trumpian protectionism.

He opposes credential-­oriented higher education, for example, which he says is a way of mass-producing elites without also mass-­producing elite jobs for them to occupy. Architects of such policies, he told me, are “creating surplus elites, and some become counter-elites.” A smarter approach would be to keep the elite numbers small, and the real wages of the general population on a constant rise.

We need to stop the runaway process of elite overproduction, but I don’t know what will work to do that, and nobody else does. Do you increase taxation? Raise the minimum wage? Universal basic income?” He conceded that each of these possibilities would have unpredictable effects

practice “adaptive management,” changing and modulating your approach as you go.

Eventually, Turchin hopes, our understanding of historical dynamics will mature to the point that no government will make policy without reflecting on whether it is hurtling toward a mathematically pre­ordained disaster.

He says he could imagine an Asimovian agency that keeps tabs on leading indicators and advises accordingly. It would be like the Federal Reserve, but instead of monitoring inflation and controlling monetary supply, it would be tasked with averting total civilizational collapse.

Since at least the 19th century, the discipline has embraced the idea that history is irreducibly complex, and by now most historians believe that the diversity of human activity will foil any attempt to come up with general laws, especially predictive ones.

The idea that one thing causes another, and that the causal pattern can tell you about sequences of events in another place or century, is foreign territory.

Turchin counters that he has heard claims of irreducible complexity before, and that steady application of the scientific method has succeeded in managing that complexity.

One social scientist who can speak to Turchin in his own mathematical argot is Dingxin Zhao, a sociology professor at the University of Chicago who is—incredibly—­also a former mathematical ecologist.

in a way I am sympathetic to Turchin. If you come to social science from natural sciences, you have a powerful way of looking at the world. But you may also make big mistakes.”

Turchin is nonetheless filling a historiographical niche left empty by academic historians with allergies not just to science but to a wide-angle view of the past.

humans are much wilier social creatures, Zhao said. A woodpecker will eat a termite, but it “will not explain that he is doing so because it is his divine right.” Humans pull ideological power moves like this all the time, Zhao said, and to understand “the decisions of a Donald Trump, or a Xi Jinping,” a natural scientist has to incorporate the myriad complexities of human strategy, emotion, and belief. “I made that change,” Zhao told me, “and Peter Turchin has not.”

Zhao said that human beings are just much more complicated than bugs.

He places himself in a Russian tradition prone to thinking sweeping, Tolstoyan thoughts about the path of history. By comparison, American historians mostly look like micro-historians.

Rather than bending toward progress, the arc in his view bends all the way back on itself, in a never-­ending loop of boom and bust. This puts him at odds with American historians, many of whom harbor an unspoken faith that liberal democracy is the end state of all history.

Writing history in this sweeping, cyclical way is easier if you are trained outside the field. “If you look at who is doing these megahistories, more often than not, it’s not actual historians,”

The genre’s most famous book, Guns, Germs, and Steel (1997), beheld 13,000 years of human history in a single volume. Its author, Jared Diamond, spent the first half of his career as one of the world’s foremost experts on the physiology of the gall­bladder.

Steven Pinker, a cognitive psychologist who studies how children acquire parts of speech, has written a megahistory about the decline of violence across thousands of years, and about human flourishing since the Enlightenment

Most historians I asked about these men—and for some reason megahistory is nearly always a male pursuit—used terms like laughingstock and patently tendentious to describe them.

Pinker retorts that historians are resentful of the attention “disciplinary carpet­baggers” like himself have received for applying scientific methods to the humanities and coming up with conclusions that had eluded the old methods.

“Given the noisiness of human behavior and the prevalence of cognitive biases, it’s easy to delude oneself about a historical period or trend by picking whichever event suits one’s narrative,” he says. The only answer is to use large data sets.

“There’s no reason that traditional history and data science can’t merge into a cooperative enterprise,” Pinker wrote. “Knowing stuff is hard; we need to use every available tool.”

Turchin’s conclusions are only as good as his databases, she told me, and any database that tries to code something as complex as who constitutes a society’s elites—then tries to make like-to-like comparisons across millennia and oceans—will meet with skepticism from traditional historians

Turchin’s data are also limited to big-­picture characteristics observed over 10,000 years, or about 200 lifetimes. By scientific standards, a sample size of 200 is small, even if it is all humanity has.

Privately, some historians have told me they consider the tools he uses powerful, if a little crude. Clio­dynamics is now on a long list of methods that arrived on the scene promising to revolutionize history. Many were fads, but some survived that stage to take their rightful place in an expanding historiographical tool kit

Turchin’s methods have already shown their power. Cliodynamics offers scientific hypotheses, and human history will give us more and more opportunities to check its predictions—­revealing whether Peter Turchin is a Hari Seldon or a mere Nostradamus.

Terrorism, public health, and police violence are all life-and-death issues, and all involve the state, so they’re more consequential than the criticism, shunning, and loss of professional opportunities associated with cancel culture. But that doesn’t mean the latter isn’t a problem.

We can, and should, care about more than one thing at a time, and many things that aren’t the worst problem deserve attention.

Nevertheless, it’s important to assess problems accurately.

Michael Hobbes calls all this worrying about wokeness a “moral panic.” That’s a term some use online to wave away serious concerns, but Hobbes uses it the way sociologist Stanley Cohen did in the 1970s, as a phenomenon where something becomes “defined as a threat to societal values and interests” based on media accounts that “exaggerate the seriousness, extent, typicality and/or inevitability of harm.”

The point here is not that stranger abductions never happened, but that they didn’t happen nearly as much as the media, concerned parents, and lawmakers thought. And because stranger kidnappings were not a national crisis, but treated as one, the “solution” made things worse.

Along similar lines, Hobbes argues that anti-woke alarm-bell-ringing relies on a relatively small number of oft-repeated anecdotes. Some don’t stand up to scrutiny, and some of those that do are low-stakes. The resulting moral panic fuels, among other things, a wave of red state legislation aimed at banning “critical race theory” that uses vague language and effectively cracks down on teaching about racism in American history.

For that, we should look to data, and here again the problem looks smaller than anti-woke liberals make it out to be

In the universe of cancel culture cases, I find more incidents concerning than Hobbes and fewer concerning than Young, but “this one incident wasn’t actually bad” vs. “yes it really was” doesn’t answer the question about size and scope. It doesn’t tell us what, if anything, society should do about it.

In Liberal Currents, Adam Gurri cites the Foundation for Individual Rights in Education (FIRE), which documented 426 “targeting incidents involving scholars at public and private American institutions of higher education” since 2015 and 492 “disinvitation attempts” since 1998

The organization Canceled People lists 217 cases of “cancellation” since 1991, while the National Association of Scholars (NAS) lists 194 cancellations in academia since 2004 (plus two in the 20th century).

Based on these numbers, Gurri concludes, “If any other problem in social life was occurring at this frequency and at this scale, we would consider it effectively solved.”

There are nearly 4,000 colleges and universities in the United States. U.S. News’ 2021 rankings of the best schools lists 1,452. Using that smaller number and NAS’s figure of 194 academic cancellations since 2004, the chance of a college or university experiencing a cancellation in a given year is less than 0.8 percent.

There are some concerning cases in the NAS database too, in which professors were fired for actions that should be covered under a basic principle of academic freedom — for example, reading aloud a Mark Twain passage that included a racial slur, even after giving students advance notice — so this isn’t a total non-issue. But the number of low stakes and relatively unobjectionable cases means the risk is lower than 0.8 percent (and it’s even lower than that, since NAS includes Canada and my denominator is ranked schools in the United States).

Similarly, FIRE classifies about 30 percent of the attempted disinvitations in its database as from the right. About 60 percent are from the left — the other 10 percent N/A — so if you want to argue that the left does this more, you’ve got some evidence. But still, the number of cases from the left is lower than the total. And more than half of FIRE’s attempted disinvitations did not result in anyone getting disinvited.

Using U.S. News’ ranked schools as the denominator, the chance of left-wing protestors trying to get a speaker disinvited at a college or university in a given year is about 0.5 percent. The chance of an actual disinvitation is less than 0.25 percent. And that’s in the entire school. To put this in perspective, my political science department alone hosts speakers most weeks of the semester.

Two things jump out here:

Bari Weiss and Anne Applebaum both cite a Cato study purporting to show this effect:

even if we assume these databases capture a fraction of actual instances — which would be surprising, given the media attention on this topic, but even so — the data does not show an illiberal left-wing movement in control of academia.

The number agreeing that the political climate prevents them from saying things they believe ranges from 42% to 77%, which is high across political views. That suggests self-censorship is, to a significant degree, a factor of the political, cultural, and technological environment, rather than caused by any particular ideology.

Conservatives report self-censoring more than liberals do.

The same study shows that the biggest increase in self-censorship from 2017 to 2020 was among strong liberals (+12), while strong conservatives increased the least (+1).

If this data told a story of ascendent Maoists suppressing conservative speech, it would probably be the opposite, with the left becoming more confident of expressing their views — on race, gender, etc. — while the right becomes disproportionately more fearful. Culture warriors fixate on wokeness, but when asked about the political climate, many Americans likely thought about Trumpism

Nevertheless, this data does show conservatives are more likely to say the political climate prevents them from expressing their beliefs. But what it doesn’t show is which beliefs or why.

Self-censoring can be a problem, but also not. The adage “do not discuss politics or religion in general company” goes back to at least 1879. If someone today is too scared to say “Robin DiAngelo’s conception of ‘white fragility’ does not stand up to logical scrutiny,” that’s bad. If they’re too scared to shout racial slurs at minorities, that isn’t. A lot depends on the content of the speech.

When I was a teenager in the 1990s, anti-gay slurs were common insults among boys, and tough-guy talk in movies. Now it’s a lot less common, one of the things pushed out of polite society, like the n-word, Holocaust denial, and sexual harassment. I think that’s a positive.

Another problem with the anti-woke interpretation of the Cato study is media constantly tells conservatives they’re under dire threat.

Fox News, including Tucker Carlson (the most-watched show on basic cable), Ben Shapiro and Dan Bongino (frequently among the most-shared on Facebook), and other right-wing outlets devote tons of coverage to cancel culture, riling up conservatives with hyperbolic claims that people are coming for them

Anti-woke liberals in prestigious mainstream outlets tell them it’s the Cultural Revolution

Then a survey asks if the political climate prevents them from saying what they believe, and, primed by media, they say yes.

With so many writers on the anti-woke beat, it’s not especially plausible that we’re missing many cases of transgender servers getting people canceled for using the wrong pronoun in coffee shops to the point that everyone who isn’t fully comfortable with the terminology should live in fear. By overstating the threat of cancellation and the power of woke activists, anti-woke liberals are chilling speech they aim to protect.

a requirement to both-sides the Holocaust is a plausible read of the legal text. It’s an unsurprising result of empowering the state to suppress ideas in an environment with bad faith culture warriors, such as Chris Rufo and James Lindsay, advocating state censorship and deliberately stoking panic to get it.

Texas, Florida, and other states trying to suppress unwanted ideas in both K-12 and higher ed isn’t the Cultural Revolution either — no state-sanctioned mass violence here — but it’s coming from government, making it a bigger threat to speech and academic freedom.

To put this in perspective, antiracist guru Ibram X. Kendi has called for an “anti-racist Constitutional amendment,” which would “make unconstitutional racial inequity over a certain threshold, as well as racist ideas by public officials,” and establish a Department of Anti-Racism to enforce it. It’s a terrible proposal that would repeal the First Amendment and get the state heavily involved in policing speech (which, even if well-intentioned, comes with serious risks of abuse).

It also doesn’t stand the slightest chance of happening.

It’s fair to characterize this article as anti-anti-woke. And I usually don’t like anti-anti- arguments, especially anti-anti-Trump (because it’s effectively pro). But in this case I’m doing it because I reject the binary.

American politics is often binary.

Culture is not. It’s an ever-changing mishmash, with a large variety of influential participants

There have been unmistakable changes in American culture — Western culture, really — regarding race and gender, but there are way more than two sides to that. You don’t have to be woke or anti-woke. It’s not a political campaign or a war. You can think all sorts of things, mixing and matching from these ideas and others.

I won’t say “this is trivial” nor “this stuff is great,” because I don’t think either. At least not if “this” means uncompromising Maoists seeking domination.

I think that’s bad, but it’s not especially common. It’s not fiction — I’m online a lot, I have feet in both media and academia, I’ve seen it too — but, importantly, it’s not in control

I think government censorship is inherently more concerning than private censorship, and that we can’t sufficiently counter the push for state idea-suppression without countering the overstated fears that rationalize it.

I think a lot of the private censorship problem can be addressed by executives and administrators — the ones who actually have power over businesses and universities — showing a bit of spine. Don’t fold at the first sign of protest. Take some time to look into it yourself, and make a judgment call on whether discipline is merited and necessary. Often, the activist mob will move on in a few days anyway.

I think that, with so much of the conversation focusing on extremes, people often miss when administrators do this.

I think violence is physical, and that while speech can be quite harmful, it’s better to think of these two things as categorically different than to insist harmful speech is literally violence.

at a baseline, treating people as equals means respecting who they say they are. The vast majority are not edge cases like a competitive athlete, but regular people trying to live their lives. Let them use the bathroom in peace.

I think the argument that racism and other forms of bigotry operate at a systemic or institutional, in addition to individual, level is insightful, intuitive, and empirically supported. We can improve people’s lives by taking that into account when crafting laws, policies, and practices.

I think identity and societal structures shape people’s lives (whether they want it to or not) but they’re far from the only factors. Treating them as the only, or even predominant, factor essentializes more than it empowers.

I think transgender and non-binary people have a convincing case for equality. I don’t think that points to clear answers on every question—what’s the point of gender segregated sports?

I think free association is an essential value too. Which inherently includes the right of disassociation.

I think these situations often fall into a gray area, and businesses should be able to make their own judgment calls about personnel, since companies have a reasonable interest in protecting their brand.

I think free speech is an essential value, not just at the legal level, but culturally as well. I think people who would scrap it, from crusading antiracists to social conservatives pining for Viktor Orban’s Hungary, have a naively utopian sense of how that would go (both in general and for them specifically). Getting the state involved in speech suppression is a bad idea.

I think America’s founding was a big step forward for government and individual liberty, and early America was a deeply racist, bigoted place that needed Amendments (13-15; 19), Civil Rights Acts, and landmark court cases to become a liberal democracy. I don’t think it’s hard to hold both of those in your head at the same time.

I think students learning the unvarnished truth about America’s racist past is good, and that teaching students they are personally responsible for the sins of the past is not.

I think synthesis of these cultural forces is both desirable and possible. Way more people think both that bigotry is bad and individual freedom is good than online arguments lead you to believe.

I don’t think the sides are as far apart as they think.

I think we should disaggregate cancel culture and left-wing identity politics. Cancellation should be understood as an internet phenomenon.

If it ever was just something the left does, it isn’t anymore.

I think a lot of us could agree that social media mobbing and professional media attention on minor incidents is wrong, especially as part of a campaign to get someone fired. In general, disproportionally severe social and professional sanctions is a problem, no matter the alleged cause.

I think most anti-woke liberals really do want to defend free speech and academic freedom. But I don’t think their panic-stoking hyperbole is helping.

Sunday’s incident was the 883rd fatal shooting by a law enforcement officer so far in 2015, according to the Guardian’s records. Another 47 people died after being shocked with an officer’s Taser, 33 died after being struck by a law enforcement officer’s vehicle, and 36 were killed in custody. Another received a deadly blow to the head during a fight with an officer.

The shooting was also the 183rd death recorded in California, by far the greatest total of any state. Nine states, however, have recorded more deaths per capita, with Oklahoma having the highest rate.

The US government publishes no comprehensive record of people killed by law enforcement, even after a series of controversial deaths unleashed a national protest movement and demands from activists and lawmakers alike for better data on the subject.

An analysis of the statistics collected so far found the rate of deaths currently stands at 3.1 per day. This rate has remained relatively steady throughout the year, peaking through the month of March to a daily rate of almost four and dipping to an average of 2.6 through June.

On Monday, the platform went dark after a hacker took over the accounts of 178 users, including Torba and the Republican congresswoman Marjorie Taylor Greene.

Gab, a Twitter-like website promoted by Torba as a bastion of free speech, has long been a forum of last resort for extremists and conspiracy theorists who have been banned on other online platforms. It attained worldwide notoriety in 2018 when a user, Robert Bowers, wrote on the site that he was “going in”, shortly before allegedly entering the Tree of Life synagogue in Pittsburgh, Pennsylvania, and killing eleven people.

The leaked files contained what appears to be a database of over 4.1 million registered users on the site and tags identifying subscribers as “investors”, “verified” users and “pro” users.

The 2017 share offering, for example, required a minimum investment of $199.10, and rewarded investors who contributed a greater amount with “perks”. Users who invested $200 could display a “Gab investor badge” on the site. The badges corresponded with a tag in the database, which allowed investors to be looked at in detail.

Some of the people associated with investors’ accounts had high-profile jobs and public roles, while spewing hate and extremist beliefs online.

The data breach also appears to offer some insight into users tagged as “verified” by Gab, which according to the platform’s own explanation means that they have completed a verification process that includes matching their display name to a government ID.

And it appears to include a list of users registered as “pros”, which allows users to access additional features and a badge at a price starting at $99 year. The database indicates over 18,000 users had paid to be pro users at the time of the breach. Nearly 4,000 users were flagged as donors to Gab’s repeated attempts to attract voluntary gifts from users.

Direct messages included in the leak appear to show close communication between Torba and a major QAnon influencer who is labeled a Gab investor, seemingly reinforcing the CEO’s public efforts to make Gab a home for adherents to the QAnon conspiracy theory, which helped fuel the 6 January attack on the nation’s Capitol.

According to Wired, the data exposed in the apparent hack was sourced by a hacker who had found a security vulnerability in the site.

“Gab was negligent at best and malicious at worst” in its approach to security, she added. “It is hard to envision a scenario where a company cared less about user data than this one.”

Two sources said that during the early weeks of the war they were permitted to kill 15 or 20 civilians during airstrikes on low-ranking militants. Attacks on such targets were typically carried out using unguided munitions known as “dumb bombs”, the sources said, destroying entire homes and killing all their occupants.

“You don’t want to waste expensive bombs on unimportant people – it’s very expensive for the country and there’s a shortage [of those bombs],” one intelligence officer said. Another said the principal question they were faced with was whether the “collateral damage” to civilians allowed for an attack.

“Because we usually carried out the attacks with dumb bombs, and that meant literally dropping the whole house on its occupants. But even if an attack is averted, you don’t care – you immediately move on to the next target. Because of the system, the targets never end. You have another 36,000 waiting.”

ccording to conflict experts, if Israel has been using dumb bombs to flatten the homes of thousands of Palestinians who were linked, with the assistance of AI, to militant groups in Gaza, that could help explain the shockingly high death toll in the war.

Details about the specific kinds of data used to train Lavender’s algorithm, or how the programme reached its conclusions, are not included in the accounts published by +972 or Local Call. However, the sources said that during the first few weeks of the war, Unit 8200 refined Lavender’s algorithm and tweaked its search parameters.

Responding to the publication of the testimonies in +972 and Local Call, the IDF said in a statement that its operations were carried out in accordance with the rules of proportionality under international law. It said dumb bombs are “standard weaponry” that are used by IDF pilots in a manner that ensures “a high level of precision”.

“The IDF does not use an artificial intelligence system that identifies terrorist operatives or tries to predict whether a person is a terrorist,” it added. “Information systems are merely tools for analysts in the target identification process.”

In earlier military operations conducted by the IDF, producing human targets was often a more labour-intensive process. Multiple sources who described target development in previous wars to the Guardian, said the decision to “incriminate” an individual, or identify them as a legitimate target, would be discussed and then signed off by a legal adviser.

n the weeks and months after 7 October, this model for approving strikes on human targets was dramatically accelerated, according to the sources. As the IDF’s bombardment of Gaza intensified, they said, commanders demanded a continuous pipeline of targets.

“We were constantly being pressured: ‘Bring us more targets.’ They really shouted at us,” said one intelligence officer. “We were told: now we have to fuck up Hamas, no matter what the cost. Whatever you can, you bomb.”

Lavender was developed by the Israel Defense Forces’ elite intelligence division, Unit 8200, which is comparable to the US’s National Security Agency or GCHQ in the UK.

After randomly sampling and cross-checking its predictions, the unit concluded Lavender had achieved a 90% accuracy rate, the sources said, leading the IDF to approve its sweeping use as a target recommendation tool.

Lavender created a database of tens of thousands of individuals who were marked as predominantly low-ranking members of Hamas’s military wing, they added. This was used alongside another AI-based decision support system, called the Gospel, which recommended buildings and structures as targets rather than individuals.

The accounts include first-hand testimony of how intelligence officers worked with Lavender and how the reach of its dragnet could be adjusted. “At its peak, the system managed to generate 37,000 people as potential human targets,” one of the sources said. “But the numbers changed all the time, because it depends on where you set the bar of what a Hamas operative is.”

broadly, and then the machine started bringing us all kinds of civil defence personnel, police officers, on whom it would be a shame to waste bombs. They help the Hamas government, but they don’t really endanger soldiers.”

Before the war, US and Israeli estimated membership of Hamas’s military wing at approximately 25-30,000 people.

there was a decision to treat Palestinian men linked to Hamas’s military wing as potential targets, regardless of their rank or importance.

According to +972 and Local Call, the IDF judged it permissible to kill more than 100 civilians in attacks on a top-ranking Hamas officials. “We had a calculation for how many [civilians could be killed] for the brigade commander, how many [civilians] for a battalion commander, and so on,” one source said.

Another source, who justified the use of Lavender to help identify low-ranking targets, said that “when it comes to a junior militant, you don’t want to invest manpower and time in it”. They said that in wartime there was insufficient time to carefully “incriminate every target”

So you’re willing to take the margin of error of using artificial intelligence, risking collateral damage and civilians dying, and risking attacking by mistake, and to live with it,” they added.

When it came to targeting low-ranking Hamas and PIJ suspects, they said, the preference was to attack when they were believed to be at home. “We were not interested in killing [Hamas] operatives only when they were in a military building or engaged in a military activity,” one said. “It’s much easier to bomb a family’s home. The system is built to look for them in these situations.”

Such a strategy risked higher numbers of civilian casualties, and the sources said the IDF imposed pre-authorised limits on the number of civilians it deemed acceptable to kill in a strike aimed at a single Hamas militant. The ratio was said to have changed over time, and varied according to the seniority of the target.

The IDF’s targeting processes in the most intensive phase of the bombardment were also relaxed, they said. “There was a completely permissive policy regarding the casualties of [bombing] operations,” one source said. “A policy so permissive that in my opinion it had an element of revenge.”

“There were regulations, but they were just very lenient,” another added. “We’ve killed people with collateral damage in the high double digits, if not low triple digits. These are things that haven’t happened before.” There appears to have been significant fluctuations in the figure that military commanders would tolerate at different stages of the war

One source said that the limit on permitted civilian casualties “went up and down” over time, and at one point was as low as five. During the first week of the conflict, the source said, permission was given to kill 15 non-combatants to take out junior militants in Gaza

at one stage earlier in the war they were authorised to kill up to “20 uninvolved civilians” for a single operative, regardless of their rank, military importance, or age.

“It’s not just that you can kill any person who is a Hamas soldier, which is clearly permitted and legitimate in terms of international law,” they said. “But they directly tell you: ‘You are allowed to kill them along with many civilians.’ … In practice, the proportionality criterion did not exist.”

Experts in international humanitarian law who spoke to the Guardian expressed alarm at accounts of the IDF accepting and pre-authorising collateral damage ratios as high as 20 civilians, particularly for lower-ranking militants. They said militaries must assess proportionality for each individual strike.

An international law expert at the US state department said they had “never remotely heard of a one to 15 ratio being deemed acceptable, especially for lower-level combatants. There’s a lot of leeway, but that strikes me as extreme”.

Sarah Harrison, a former lawyer at the US Department of Defense, now an analyst at Crisis Group, said: “While there may be certain occasions where 15 collateral civilian deaths could be proportionate, there are other times where it definitely wouldn’t be. You can’t just set a tolerable number for a category of targets and say that it’ll be lawfully proportionate in each case.”

Whatever the legal or moral justification for Israel’s bombing strategy, some of its intelligence officers appear now to be questioning the approach set by their commanders. “No one thought about what to do afterward, when the war is over, or how it will be possible to live in Gaza,” one said.

Another said that after the 7 October attacks by Hamas, the atmosphere in the IDF was “painful and vindictive”. “There was a dissonance: on the one hand, people here were frustrated that we were not attacking enough. On the other hand, you see at the end of the day that another thousand Gazans have died, most of them civilians.”

Dr. Shi’s group was fascinated by how coronaviruses jump from species to species. To find viruses, they took samples from bats and other animals, as well as from sick people living near animals carrying these viruses or associated with the wildlife trade. Much of this work was conducted in partnership with the EcoHealth Alliance, a U.S.-based scientific organization that, since 2002, has been awarded over $80 million in federal funding to research the risks of emerging infectious diseases.

Their research showed that the viruses most similar to SARS‑CoV‑2, the virus that caused the pandemic, circulate in bats that live roughly 1,000 miles away from Wuhan. Scientists from Dr. Shi’s team traveled repeatedly to Yunnan province to collect these viruses and had expanded their search to Southeast Asia. Bats in other parts of China have not been found to carry viruses that are as closely related to SARS-CoV-2.

When the Covid-19 outbreak was detected, Dr. Shi initially wondered if the novel coronavirus had come from her laboratory, saying she had never expected such an outbreak to occur in Wuhan.

The SARS‑CoV‑2 virus is exceptionally contagious and can jump from species to species like wildfire. Yet it left no known trace of infection at its source or anywhere along what would have been a thousand-mile journey before emerging in Wuhan.

The year before the outbreak, the Wuhan institute, working with U.S. partners, had proposed creating viruses with SARS‑CoV‑2’s defining feature

The laboratory pursued risky research that resulted in viruses becoming more infectious: Coronaviruses were grown from samples from infected animals and genetically reconstructed and recombined to create new viruses unknown in nature. These new viruses were passed through cells from bats, pigs, primates and humans and were used to infect civets and humanized mice (mice modified with human genes). In essence, this process forced these viruses to adapt to new host species, and the viruses with mutations that allowed them to thrive emerged as victors.

Worse still, as the pandemic raged, their American collaborators failed to publicly reveal the existence of the Defuse proposal. The president of EcoHealth, Peter Daszak, recently admitted to Congress that he doesn’t know about virus samples collected by the Wuhan institute after 2015 and never asked the lab’s scientists if they had started the work described in Defuse.

By 2019, Dr. Shi’s group had published a database describing more than 22,000 collected wildlife samples. But external access was shut off in the fall of 2019, and the database was not shared with American collaborators even after the pandemic started, when such a rich virus collection would have been most useful in tracking the origin of SARS‑CoV‑2. It remains unclear whether the Wuhan institute possessed a precursor of the pandemic virus.

In 2021, The Intercept published a leaked 2018 grant proposal for a research project named Defuse, which had been written as a collaboration between EcoHealth, the Wuhan institute and Ralph Baric at the University of North Carolina, who had been on the cutting edge of coronavirus research for years. The proposal described plans to create viruses strikingly similar to SARS‑CoV‑2.

Coronaviruses bear their name because their surface is studded with protein spikes, like a spiky crown, which they use to enter animal cells. The Defuse project proposed to search for and create SARS-like viruses carrying spikes with a unique feature: a furin cleavage site — the same feature that enhances SARS‑CoV‑2’s infectiousness in humans, making it capable of causing a pandemic. Defuse was never funded by the United States.

owever, in his testimony on Monday, Dr. Fauci explained that the Wuhan institute would not need to rely on U.S. funding to pursue research independently.

While it’s possible that the furin cleavage site could have evolved naturally (as seen in some distantly related coronaviruses), out of the hundreds of SARS-like viruses cataloged by scientists, SARS‑CoV‑2 is the only one known to possess a furin cleavage site in its spike. And the genetic data suggest that the virus had only recently gained the furin cleavage site before it started the pandemic.

Ultimately, a never-before-seen SARS-like virus with a newly introduced furin cleavage site, matching the description in the Wuhan institute’s Defuse proposal, caused an outbreak in Wuhan less than two years after the proposal was drafted.

When the Wuhan scientists published their seminal paper about Covid-19 as the pandemic roared to life in 2020, they did not mention the virus’s furin cleavage site — a feature they should have been on the lookout for, according to their own grant proposal, and a feature quickly recognized by other scientists.

At the Wuhan Institute of Virology, a team of scientists had been hunting for SARS-like viruses for over a decade, led by Shi Zhengl

In May, citing failures in EcoHealth’s monitoring of risky experiments conducted at the Wuhan lab, the Biden administration suspended all federal funding for the organization and Dr. Daszak, and initiated proceedings to bar them from receiving future grants. In his testimony on Monday, Dr. Fauci said that he supported the decision to suspend and bar EcoHealth.

Separately, Dr. Baric described the competitive dynamic between his research group and the institute when he told Congress that the Wuhan scientists would probably not have shared their most interesting newly discovered viruses with him. Documents and email correspondence between the institute and Dr. Baric are still being withheld from the public while their release is fiercely contested in litigation.

In the end, American partners very likely knew of only a fraction of the research done in Wuhan. According to U.S. intelligence sources, some of the institute’s virus research was classified or conducted with or on behalf of the Chinese military.

In the congressional hearing on Monday, Dr. Fauci repeatedly acknowledged the lack of visibility into experiments conducted at the Wuhan institute, saying, “None of us can know everything that’s going on in China, or in Wuhan, or what have you. And that’s the reason why — I say today, and I’ve said at the T.I.,” referring to his transcribed interview with the subcommittee, “I keep an open mind as to what the origin is.”

The Wuhan lab pursued this type of work under low biosafety conditions that could not have contained an airborne virus as infectious as SARS‑CoV‑2.

Labs working with live viruses generally operate at one of four biosafety levels (known in ascending order of stringency as BSL-1, 2, 3 and 4) that describe the work practices that are considered sufficiently safe depending on the characteristics of each pathogen. The Wuhan institute’s scientists worked with SARS-like viruses under inappropriately low biosafety conditions.

​​Biosafety levels are not internationally standardized, and some countries use more permissive protocols than others.

In one experiment, Dr. Shi’s group genetically engineered an unexpectedly deadly SARS-like virus (not closely related to SARS‑CoV‑2) that exhibited a 10,000-fold increase in the quantity of virus in the lungs and brains of humanized mice. Wuhan institute scientists handled these live viruses at low biosafety levels, including BSL-2.

Even the much more stringent containment at BSL-3 cannot fully prevent SARS‑CoV‑2 from escaping. Two years into the pandemic, the virus infected a scientist in a BSL-3 laboratory in Taiwan, which was, at the time, a zero-Covid country. The scientist had been vaccinated and was tested only after losing the sense of smell. By then, more than 100 close contacts had been exposed. Human error is a source of exposure even at the highest biosafety levels, and the risks are much greater for scientists working with infectious pathogens at low biosafety.

An early draft of the Defuse proposal stated that the Wuhan lab would do their virus work at BSL-2 to make it “highly cost-effective.” Dr. Baric added a note to the draft highlighting the importance of using BSL-3 to contain SARS-like viruses that could infect human cells, writing that “U.S. researchers will likely freak out.”

Years later, after SARS‑CoV‑2 had killed millions, Dr. Baric wrote to Dr. Daszak: “I have no doubt that they followed state determined rules and did the work under BSL-2. Yes China has the right to set their own policy. You believe this was appropriate containment if you want but don’t expect me to believe it. Moreover, don’t insult my intelligence by trying to feed me this load of BS.”

SARS‑CoV‑2 is a stealthy virus that transmits effectively through the air, causes a range of symptoms similar to those of other common respiratory diseases and can be spread by infected people before symptoms even appear. If the virus had escaped from a BSL-2 laboratory in 2019, the leak most likely would have gone undetected until too late.

One alarming detail — leaked to The Wall Street Journal and confirmed by current and former U.S. government officials — is that scientists on Dr. Shi’s team fell ill with Covid-like symptoms in the fall of 2019. One of the scientists had been named in the Defuse proposal as the person in charge of virus discovery work. The scientists denied having been sick.

The hypothesis that Covid-19 came from an animal at the Huanan Seafood Market in Wuhan is not supported by strong evidence.

In December 2019, Chinese investigators assumed the outbreak had started at a centrally located market frequented by thousands of visitors daily. This bias in their search for early cases meant that cases unlinked to or located far away from the market would very likely have been missed

To make things worse, the Chinese authorities blocked the reporting of early cases not linked to the market and, claiming biosafety precautions, ordered the destruction of patient samples on January 3, 2020, making it nearly impossible to see the complete picture of the earliest Covid-19 cases. Information about dozens of early cases from November and December 2019 remains inaccessible.

A pair of papers published in Science in 2022 made the best case for SARS‑CoV‑2 having emerged naturally from human-animal contact at the Wuhan market by focusing on a map of the early cases and asserting that the virus had jumped from animals into humans twice at the market in 2019

More recently, the two papers have been countered by other virologists and scientists who convincingly demonstrate that the available market evidence does not distinguish between a human superspreader event and a natural spillover at the market.

Furthermore, the existing genetic and early case data show that all known Covid-19 cases probably stem from a single introduction of SARS‑CoV‑2 into people, and the outbreak at the Wuhan market probably happened after the virus had already been circulating in humans.

Not a single infected animal has ever been confirmed at the market or in its supply chain. Without good evidence that the pandemic started at the Huanan Seafood Market, the fact that the virus emerged in Wuhan points squarely at its unique SARS-like virus laboratory.

With today’s technology, scientists can detect how respiratory viruses — including SARS, MERS and the flu — circulate in animals while making repeated attempts to jump across species. Thankfully, these variants usually fail to transmit well after crossing over to a new species and tend to die off after a small number of infections

investigators have not reported finding any animals infected with SARS‑CoV‑2 that had not been infected by humans. Yet, infected animal sources and other connective pieces of evidence were found for the earlier SARS and MERS outbreaks as quickly as within a few days, despite the less advanced viral forensic technologies of two decades ago.

Even though Wuhan is the home base of virus hunters with world-leading expertise in tracking novel SARS-like viruses, investigators have either failed to collect or report key evidence that would be expected if Covid-19 emerged from the wildlife trade. For example, investigators have not determined that the earliest known cases had exposure to intermediate host animals before falling ill.

No antibody evidence shows that animal traders in Wuhan are regularly exposed to SARS-like viruses, as would be expected in such situations.

In previous outbreaks of coronaviruses, scientists were able to demonstrate natural origin by collecting multiple pieces of evidence linking infected humans to infected animals

In contrast, virologists and other scientists agree that SARS‑CoV‑2 required little to no adaptation to spread rapidly in humans and other animals. The virus appears to have succeeded in causing a pandemic upon its only detected jump into humans.

it was a SARS-like coronavirus with a unique furin cleavage site that emerged in Wuhan, less than two years after scientists, sometimes working under inadequate biosafety conditions, proposed collecting and creating viruses of that same design.

a laboratory accident is the most parsimonious explanation of how the pandemic began.

Given what we now know, investigators should follow their strongest leads and subpoena all exchanges between the Wuhan scientists and their international partners, including unpublished research proposals, manuscripts, data and commercial orders. In particular, exchanges from 2018 and 2019 — the critical two years before the emergence of Covid-19 — are very likely to be illuminating (and require no cooperation from the Chinese government to acquire), yet they remain beyond the public’s view more than four years after the pandemic began.

it is undeniable that U.S. federal funding helped to build an unprecedented collection of SARS-like viruses at the Wuhan institute, as well as contributing to research that enhanced them.

Advocates and funders of the institute’s research, including Dr. Fauci, should cooperate with the investigation to help identify and close the loopholes that allowed such dangerous work to occur. The world must not continue to bear the intolerable risks of research with the potential to cause pandemics.

A successful investigation of the pandemic’s root cause would have the power to break a decades-long scientific impasse on pathogen research safety, determining how governments will spend billions of dollars to prevent future pandemics. A credible investigation would also deter future acts of negligence and deceit by demonstrating that it is indeed possible to be held accountable for causing a viral pandemic

Last but not least, people of all nations need to see their leaders — and especially, their scientists — heading the charge to find out what caused this world-shaking event. Restoring public trust in science and government leadership requires it.

Klein adds the third element to our story, which he calls “governmentalization.” Words having to do with experts have shown a steady rise.

On the subject of individualization, he found that the word “preferences” was barely used until about 1930, but usage has surged since. On the general subject of demoralization, he finds a long decline of usage in terms like “faith,” “wisdom,” “ought,” “evil” and “prudence,” and a sharp rise in what you might call social science terms like “subjectivity,” “normative,” “psychology” and “information.”

The second element of the story is demoralization. A study by Pelin Kesebir and Selin Kesebir found that general moral terms like “virtue,” “decency” and “conscience” were used less frequently over the course of the 20th century. Words associated with moral excellence, like “honesty,” “patience” and “compassion” were used much less frequently.

Over the past half-century, society has become more individualistic. As it has become more individualistic, it has also become less morally aware, because social and moral fabrics are inextricably linked. The atomization and demoralization of society have led to certain forms of social breakdown, which government has tried to address, sometimes successfully and often impotently.

This story, if true, should cause discomfort on right and left.

Conservatives sometimes argue that if we could just reduce government to the size it was back in, say, the 1950s, then America would be vibrant and free again. But the underlying sociology and moral culture is just not there anymore. Government could be smaller when the social fabric was more tightly knit, but small government will have different and more cataclysmic effects today when it is not.

Liberals sometimes argue that our main problems come from the top: a self-dealing elite, the oligarchic bankers. But the evidence suggests that individualism and demoralization are pervasive up and down society, and may be even more pervasive at the bottom. Liberals also sometimes talk as if our problems are fundamentally economic, and can be addressed politically, through redistribution. But maybe the root of the problem is also cultural. The social and moral trends swamp the proposed redistributive remedies.

these gradual shifts in language reflect tectonic shifts in culture. We write less about community bonds and obligations because they’re less central to our lives.

The way the system is designed, even had Amri fully cooperated, however, the Germans would not have had access to his criminal record in Italy. The computer databases used in Europe to vet migrants in the first instance does not include such data.

European law enforcement officials say that sharing information across borders has sharply increased in the 13 months since the Bataclan attack in Paris. But just because information is in a database does not always mean that it gets used

But I still care deeply for the liberty of my conservative students. For this reason I have some advice for those in college who are concerned about their freedom of speech.

My Conservatives Club colleagues and I received national attention. Some were featured on the cover of The New York Times Magazine, and others got booked on popular TV shows like “The O’Reilly Factor.” In short, for an oppressed minority group whose speech was stifled, we had an awful lot of press.

My fellow conservative students and I half-ironically borrowed the language of the multiculturalist left and applied it to ourselves.

Most of us conservatives didn’t suffer from similar injustices, but we saw ourselves nevertheless as victims of ideological oppression.

It seemed that everyone was celebrating diversity and multiculturalism, and I didn’t see a role for myself in that. It occurred to me, as it has to countless other conservative students, that I might also be a kind of minority — an “ideological minority” — because of my conservative political views.

What I’m getting at is that I was never a victim. My message to conservative students is that neither are you.

false equivalence is not helping you prepare for the wider world.

You know the world doesn’t love a victim. Don’t adopt a posture you disagree with just because it plays well in conservative media.

You have a voice and ideas that people need to hear, but don’t compare disagreement with your ideas to suppression.

Many conservative students denounced recent protests against the college tour of the right-wing provocateur Milo Yiannopoulos as an attack on free speech. In all likelihood, those were protests against Mr. Yiannopoulos’s bullying and ridicule.

A “conservative campus speaker” is not someone who belittles left-wing students, but whose visit is for teaching and learning.

It is important you invite speakers to campus. We certainly did. But seek out the highly credentialed thinkers, like Walter Williams, Victor Davis Hanson or Yuval Levin.

The Foundation for Individual Rights in Education keeps a handy database of nationwide attempts to withdraw invitations to speakers, and 2016 was a record year. But almost a third of those attempts were directed at Mr. Yiannopoulos alone. That should be food for thought

Until you stop settling for what’s surely the satisfying release of Mr. Yiannopoulos’s deliberate and superficial antagonism, you can’t know whether it’s the actual conservatism that your fellow students have rejected.

If you take responsibility for the parts of your education you control, and focus your energy on learning, writing, speaking and debating (instead of shock-value pranks, appeals to victimhood or dismissal of academia at large), you will grow and succeed

Read Edmund Burke, Matthew Arnold, Russell Kirk, Thomas Sowell, Michael Oakeshott and Peggy Noonan. Acknowledge arguments you disagree with on their own terms, and respond to their substance.