Group items tagged

They do this by sucking up and analyzing your data, using it to predict what will cause you to react most strongly, and then giving you more of that.

Algorithms that maximize attention give an advantage to negative messages. People tend to react more to inputs that land low on the brainstem. Fear and anger produce a lot more engagement and sharing than joy

The result is that the algorithms favor sensational content over substance.

for mass media, this was constrained by one-size-fits-all content and by the limitations of delivery platforms. Not so for internet platforms on smartphones. They have created billions of individual channels, each of which can be pushed further into negativity and extremism without the risk of alienating other audience members

On Facebook, it’s your news feed, while on Google it’s your individually customized search results. The result is that everyone sees a different version of the internet tailored to create the illusion that everyone else agrees with them.

It took Brexit for me to begin to see the danger of this dynamic. I’m no expert on British politics, but it seemed likely that Facebook might have had a big impact on the vote because one side’s message was perfect for the algorithms and the other’s wasn’t. The “Leave” campaign made an absurd promise—there would be savings from leaving the European Union that would fund a big improvement in the National Health System—while also exploiting xenophobia by casting Brexit as the best way to protect English culture and jobs from immigrants. It was too-good-to-be-true nonsense mixed with fearmongering.

Facebook was a much cheaper and more effective platform for Leave in terms of cost per user reached. And filter bubbles would ensure that people on the Leave side would rarely have their questionable beliefs challenged. Facebook’s model may have had the power to reshape an entire continent.

Tristan Harris, formerly the design ethicist at Google. Tristan had just appeared on 60 Minutes to discuss the public health threat from social networks like Facebook. An expert in persuasive technology, he described the techniques that tech platforms use to create addiction and the ways they exploit that addiction to increase profits. He called it “brain hacking.”

The most important tool used by Facebook and Google to hold user attention is filter bubbles. The use of algorithms to give consumers “what they want” leads to an unending stream of posts that confirm each user’s existing beliefs

Continuous reinforcement of existing beliefs tends to entrench those beliefs more deeply, while also making them more extreme and resistant to contrary facts

No one stopped them from siphoning off the profits of content creators. No one stopped them from gathering data on every aspect of every user’s internet life. No one stopped them from amassing market share not seen since the days of Standard Oil.

Facebook takes the concept one step further with its “groups” feature, which encourages like-minded users to congregate around shared interests or beliefs. While this ostensibly provides a benefit to users, the larger benefit goes to advertisers, who can target audiences even more effectively.

We theorized that the Russians had identified a set of users susceptible to its message, used Facebook’s advertising tools to identify users with similar profiles, and used ads to persuade those people to join groups dedicated to controversial issues. Facebook’s algorithms would have favored Trump’s crude message and the anti-Clinton conspiracy theories that thrilled his supporters, with the likely consequence that Trump and his backers paid less than Clinton for Facebook advertising per person reached.

The ads were less important, though, than what came next: once users were in groups, the Russians could have used fake American troll accounts and computerized “bots” to share incendiary messages and organize events.

Trolls and bots impersonating Americans would have created the illusion of greater support for radical ideas than actually existed.

Real users “like” posts shared by trolls and bots and share them on their own news feeds, so that small investments in advertising and memes posted to Facebook groups would reach tens of millions of people.

A similar strategy prevailed on other platforms, including Twitter. Both techniques, bots and trolls, take time and money to develop—but the payoff would have been huge.

2016 was just the beginning. Without immediate and aggressive action from Washington, bad actors of all kinds would be able to use Facebook and other platforms to manipulate the American electorate in future elections.

Renee DiResta, an expert in how conspiracy theories spread on the internet. Renee described how bad actors plant a rumor on sites like 4chan and Reddit, leverage the disenchanted people on those sites to create buzz, build phony news sites with “press” versions of the rumor, push the story onto Twitter to attract the real media, then blow up the story for the masses on Facebook.

It was sophisticated hacker technique, but not expensive. We hypothesized that the Russians were able to manipulate tens of millions of American voters for a sum less than it would take to buy an F-35 fighter jet.

Algorithms can be beautiful in mathematical terms, but they are only as good as the people who create them. In the case of Facebook and Google, the algorithms have flaws that are increasingly obvious and dangerous.

Thanks to the U.S. government’s laissez-faire approach to regulation, the internet platforms were able to pursue business strategies that would not have been allowed in prior decades. No one stopped them from using free products to centralize the internet and then replace its core functions.

To the contrary: the platforms help people self-segregate into like-minded filter bubbles, reducing the risk of exposure to challenging ideas.

No one stopped them from running massive social and psychological experiments on their users. No one demanded that they police their platforms. It has been a sweet deal.

Facebook and Google are now so large that traditional tools of regulation may no longer be effective.

The largest antitrust fine in EU history bounced off Google like a spitball off a battleship.

It reads like the plot of a sci-fi novel: a technology celebrated for bringing people together is exploited by a hostile power to drive people apart, undermine democracy, and create misery. This is precisely what happened in the United States during the 2016 election.

We had constructed a modern Maginot Line—half the world’s defense spending and cyber-hardened financial centers, all built to ward off attacks from abroad—never imagining that an enemy could infect the minds of our citizens through inventions of our own making, at minimal cost

Not only was the attack an overwhelming success, but it was also a persistent one, as the political party that benefited refuses to acknowledge reality. The attacks continue every day, posing an existential threat to our democratic processes and independence.

Facebook, Google, Twitter, and other platforms were manipulated by the Russians to shift outcomes in Brexit and the U.S. presidential election, and unless major changes are made, they will be manipulated again. Next time, there is no telling who the manipulators will be.

Unfortunately, there is no regulatory silver bullet. The scope of the problem requires a multi-pronged approach.

Polls suggest that about a third of Americans believe that Russian interference is fake news, despite unanimous agreement to the contrary by the country’s intelligence agencies. Helping those people accept the truth is a priority. I recommend that Facebook, Google, Twitter, and others be required to contact each person touched by Russian content with a personal message that says, “You, and we, were manipulated by the Russians. This really happened, and here is the evidence.” The message would include every Russian message the user received.

This idea, which originated with my colleague Tristan Harris, is based on experience with cults. When you want to deprogram a cult member, it is really important that the call to action come from another member of the cult, ideally the leader.

decentralization had a cost: no one had an incentive to make internet tools easy to use. Frustrated by those tools, users embraced easy-to-use alternatives from Facebook and Google. This allowed the platforms to centralize the internet, inserting themselves between users and content, effectively imposing a tax on both sides. This is a great business model for Facebook and Google—and convenient in the short term for customers—but we are drowning in evidence that there are costs that society may not be able to afford.

Second, the chief executive officers of Facebook, Google, Twitter, and others—not just their lawyers—must testify before congressional committees in open session

This is important not just for the public, but also for another crucial constituency: the employees who keep the tech giants running. While many of the folks who run Silicon Valley are extreme libertarians, the people who work there tend to be idealists. They want to believe what they’re doing is good. Forcing tech CEOs like Mark Zuckerberg to justify the unjustifiable, in public—without the shield of spokespeople or PR spin—would go a long way to puncturing their carefully preserved cults of personality in the eyes of their employees.

We also need regulatory fixes. Here are a few ideas.

First, it’s essential to ban digital bots that impersonate humans. They distort the “public square” in a way that was never possible in history, no matter how many anonymous leaflets you printed.

At a minimum, the law could require explicit labeling of all bots, the ability for users to block them, and liability on the part of platform vendors for the harm bots cause.

Second, the platforms should not be allowed to make any acquisitions until they have addressed the damage caused to date, taken steps to prevent harm in the future, and demonstrated that such acquisitions will not result in diminished competition.

An underappreciated aspect of the platforms’ growth is their pattern of gobbling up smaller firms—in Facebook’s case, that includes Instagram and WhatsApp; in Google’s, it includes YouTube, Google Maps, AdSense, and many others—and using them to extend their monopoly power.

This is important, because the internet has lost something very valuable. The early internet was designed to be decentralized. It treated all content and all content owners equally. That equality had value in society, as it kept the playing field level and encouraged new entrants.

There’s no doubt that the platforms have the technological capacity to reach out to every affected person. No matter the cost, platform companies must absorb it as the price for their carelessness in allowing the manipulation.

Third, the platforms must be transparent about who is behind political and issues-based communication.

Transparency with respect to those who sponsor political advertising of all kinds is a step toward rebuilding trust in our political institutions.

Fourth, the platforms must be more transparent about their algorithms. Users deserve to know why they see what they see in their news feeds and search results. If Facebook and Google had to be up-front about the reason you’re seeing conspiracy theories—namely, that it’s good for business—they would be far less likely to stick to that tactic

Allowing third parties to audit the algorithms would go even further toward maintaining transparency. Facebook and Google make millions of editorial choices every hour and must accept responsibility for the consequences of those choices. Consumers should also be able to see what attributes are causing advertisers to target them.

Fifth, the platforms should be required to have a more equitable contractual relationship with users. Facebook, Google, and others have asserted unprecedented rights with respect to end-user license agreements (EULAs), the contracts that specify the relationship between platform and user.

All software platforms should be required to offer a legitimate opt-out, one that enables users to stick with the prior version if they do not like the new EULA.

“Forking” platforms between old and new versions would have several benefits: increased consumer choice, greater transparency on the EULA, and more care in the rollout of new functionality, among others. It would limit the risk that platforms would run massive social experiments on millions—or billions—of users without appropriate prior notification. Maintaining more than one version of their services would be expensive for Facebook, Google, and the rest, but in software that has always been one of the costs of success. Why should this generation get a pass?

Sixth, we need a limit on the commercial exploitation of consumer data by internet platforms. Customers understand that their “free” use of platforms like Facebook and Google gives the platforms license to exploit personal data. The problem is that platforms are using that data in ways consumers do not understand, and might not accept if they did.

Not only do the platforms use your data on their own sites, but they also lease it to third parties to use all over the internet. And they will use that data forever, unless someone tells them to stop.

There should be a statute of limitations on the use of consumer data by a platform and its customers. Perhaps that limit should be ninety days, perhaps a year. But at some point, users must have the right to renegotiate the terms of how their data is used.

Seventh, consumers, not the platforms, should own their own data. In the case of Facebook, this includes posts, friends, and events—in short, the entire social graph. Users created this data, so they should have the right to export it to other social networks.

It would be analogous to the regulation of the AT&T monopoly’s long-distance business, which led to lower prices and better service for consumers.

Eighth, and finally, we should consider that the time has come to revive the country’s traditional approach to monopoly. Since the Reagan era, antitrust law has operated under the principle that monopoly is not a problem so long as it doesn’t result in higher prices for consumers.

Under that framework, Facebook and Google have been allowed to dominate several industries—not just search and social media but also email, video, photos, and digital ad sales, among others—increasing their monopolies by buying potential rivals like YouTube and Instagram.

While superficially appealing, this approach ignores costs that don’t show up in a price tag. Addiction to Facebook, YouTube, and other platforms has a cost. Election manipulation has a cost. Reduced innovation and shrinkage of the entrepreneurial economy has a cost. All of these costs are evident today. We can quantify them well enough to appreciate that the costs to consumers of concentration on the internet are unacceptably high.

Again and again, the Facebook Papers show staffers sounding alarms about the dangers posed by the platform—how Facebook amplifies extremism and misinformation, how it incites violence, how it encourages radicalization and political polarization. Again and again, staffers reckon with the ways in which Facebook’s decisions stoke these harms, and they plead with leadership to do more.

And again and again, staffers say, Facebook’s leaders ignore them.

Facebook has dismissed the concerns of its employees in manifold ways.

One of its cleverer tactics is to argue that staffers who have raised the alarm about the damage done by their employer are simply enjoying Facebook’s “very open culture,” in which people are encouraged to share their opinions, a spokesperson told me. This stance allows Facebook to claim transparency while ignoring the substance of the complaints, and the implication of the complaints: that many of Facebook’s employees believe their company operates without a moral compass.

When you stitch together the stories that spanned the period between Joe Biden’s election and his inauguration, it’s easy to see Facebook as instrumental to the attack on January 6. (A spokesperson told me that the notion that Facebook played an instrumental role in the insurrection is “absurd.”)

what emerges from a close reading of Facebook documents, and observation of the manner in which the company connects large groups of people quickly, is that Facebook isn’t a passive tool but a catalyst. Had the organizers tried to plan the rally using other technologies of earlier eras, such as telephones, they would have had to identify and reach out individually to each prospective participant, then persuade them to travel to Washington. Facebook made people’s efforts at coordination highly visible on a global scale.

The platform not only helped them recruit participants but offered people a sense of strength in numbers. Facebook proved to be the perfect hype machine for the coup-inclined.

In November 2019, Facebook staffers noticed they had a serious problem. Facebook offers a collection of one-tap emoji reactions. Today, they include “like,” “love,” “care,” “haha,” “wow,” “sad,” and “angry.” Company researchers had found that the posts dominated by “angry” reactions were substantially more likely to go against community standards, including prohibitions on various types of misinformation, according to internal documents.

In July 2020, researchers presented the findings of a series of experiments. At the time, Facebook was already weighting the reactions other than “like” more heavily in its algorithm—meaning posts that got an “angry” reaction were more likely to show up in users’ News Feeds than posts that simply got a “like.” Anger-inducing content didn’t spread just because people were more likely to share things that made them angry; the algorithm gave anger-inducing content an edge. Facebook’s Integrity workers—employees tasked with tackling problems such as misinformation and espionage on the platform—concluded that they had good reason to believe targeting posts that induced anger would help stop the spread of harmful content.

By dialing anger’s weight back to zero in the algorithm, the researchers found, they could keep posts to which people reacted angrily from being viewed by as many users. That, in turn, translated to a significant (up to 5 percent) reduction in the hate speech, civic misinformation, bullying, and violent posts—all of which are correlated with offline violence—to which users were exposed.

Facebook rolled out the change in early September 2020, documents show; a Facebook spokesperson confirmed that the change has remained in effect. It was a real victory for employees of the Integrity team.

But it doesn’t normally work out that way. In April 2020, according to Frances Haugen’s filings with the SEC, Facebook employees had recommended tweaking the algorithm so that the News Feed would deprioritize the surfacing of content for people based on their Facebook friends’ behavior. The idea was that a person’s News Feed should be shaped more by people and groups that a person had chosen to follow. Up until that point, if your Facebook friend saw a conspiracy theory and reacted to it, Facebook’s algorithm might show it to you, too. The algorithm treated any engagement in your network as a signal that something was worth sharing. But now Facebook workers wanted to build circuit breakers to slow this form of sharing.

Experiments showed that this change would impede the distribution of hateful, polarizing, and violence-inciting content in people’s News Feeds. But Zuckerberg “rejected this intervention that could have reduced the risk of violence in the 2020 election,” Haugen’s SEC filing says. An internal message characterizing Zuckerberg’s reasoning says he wanted to avoid new features that would get in the way of “meaningful social interactions.” But according to Facebook’s definition, its employees say, engagement is considered “meaningful” even when it entails bullying, hate speech, and reshares of harmful content.

This episode, like Facebook’s response to the incitement that proliferated between the election and January 6, reflects a fundamental problem with the platform

Facebook’s megascale allows the company to influence the speech and thought patterns of billions of people. What the world is seeing now, through the window provided by reams of internal documents, is that Facebook catalogs and studies the harm it inflicts on people. And then it keeps harming people anyway.

“I am worried that Mark’s continuing pattern of answering a different question than the question that was asked is a symptom of some larger problem,” wrote one Facebook employee in an internal post in June 2020, referring to Zuckerberg. “I sincerely hope that I am wrong, and I’m still hopeful for progress. But I also fully understand my colleagues who have given up on this company, and I can’t blame them for leaving. Facebook is not neutral, and working here isn’t either.”

It is quite a thing to see, the sheer number of Facebook employees—people who presumably understand their company as well as or better than outside observers—who believe their employer to be morally bankrupt.

I spoke with several former Facebook employees who described the company’s metrics-driven culture as extreme, even by Silicon Valley standards

Facebook workers are under tremendous pressure to quantitatively demonstrate their individual contributions to the company’s growth goals, they told me. New products and features aren’t approved unless the staffers pitching them demonstrate how they will drive engagement.

e worries have been exacerbated lately by fears about a decline in new posts on Facebook, two former employees who left the company in recent years told me. People are posting new material less frequently to Facebook, and its users are on average older than those of other social platforms.

One of Facebook’s Integrity staffers wrote at length about this dynamic in a goodbye note to colleagues in August 2020, describing how risks to Facebook users “fester” because of the “asymmetrical” burden placed on employees to “demonstrate legitimacy and user value” before launching any harm-mitigation tactics—a burden not shared by those developing new features or algorithm changes with growth and engagement in mind

The note said:We were willing to act only after things had spiraled into a dire state … Personally, during the time that we hesitated, I’ve seen folks from my hometown go further and further down the rabbithole of QAnon and Covid anti-mask/anti-vax conspiracy on FB. It has been painful to observe.

Current and former Facebook employees describe the same fundamentally broken culture—one in which effective tactics for making Facebook safer are rolled back by leadership or never approved in the first place.

That broken culture has produced a broken platform: an algorithmic ecosystem in which users are pushed toward ever more extreme content, and where Facebook knowingly exposes its users to conspiracy theories, disinformation, and incitement to violence.

One example is a program that amounts to a whitelist for VIPs on Facebook, allowing some of the users most likely to spread misinformation to break Facebook’s rules without facing consequences. Under the program, internal documents show, millions of high-profile users—including politicians—are left alone by Facebook even when they incite violence

whitelisting influential users with massive followings on Facebook isn’t just a secret and uneven application of Facebook’s rules; it amounts to “protecting content that is especially likely to deceive, and hence to harm, people on our platforms.”

Facebook workers tried and failed to end the program. Only when its existence was reported in September by The Wall Street Journal did Facebook’s Oversight Board ask leadership for more information about the practice. Last week, the board publicly rebuked Facebook for not being “fully forthcoming” about the program.

As a result, Facebook has stoked an algorithm arms race within its ranks, pitting core product-and-engineering teams, such as the News Feed team, against their colleagues on Integrity teams, who are tasked with mitigating harm on the platform. These teams establish goals that are often in direct conflict with each other.

“We can’t pretend we don’t see information consumption patterns, and how deeply problematic they are for the longevity of democratic discourse,” a user-experience researcher wrote in an internal comment thread in 2019, in response to a now-infamous memo from Andrew “Boz” Bosworth, a longtime Facebook executive. “There is no neutral position at this stage, it would be powerfully immoral to commit to amorality.”

Zuckerberg has defined Facebook’s mission as making “social infrastructure to give people the power to build a global community that works for all of us,” but in internal research documents his employees point out that communities aren’t always good for society:

When part of a community, individuals typically act in a prosocial manner. They conform, they forge alliances, they cooperate, they organize, they display loyalty, they expect obedience, they share information, they influence others, and so on. Being in a group changes their behavior, their abilities, and, importantly, their capability to harm themselves or others

Thus, when people come together and form communities around harmful topics or identities, the potential for harm can be greater.

The infrastructure choices that Facebook is making to keep its platform relevant are driving down the quality of the site, and exposing its users to more dangers

hose dangers are also unevenly distributed, because of the manner in which certain subpopulations are algorithmically ushered toward like-minded groups

And the subpopulations of Facebook users who are most exposed to dangerous content are also most likely to be in groups where it won’t get reported.

And it knows that 3 percent of Facebook users in the United States are super-consumers of conspiracy theories, accounting for 37 percent of known consumption of misinformation on the platform.

Zuckerberg’s positioning of Facebook’s role in the insurrection is odd. He lumps his company in with traditional media organizations—something he’s ordinarily loath to do, lest the platform be expected to take more responsibility for the quality of the content that appears on it—and suggests that Facebook did more, and did better, than journalism outlets in its response to January 6. What he fails to say is that journalism outlets would never be in the position to help investigators this way, because insurrectionists don’t typically use newspapers and magazines to recruit people for coups.

Facebook wants people to believe that the public must choose between Facebook as it is, on the one hand, and free speech, on the other. This is a false choice. Facebook has a sophisticated understanding of measures it could take to make its platform safer without resorting to broad or ideologically driven censorship tactics.

Facebook knows that no two people see the same version of the platform, and that certain subpopulations experience far more dangerous versions than others do

Facebook knows that people who are isolated—recently widowed or divorced, say, or geographically distant from loved ones—are disproportionately at risk of being exposed to harmful content on the platform.

It knows that repeat offenders are disproportionately responsible for spreading misinformation.

All of this makes the platform rely more heavily on ways it can manipulate what its users see in order to reach its goals. This explains why Facebook is so dependent on the infrastructure of groups, as well as making reshares highly visible, to keep people hooked.

It could consistently enforce its policies regardless of a user’s political power.

Facebook could ban reshares.

It could choose to optimize its platform for safety and quality rather than for growth.

It could tweak its algorithm to prevent widespread distribution of harmful content.

Facebook could create a transparent dashboard so that all of its users can see what’s going viral in real time.

It could make public its rules for how frequently groups can post and how quickly they can grow.

It could also automatically throttle groups when they’re growing too fast, and cap the rate of virality for content that’s spreading too quickly.

Facebook could shift the burden of proof toward people and communities to demonstrate that they’re good actors—and treat reach as a privilege, not a right

You must be vigilant about the informational streams you swim in, deliberate about how you spend your precious attention, unforgiving of those who weaponize your emotions and cognition for their own profit, and deeply untrusting of any scenario in which you’re surrounded by a mob of people who agree with everything you’re saying.

It could do all of these things. But it doesn’t.

Lately, people have been debating just how nefarious Facebook really is. One argument goes something like this: Facebook’s algorithms aren’t magic, its ad targeting isn’t even that good, and most people aren’t that stupid.

All of this may be true, but that shouldn’t be reassuring. An algorithm may just be a big dumb means to an end, a clunky way of maneuvering a massive, dynamic network toward a desired outcome. But Facebook’s enormous size gives it tremendous, unstable power.

Facebook takes whole populations of people, pushes them toward radicalism, and then steers the radicalized toward one another.

When the most powerful company in the world possesses an instrument for manipulating billions of people—an instrument that only it can control, and that its own employees say is badly broken and dangerous—we should take notice.

The lesson for individuals is this:

Facebook could say that its platform is not for everyone. It could sound an alarm for those who wander into the most dangerous corners of Facebook, and those who encounter disproportionately high levels of harmful content

Without seeing how Facebook works at a finer resolution, in real time, we won’t be able to understand how to make the social web compatible with democracy.

Like a good platform, Mr. Trump has found a way to bring communities with relevant interests together while not thinking too much about the long-term costs.

Like all platforms, Mr. Trump is a natural engine of radicalization — for those who support him and those who oppose him.

The new platform announces itself with a catchy motto explicitly stating its intentions: “Make the world more open and connected,” as Facebook declared in its early days; “Make America Great Again,” as the president declares today. But still we avoid asking the hard question: What would happen if the nascent platform achieves those goals? We don’t think too hard about any of it. Even those who don’t like it partake in the platform, feeding it our attention. What’s the harm? After all, it’s free.

For this reason, like any good platform, Mr. Trump is a time suck. Evenings, weekends, holidays, you name it — are all derailed by his demand for your time and attention. Both are the ultimate currency to the Trump platform, allowing him to remain the central figure in American life.

And then there’s our relationship to the Trump platform, which should feel familiar to tech observers. It arrives unexpectedly and is nothing quite like what came before it. The shiny object becomes a media darling. Since it’s a novel experience, the new platform is not taken seriously as a world-changing force

Consuming more of him leads only to a hardening of one’s ideology. Each rally and every successive tweet is more extreme than the last, propelling most of Mr. Trump’s followers deeper down the rabbit hole and intensifying their enthusiasm or disgust for the president

In time we learn that’s not the case. The platform, we find, demands a great deal. Slowly and sneakily it takes and takes little pieces of us. Our data, our attention. It’s not until it’s too late that we learn the platform isn’t free — it only appears so. We learn, to our dismay, that in fact we’ve paid a great price.

As platforms grow more comfortable with their power, they are recognizing that they have options beyond taking posts down or leaving them up. In addition to warning labels, Facebook implemented other “break glass” measures to stem misinformation as the election approached.

Nothing symbolizes this shift as neatly as Facebook’s decision in October (and Twitter’s shortly after) to start banning Holocaust denial. Almost exactly a year earlier, Zuckerberg had proudly tied himself to the First Amendment in a widely publicized “stand for free expression” at Georgetown University.

The evolution continues. Facebook announced earlier this month that it will join platforms such as YouTube and TikTok in removing, not merely labeling or down-ranking, false claims about COVID-19 vaccines.

the pandemic also showed that complete neutrality is impossible. Even though it’s not clear that removing content outright is the best way to correct misperceptions, Facebook and other platforms plainly want to signal that, at least in the current crisis, they don’t want to be seen as feeding people information that might kill them.

It tweaked its algorithm to boost authoritative sources in the news feed and turned off recommendations to join groups based around political or social issues. Facebook is reversing some of these steps now, but it cannot make people forget this toolbox exists in the future

Down-ranking, labeling, or deleting content on an internet platform does not address the social or political circumstances that caused it to be posted in the first place

Even before the pandemic, YouTube had begun adjusting its recommendation algorithm to reduce the spread of borderline and harmful content, and is introducing pop-up nudges to encourage user

Platforms don’t deserve praise for belatedly noticing dumpster fires that they helped create and affixing unobtrusive labels to them

Warning labels for misinformation might make some commentators feel a little better, but whether labels actually do much to contain the spread of false information is still unknown.

News reporting suggests that insiders at Facebook knew they could and should do more about misinformation, but higher-ups vetoed their ideas. YouTube barely acted to stem the flood of misinformation about election results on its platform.

When internet platforms announce new policies, assessing whether they can and will enforce them consistently has always been difficult. In essence, the companies are grading their own work. But too often what can be gleaned from the outside suggests that they’re failing.

And if 2020 finally made clear to platforms the need for greater content moderation, it also exposed the inevitable limits of content moderation.

Content moderation comes to every content platform eventually, and platforms are starting to realize this faster than ever.

even the most powerful platform will never be able to fully compensate for the failures of other governing institutions or be able to stop the leader of the free world from constructing an alternative reality when a whole media ecosystem is ready and willing to enable him. As Renée DiResta wrote in The Atlantic last month, “reducing the supply of misinformation doesn’t eliminate the demand.”

Even so, this year’s events showed that nothing is innate, inevitable, or immutable about platforms as they currently exist. The possibilities for what they might become—and what role they will play in society—are limited more by imagination than any fixed technological constraint, and the companies appear more willing to experiment than ever.

imilar increases occurred at the same time for girls in Canada for mood disorders and for self-harm. Girls in the U.K. also experienced very large increases in anxiety, depression, and self-harm (with much smaller increases for boys).

Some have argued that these increases reflect nothing more than Gen Z’s increased willingness to disclose their mental-health problems. But researchers have found corresponding increases in measurable behaviors such as suicide (for both sexes), and emergency-department admissions for self-harm (for girls only). From 2010 to 2014, rates of hospital admission for self-harm did not increase at all for women in their early 20s, or for boys or young men, but they doubled for girls ages 10 to 14.

The available evidence suggests that Facebook’s products have probably harmed millions of girls. If public officials want to make that case, it could go like this:

from 2010 to 2014, high-school students moved much more of their lives onto social-media platforms.

National surveys of American high-school students show that only about 63 percent reported using a “social networking site” on a daily basis back in 2010.

But as smartphone ownership increased, access became easier and visits became more frequent. By 2014, 80 percent of high-school students said they used a social-media platform on a daily basis, and 24 percent said that they were online “almost constantly.”

2. The timing points to social media.

Notably, girls became much heavier users of the new visually oriented platforms, primarily Instagram (which by 2013 had more than 100 million users), followed by Snapchat, Pinterest, and Tumblr.

Boys are glued to their screens as well, but they aren’t using social media as much; they spend far more time playing video games. When a boy steps away from the console, he does not spend the next few hours worrying about what other players are saying about him

Instagram, in contrast, can loom in a girl’s mind even when the app is not open, driving hours of obsessive thought, worry, and shame.

3. The victims point to Instagram.

In 2017, British researchers asked 1,500 teens to rate how each of the major social-media platforms affected them on certain well-being measures, including anxiety, loneliness, body image, and sleep. Instagram scored as the most harmful, followed by Snapchat and then Facebook.

Facebook’s own research, leaked by the whistleblower Frances Haugen, has a similar finding: “Teens blame Instagram for increases in the rate of anxiety and depression … This reaction was unprompted and consistent across all groups.” The researchers also noted that “social comparison is worse” on Instagram than on rival apps.

4. No other suspect is equally plausible.

A recent experiment confirmed these observations: Young women were randomly assigned to use Instagram, use Facebook, or play a simple video game for seven minutes. The researchers found that “those who used Instagram, but not Facebook, showed decreased body satisfaction, decreased positive affect, and increased negative affect.”

Snapchat’s filters “keep the focus on the face,” whereas Instagram “focuses heavily on the body and lifestyle.

(Boys lost less, and may even have gained, when they took up multiplayer fantasy games, especially those that put them into teams.)

The subset of studies that allow researchers to isolate social media, and Instagram in particular, show a much stronger relationship with poor mental health. The same goes for those that zoom in on girls rather than all teens.

In a 2019 internal essay, Andrew Bosworth, a longtime company executive, wrote:While Facebook may not be nicotine I think it is probably like sugar. Sugar is delicious and for most of us there is a special place for it in our lives. But like all things it benefits from moderation.

Bosworth was proposing what medical researchers call a “dose-response relationship.” Sugar, salt, alcohol, and many other substances that are dangerous in large doses are harmless in small ones.

his framing also implies that any health problems caused by social media result from the user’s lack of self-control. That’s exactly what Bosworth concluded: “Each of us must take responsibility for ourselves.” The dose-response frame also points to cheap solutions that pose no threat to its business model. The company can simply offer more tools to help Instagram and Facebook users limit their consumption.

social-media platforms are not like sugar. They don’t just affect the individuals who overindulge. Rather, when teens went from texting their close friends on flip phones in 2010 to posting carefully curated photographs and awaiting comments and likes by 2014, the change rewired everyone’s social life.

Improvements in technology generally help friends connect, but the move onto social-media platforms also made it easier—indeed, almost obligatory––for users to perform for one another.

Public performance is risky. Private conversation is far more playful. A bad joke or poorly chosen word among friends elicits groans, or perhaps a rebuke and a chance to apologize. Getting repeated feedback in a low-stakes environment is one of the main ways that play builds social skills, physical skills, and the ability to properly judge risk. Play also strengthens friendships.

When girls started spending hours each day on Instagram, they lost many of the benefits of play.

First, Congress should pass legislation compelling Facebook, Instagram, and all other social-media platforms to allow academic researchers access to their data. One such bill is the Platform Transparency and Accountability Act, proposed by the Stanford University researcher Nate Persily.

The wrong photo can lead to school-wide or even national infamy, cyberbullying from strangers, and a permanent scarlet letter

Performative social media also puts girls into a trap: Those who choose not to play the game are cut off from their classmates

Instagram and, more recently, TikTok have become wired into the way teens interact, much as the telephone became essential to past generations.

f those platforms. Without a proper control group, we can’t be certain that the experiment has been a catastrophic failure, but it probably has been. Until someone comes up with a more plausible explanation for what has happened to Gen Z girls, the most prudent course of action for regulators, legislators, and parents is to take steps to mitigate the harm.

Correlation does not prove causation, but nobody has yet found an alternative explanation for the massive, sudden, gendered, multinational deterioration of teen mental health during the period in question.

Second, Congress should toughen the 1998 Children’s Online Privacy Protection Act. An early version of the legislation proposed 16 as the age at which children should legally be allowed to give away their data and their privacy.

Unfortunately, e-commerce companies lobbied successfully to have the age of “internet adulthood” set instead at 13. Now, more than two decades later, today’s 13-year-olds are not doing well. Federal law is outdated and inadequate. The age should be raised. More power should be given to parents, less to companies.

Third, while Americans wait for lawmakers to act, parents can work with local schools to establish a norm: Delay entry to Instagram and other social platforms until high school.

Right now, families are trapped. I have heard many parents say that they don’t want their children on Instagram, but they allow them to lie about their age and open accounts because, well, that’s what everyone else has done.

This new ability of individuals to make a living directly from their audiences isn’t just transforming journalism. It’s also been the case for adult performers on OnlyFans, musicians on Patreon, B-list celebrities on Cameo. In Hollywood, too, power has migrated toward talent, whether it’s marquee showrunners or actors. This power shift is a major headache for big institutions, from The New York Times to record labels.

Substack’s thesis is, in part, that media companies underpay their most prominent writers. So far, that seems to be bearing out.

many of the writers who took advances now regret doing so: They would have made more money by simply collecting subscription revenue, and paying Substack 10 percent, than making the more complex deals with money up front.

The former Vox writer Matthew Yglesias calculated that taking the advance wound up costing him nearly $400,000 in subscription revenue paid to Substack

One of the writers who left Substack over transgender issues, Jude Doyle, argued that its system of advances amounted to a kind of editorial policy. But the analogy to a media company isn’t clear.

Grace Lavery said she wanted Substack to be more aggressive about stopping harassment, but said she didn’t think threats to boycott the email service over writers she disagrees with made political sense.

“Boycotting Substack because of Jesse Singal would be like boycotting a paper company” over a writer who has books printed on their paper, she said.

The real threat is competing platforms with a different model. The most technically powerful of those is probably Ghost, which allows writers to send and charge for newsletters, with monthly fees starting at $9.

While Substack is backed by the venture capital firm Andreessen Horowitz, Ghost has Wikipedia vibes: It is open-source software developed by a nonprofit.

it’s easy to leave. Unlike on Facebook or Twitter, Substack writers can simply take their email lists and direct connections to their readers with them.

Ghost represents an even purer departure from legacy media. More than half of the sites on the platform simply run the software off their own servers.

Mr. Sullivan, who said he saw Substack as his tech platform, not his publisher, has begun deliberately promoting smaller writers in an “In the Stacks” section and said he was interested in figuring out how to bundle subscriptions.

This week, eight writers who cover tech, media and culture — Mr. Warzel, Mr. Newton, Anne Helen Petersen, Nick Quah, Eric Newcomer, Delia Cai, Ryan Broderick and Kim Zetter — are starting a “virtual newsroom” called Sidechannel on Discord, a platform for text and voice conversations

“We’re coming out of this era where platforms own people, and moving into this era where people own platforms,” he said. “We have to prove to the writers we’re delivering enough value to them to keep them happy and help them succeed.”

my informal survey of Substack writers found that most are fond of the company and plan to stick around for now — but not out of the sense of loyalty, shared mission or deep identification that used to run through media companies.

“Taking V.C. money does not create in me a sense of obligation,”

On Monday, the platform went dark after a hacker took over the accounts of 178 users, including Torba and the Republican congresswoman Marjorie Taylor Greene.

Gab, a Twitter-like website promoted by Torba as a bastion of free speech, has long been a forum of last resort for extremists and conspiracy theorists who have been banned on other online platforms. It attained worldwide notoriety in 2018 when a user, Robert Bowers, wrote on the site that he was “going in”, shortly before allegedly entering the Tree of Life synagogue in Pittsburgh, Pennsylvania, and killing eleven people.

The 2017 share offering, for example, required a minimum investment of $199.10, and rewarded investors who contributed a greater amount with “perks”. Users who invested $200 could display a “Gab investor badge” on the site. The badges corresponded with a tag in the database, which allowed investors to be looked at in detail.

The leaked files contained what appears to be a database of over 4.1 million registered users on the site and tags identifying subscribers as “investors”, “verified” users and “pro” users.

Some of the people associated with investors’ accounts had high-profile jobs and public roles, while spewing hate and extremist beliefs online.

The data breach also appears to offer some insight into users tagged as “verified” by Gab, which according to the platform’s own explanation means that they have completed a verification process that includes matching their display name to a government ID.

And it appears to include a list of users registered as “pros”, which allows users to access additional features and a badge at a price starting at $99 year. The database indicates over 18,000 users had paid to be pro users at the time of the breach. Nearly 4,000 users were flagged as donors to Gab’s repeated attempts to attract voluntary gifts from users.

Direct messages included in the leak appear to show close communication between Torba and a major QAnon influencer who is labeled a Gab investor, seemingly reinforcing the CEO’s public efforts to make Gab a home for adherents to the QAnon conspiracy theory, which helped fuel the 6 January attack on the nation’s Capitol.

According to Wired, the data exposed in the apparent hack was sourced by a hacker who had found a security vulnerability in the site.

“Gab was negligent at best and malicious at worst” in its approach to security, she added. “It is hard to envision a scenario where a company cared less about user data than this one.”

in the end, Facebook’s interest was fleeting. Mr. Zuckerberg and other senior executives largely shelved the basic research, according to previously unreported internal documents and people familiar with the effort, and weakened or blocked efforts to apply its conclusions to Facebook products.

At Facebook, “There was this soul-searching period after 2016 that seemed to me this period of really sincere, ‘Oh man, what if we really did mess up the world?’

Another concern, they and others said, was that some proposed changes would have disproportionately affected conservative users and publishers, at a time when the company faced accusations from the right of political bias.

Americans were drifting apart on fundamental societal issues well before the creation of social media, decades of Pew Research Center surveys have shown. But 60% of Americans think the country’s biggest tech companies are helping further divide the country, while only 11% believe they are uniting it, according to a Gallup-Knight survey in March.

Facebook policy chief Joel Kaplan, who played a central role in vetting proposed changes, argued at the time that efforts to make conversations on the platform more civil were “paternalistic,” said people familiar with his comments.

The high number of extremist groups was concerning, the presentation says. Worse was Facebook’s realization that its algorithms were responsible for their growth. The 2016 presentation states that “64% of all extremist group joins are due to our recommendation tools” and that most of the activity came from the platform’s “Groups You Should Join” and “Discover” algorithms: “Our recommendation systems grow the problem.”

In a sign of how far the company has moved, Mr. Zuckerberg in January said he would stand up “against those who say that new types of communities forming on social media are dividing us.” People who have heard him speak privately said he argues social media bears little responsibility for polarization.

Fixing the polarization problem would be difficult, requiring Facebook to rethink some of its core products. Most notably, the project forced Facebook to consider how it prioritized “user engagement”—a metric involving time spent, likes, shares and comments that for years had been the lodestar of its system.

Even before the teams’ 2017 creation, Facebook researchers had found signs of trouble. A 2016 presentation that names as author a Facebook researcher and sociologist, Monica Lee, found extremist content thriving in more than one-third of large German political groups on the platform.

Swamped with racist, conspiracy-minded and pro-Russian content, the groups were disproportionately influenced by a subset of hyperactive users, the presentation notes. Most of them were private or secret.

One proposal Mr. Uribe’s team championed, called “Sparing Sharing,” would have reduced the spread of content disproportionately favored by hyperactive users, according to people familiar with it. Its effects would be heaviest on content favored by users on the far right and left. Middle-of-the-road users would gain influence.

The Common Ground team sought to tackle the polarization problem directly, said people familiar with the team. Data scientists involved with the effort found some interest groups—often hobby-based groups with no explicit ideological alignment—brought people from different backgrounds together constructively. Other groups appeared to incubate impulses to fight, spread falsehoods or demonize a population of outsiders.

Mr. Pariser said that started to change after March 2018, when Facebook got in hot water after disclosing that Cambridge Analytica, the political-analytics startup, improperly obtained Facebook data about tens of millions of people. The shift has gained momentum since, he said: “The internal pendulum swung really hard to ‘the media hates us no matter what we do, so let’s just batten down the hatches.’ ”

Building these features and combating polarization might come at a cost of lower engagement, the Common Ground team warned in a mid-2018 document, describing some of its own proposals as “antigrowth” and requiring Facebook to “take a moral stance.”

Taking action would require Facebook to form partnerships with academics and nonprofits to give credibility to changes affecting public conversation, the document says. This was becoming difficult as the company slogged through controversies after the 2016 presidential election.

Asked to combat fake news, spam, clickbait and inauthentic users, the employees looked for ways to diminish the reach of such ills. One early discovery: Bad behavior came disproportionately from a small pool of hyperpartisan users.

A second finding in the U.S. saw a larger infrastructure of accounts and publishers on the far right than on the far left. Outside observers were documenting the same phenomenon. The gap meant even seemingly apolitical actions such as reducing the spread of clickbait headlines—along the lines of “You Won’t Believe What Happened Next”—affected conservative speech more than liberal content in aggregate.

Every significant new integrity-ranking initiative had to seek the approval of not just engineering managers but also representatives of the public policy, legal, marketing and public-relations departments.

“Engineers that were used to having autonomy maybe over-rotated a bit” after the 2016 election to address Facebook’s perceived flaws, she said. The meetings helped keep that in check. “At the end of the day, if we didn’t reach consensus, we’d frame up the different points of view, and then they’d be raised up to Mark.”

Disapproval from Mr. Kaplan’s team or Facebook’s communications department could scuttle a project, said people familiar with the effort. Negative policy-team reviews killed efforts to build a classification system for hyperpolarized content. Likewise, the Eat Your Veggies process shut down efforts to suppress clickbait about politics more than on other topics.

Under Facebook’s engagement-based metrics, a user who likes, shares or comments on 1,500 pieces of content has more influence on the platform and its algorithms than one who interacts with just 15 posts, allowing “super-sharers” to drown out less-active users

Accounts with hyperactive engagement were far more partisan on average than normal Facebook users, and they were more likely to behave suspiciously, sometimes appearing on the platform as much as 20 hours a day and engaging in spam-like behavior. The behavior suggested some were either people working in shifts or bots.

“We’re explicitly not going to build products that attempt to change people’s beliefs,” one 2018 document states. “We’re focused on products that increase empathy, understanding, and humanization of the ‘other side.’ ”

The debate got kicked up to Mr. Zuckerberg, who heard out both sides in a short meeting, said people briefed on it. His response: Do it, but cut the weighting by 80%. Mr. Zuckerberg also signaled he was losing interest in the effort to recalibrate the platform in the name of social good, they said, asking that they not bring him something like that again.

Mr. Uribe left Facebook and the tech industry within the year. He declined to discuss his work at Facebook in detail but confirmed his advocacy for the Sparing Sharing proposal. He said he left Facebook because of his frustration with company executives and their narrow focus on how integrity changes would affect American politics

While proposals like his did disproportionately affect conservatives in the U.S., he said, in other countries the opposite was true.

The tug of war was resolved in part by the growing furor over the Cambridge Analytica scandal. In a September 2018 reorganization of Facebook’s newsfeed team, managers told employees the company’s priorities were shifting “away from societal good to individual value,” said people present for the discussion. If users wanted to routinely view or post hostile content about groups they didn’t like, Facebook wouldn’t suppress it if the content didn’t specifically violate the company’s rules.

If OnlyFans’s creator earnings were taken as a whole, the company would rank around No. 90 on Forbes’s list of the biggest private companies in America by revenue, ahead of Twitter (now called X), Neiman Marcus Group, New Balance, Hard Rock International and Hallmark Cards.

Many creators now operate like independent media companies, with support staffs, growth strategies and promotional budgets, and work to apply the cold quantification and data analytics of online marketing to the creation of a fantasy life.

The subscription site has often been laughed off as a tabloid punchline, a bawdy corner of the internet where young, underpaid women (teachers, nurses, cops) sell nude photos, get found out and lose their jobs.

pressures to perform for a global audience; an internet that never forgets. “There is simply no room for naivety,” one said in a guide posted to Reddit’s r/CreatorsAdvice.

America’s social media giants for years have held up online virality as the ultimate goal, doling out measurements of followers, reactions and hearts with an unspoken promise: that internet love can translate into sponsorships and endorsement deals

But OnlyFans represents the creator economy at its most blatantly transactional — a place where viewers pay upfront for creators’ labor, and intimacy is just another unit of content to monetize.

The fast ascent of OnlyFans further spotlights how the internet has helped foster a new style of modern gig work that creators see as safe, remote and self-directed,

Creators’ nonchalance about the digital sex trade has fueled a broader debate about whether the site’s promotion of feminist autonomy is a facade: just a new class of techno-capitalism, selling the same patriarchal dream.

But OnlyFans increasingly has become the model for how a new generation of online creators gets paid. Influencers popular on mainstream sites use it to capitalize on the audiences they’ve spent years building. And OnlyFans creators have turned going viral on the big social networks into a marketing strategy, using Facebook, Twitter and TikTok as sales funnels for getting new viewers to subscribe.

many creators, she added, still find it uniquely alluring — a rational choice in an often-irrational environment for gender, work and power. “Why would I spend my day doing dirty, degrading, minimum-wage labor when I can do something that brings more money in and that I have a lot more control over?”

it is targeting major “growth regions” in Latin America, Europe and Australia. (The Mexican diver Diego Balleza said he is using his $15-a-month account to save up for next year’s Paris Olympics.)

“Does an accountant always enjoy their work? No. All work has pleasure and pain, and a lot of it is boring and annoying. Does that mean they’re being exploited?”

Adams’s operation is registered in state business records as a limited liability company and offers quarterly employee performance reviews and catered lunch. It also runs with factory-like efficiency, thanks largely to a system designed in-house to track millions of data points on customers and content and ensure every video is rigorously planned and optimized.

Since sending her first photo in 2021, Adams’s OnlyFans accounts have earned $16.5 million in sales, more than 1.4 million fans and more than 11 million “likes.” She now makes about $30,000 a day — more than most American small businesses — from subscriptions, video sales, messages and tips, half of which is pure profit

Adams’s team sees its business as one of harmless, destigmatized gratification, in which both sides get what they want. The buyers are swiped over in dating apps, widowed, divorced or bored, eager to pay for the illusion of intimacy with an otherwise unattainable match. And the sellers see themselves as not all that different from the influencers they watched growing up on YouTube, charging for parts of their lives they’d otherwise share for free.

“This is normal for my generation, you know?

“I can go on TikTok right now and see ten girls wearing the bare minimum of clothing just to get people to join their page. Why not go the extra step to make money off it?”

the job can be financially precarious and mentally taxing, demanding not just the technical labor of recording, editing, managing and marketing but also the physical and emotional labor of adopting a persona to keep clients feeling special and eager to spend.

enix International Limited, is based, the company said its sales grew from $238 million in 2019 to more than $5.5 billion last year.

Its international army of creators has also grown from 348,000 in 2019 to more than 3 million today — a tenfold increase.

The company paid its owner, the Ukrainian American venture capitalist Leonid Radvinsky, $338 million in dividends last year.)

portion of its creator base and 70 percent of its annual revenue

When Tim Stokely, a London-based operator of live-cam sex sites, founded OnlyFans with his brother in 2016, he framed it as a simple way to monetize the creators who were becoming the world’s new celebrities — the same online influencers, just with a payment button. In 2019, Stokely told Wired magazine that his site was like “a bolt-on to your existing social media,” in the same way “Uber is a bolt-on to your car.”

Before OnlyFans, pornography on the internet had been largely a top-down enterprise, with agents, producers, studios and other middlemen hoarding the profits of performers’ work. OnlyFans democratized that business model, letting the workers run the show: recording their own content, deciding their prices, selling it however they’d like and reaping the full reward.

The platform bans real-world prostitution, as well as extreme or illegal content, and requires everyone who shows up on camera to verify they’re 18 or older by sending in a video selfie showing them holding a government-issued ID.

OnlyFans operates as a neutral marketplace, with no ads, trending topics or recommendation algorithms, placing few limitations on what creators can sell but also making it necessary for them to market themselves or fade away.

After sending other creators’ agents their money over PayPal, Adams’s ad workers send suggestions over the messaging app Telegram on how Bryce should be marketed, depending on the clientele. OnlyFans models whose fans tend to prefer the “girlfriend experience,” for instance, are told to talk up her authenticity: “Bryce is a real, fit girl who wants to get to know you

Like most platforms, OnlyFans suffers from a problem of incredible pay inequality, with the bulk of the profits concentrated in the bank accounts of the lucky few.

the top 1 percent of accounts made 33 percent of the money, and that most accounts took home less than $145 a month

Watching their partner have sex with someone else sometimes sparked what they called “classic little jealousy issues,” which Adams said they resolved with “more communication, more growing up.” The money was just too good. And over time, they adopted a self-affirming ideology that framed everything as just business. Things that were tough to do but got easier with practice, like shooting a sex scene, they called, in gym terms, “reps.” Things one may not want to do at first, but require some mental work to approach, became “self-limiting beliefs.”

They started hiring workers through friends and family, and what was once just Adams became a team effort, in which everyone was expected to workshop caption and video ideas. The group evaluated content under what Brian, who is 31, called a “triangulation method” that factored their comfort level with a piece of content alongside its engagement potential and “brand match.” Bryce the person gave way to Bryce the brand, a commercialized persona drafted by committee and refined for maximum marketability.

One of the operation’s most subtly critical components is a piece of software known as “the Tool,” which they developed and maintain in-house. The Tool scrapes and compiles every “like” and view on all of Adams’s social network accounts, every OnlyFans “fan action” and transaction, and every text, sext and chat message — more than 20 million lines of text so far.

It houses reams of customer data and a library of preset messages that Adams and her chatters can send to fans, helping to automate their reactions and flirtations — “an 80 percent template for a personalized response,” she said.

And it’s linked to a searchable database, in which hundreds of sex scenes are described in detail — by price, total sales, participants and general theme — and given a unique “stock keeping unit,” or SKU, much like the scannable codes on a grocery store shelf. If a fan says they like a certain sexual scenario, a team member can instantly surface any relevant scenes for an easy upsell. “Classic inventory chain,” Adams said.

The systemized database is especially handy for the young women of Adams’s chat team, known as the “girlfriends,” who work at a bench of laptops in the gym’s upper loft. The Tool helped “supercharge her messaging, which ended up, like, 3X-ing her output,” Brian said, meaning it tripled.

Keeping men talking is especially important because the chat window is where Adams’s team sends out their mass-message sales promotions, and the girlfriends never really know what to expect. One girlfriend said she’s had as many as four different sexting sessions going at once.

Adams employs a small team that helps her pay other OnlyFans creators to give away codes fans can use for free short-term trials. The team tracks redemption rates and promotional effectiveness in a voluminous spreadsheet, looking for guys who double up on discount codes, known as “stackers,” as well as bad bets and outright fraud.

Many OnlyFans creators don’t offer anything explicit, and the site has pushed to spotlight its stable of chefs, comedians and mountain bikers on a streaming channel, OFTV. But erotic content on the platform is inescapable; even some outwardly conventional creators shed their clothes behind the paywall

Creators with a more hardcore fan base, meanwhile, are told to cut to the chase: “300+ sex tapes & counting”; “Bryce doesn’t say no, she’s the most wild, authentic girl you will ever find.”

The $18 an hour she makes on the ad team, however, is increasingly dwarfed by the money Leigh makes from her personal OnlyFans account, where she sells sex scenes with her boyfriend for $10 a month. Leigh made $92,000 in gross sales in July, thanks largely to revenue from new fans who found her through Adams or the bikini videos Leigh posts to her 170,000-follower TikTok account

“This is a real job. You dedicate your time to it every single day. You’re always learning, you’re always doing new things,” she said. “I’d never thought I’d be good at business, but learning all these business tactics really empowers you. I have my own LLC; I don’t know any other 20-year-old right now that has their own LLC.”

The team is meeting all traffic goals, per their internal dashboard, which showed that through the day on a recent Thursday they’d gained 2,221,835 video plays, 19,707 landing-page clicks, 6,372 new OnlyFans subscribers and 9,024 new social-network followers. And to keep in shape, Adams and her boyfriend are abiding by a rigorous daily diet and workout plan

They eat the same Chick-fil-A salad at every lunch, track every calorie and pay a gym assistant to record data on every rep and weight of their exercise.

But the OnlyFans business is competitive, and it does not always feel to the couple like they’ve done enough. Their new personal challenge, they said, is to go viral on the other platforms as often as possible, largely through jokey TikTok clips and bikini videos that don’t give away too much.

the host told creators this sales-funnel technique was key to helping build the “cult of you”: “Someone’s fascination will become infatuation, which will make you a lot of money.”

Adams’s company has worked to reverse engineer the often-inscrutable art of virality, and Brian now estimates Adams makes about $5,000 in revenue for every million short-form video views she gets on TikTok.

Her team has begun ranking each platform by the amount of money they expect they can get from each viewer there, a metric they call “fan lifetime value.” (Subscribers who click through to her from Facebook tend to spend the most, the data show. Facebook declined to comment.)

The younger workers said they see the couple as mentors, and the two are constantly reminding them that the job of a creator is not a “lottery ticket” and requires a persistent grind. Whenever one complains about their lack of engagement, Brian said he responds, “When’s the last time you posted 60 different videos, 60 days in a row, on your Instagram Reels?”

But some have taken to it quite naturally. Rayna Rose, 19, was working last year at a hair salon, sweeping floors for $12 an hour, when an old high school classmate who worked with Adams asked whether she wanted to try OnlyFans and make $500 a video.

Rose started making videos and working as a chatter for $18 an hour but recently renegotiated her contract with Adams to focus more on her personal OnlyFans account, where she has nearly 30,000 fans, many of whom pay $10 a month.

One recent evening this summer, Adams was in the farm’s gym when her boyfriend told her he was headed to their guest room to record a collab with Rose, who was wearing a blue bikini top and braided pigtails.

“Go have fun,” Adams told them as they walked away. “Make good content.” The 15-minute video has so far sold more than 1,400 copies and accounted for more than $30,000 in sales.

Rose said she has lost friends due to her “lifestyle,” with one messaging her recently, “Can you imagine how successful you would be if you studied regularly and spent your time wisely?”

The message stung but, in Rose’s eyes, they didn’t understand her at all. She feels, for the first time, like she has a sense of purpose: She wants to be a full-time influencer. She expects to clear $200,000 in earnings this year and is now planning to move out of her parents’ house.

“I had no idea what I wanted to do with my life. And now I know,” she said. “I want to be big. I want to be, like, mainstream.”

Many popular TikTok videos of the invasion — including of Ukrainians livestreaming from their bunkers — offer real accounts of the action, according to researchers who study the platform. But other videos have been impossible to authenticate and substantiate. Some simply appear to be exploiting the interest in the invasion for views, the researchers said.

The clip was then used in many TikTok videos, some of which included a note stating that all 13 soldiers had died. Ukrainian officials later said in a Facebook post that the men were alive and had been taken prisoner, but the TikTok videos have not been corrected.

“People trust it. The result is that a lot of people are seeing false information about Ukraine and believing it.”

TikTok and other social media platforms are also under pressure from U.S. lawmakers and Ukrainian officials to curb Russian misinformation about the war, especially from state-backed media outlets such as Russia Today and Sputnik.

For years, TikTok largely escaped sustained scrutiny about its content. Unlike Facebook, which has been around since 2004, and YouTube, which was founded in 2005, TikTok only became widely used in the past five years.

The app has navigated some controversies in the past. It has faced questions over harmful fads that appeared to originate on its platform, as well as whether it allows underage users and adequately protects their privacy.

That includes TikTok’s algorithm for its “For You” page, which suggests videos based on what people have previously seen, liked or shared. Viewing one video with misinformation likely leads to more videos with misinformation being shown, Ms. Richards said.

But audio can be misused and taken out of context, Ms. Richards said.

“Video is the hardest format to moderate for all platforms,” said Alex Stamos, the director of the Stanford Internet Observatory and a former head of security at Facebook. “When combined with the fact that TikTok’s algorithm is the primary factor for what content a user sees, as opposed to friendships or follows on the big U.S. platforms, this makes TikTok a uniquely potent platform for viral propaganda.”

“I feel like lately, the videos I’m seeing are designed to get me riled up, or to emotionally manipulate me,” she said. “I get worried so now, sometimes, I find myself Googling something or checking the comments to see if it is real before I trust it.”

“I guess I don’t really know what war looks like,” she said. “But we go to TikTok to learn about everything, so it makes sense we would trust it about this too.”

it’s a substantial difference which deprives social media platforms of the kind of hands-off logic that would make it ridiculous to say phones are bad or the phone company is responsible if planning for a mass murder was carried out over the phone.

the Internet doesn’t ‘do’ anything more than make the distribution of information more efficient and radically lower the formal, informal and financial barriers to entry that used to stand in the way of various marginalized ideas.

Social media can never plead innocence like this because the platforms are designed to addict you and convince you of things.

If the question is: what can social media platforms do to protect against government-backed subversion campaigns like the one we saw in the 2016 campaign the best answer is, we don’t know. And we don’t know for a simple reason: they haven’t tried.

The point is straightforward: the mass collection of data, harnessed to modern computing power and the chance to amass unimaginable wealth has spurred vast technological innovation.

A New York Times investigation in Brazil found that, time and again, videos promoted by the site have upended central elements of daily life

YouTube’s recommendation system is engineered to maximize watchtime, among other factors, the company says, but not to favor any political ideology.

Some parents look to “Dr. YouTube” for health advice but get dangerous misinformation instead, hampering the nation’s efforts to fight diseases like Zika. Viral videos have incited death threats against public health advocates.

And in politics, a wave of right-wing YouTube stars ran for office alongside Mr. Bolsonaro, some winning by historic margins. Most still use the platform, governing the world’s fourth-largest democracy through internet-honed trolling and provocation

Teachers describe classrooms made unruly by students who quote from YouTube conspiracy videos or who, encouraged by right-wing YouTube stars, secretly record their instructors

“Before that, I didn’t have an ideological political background,” Mr. Martins said. YouTube’s auto-playing recommendations, he declared, were “my political education.”

As the system suggests more provocative videos to keep users watching, it can direct them toward extreme content they might otherwise never find. And it is designed to lead users to new topics to pique new interest

The system now drives 70 percent of total time on the platfor

Zeynep Tufekci, a social media scholar, has called it “one of the most powerful radicalizing instruments of the 21st century.”

Danah Boyd, founder of the think tank Data & Society, attributed the disruption in Brazil to YouTube’s unrelenting push for viewer engagement, and the revenues it generates.

Maurício Martins, the local vice president of Mr. Bolsonaro’s party in Niterói, credited “most” of the party’s recruitment to YouTube — including his own.

But the emotions that draw people in — like fear, doubt and anger — are often central features of conspiracy theories, and in particular, experts say, of right-wing extremism

“It was like that with everyone,”

Sometimes I’m watching videos about a game, and all of a sudden it’s a Bolsonaro video,”

More and more, his fellow students are making extremist claims, often citing as evidence YouTube stars like Mr. Moura, the guitarist-turned-conspiracist.

“If social media didn’t exist, I wouldn’t be here,” he said. “Jair Bolsonaro wouldn’t be president.”

In the months after YouTube changed its algorithm, positive mentions of Mr. Bolsonaro ballooned. So did mentions of conspiracy theories that he had floated. This began as polls still showed him to be deeply unpopular, suggesting that the platform was doing more than merely reflecting political trends.

Jonas Kaiser and Yasodara Córdova, with Adrian Rauchfleisch of National Taiwan University, programmed a Brazil-based server to enter a popular channel or search term, then open YouTube’s top recommendations, then follow the recommendations on each of those, and so on.

By repeating this thousands of times, the researchers tracked how the platform moved users from one video to the next. They found that after users watched a video about politics or even entertainment, YouTube’s recommendations often favored right-wing, conspiracy-filled channels like Mr. Moura’s

Not long after YouTube installed its new recommendation engine, Dr. Santana’s patients began telling him that they’d seen videos blaming Zika on vaccines — and, later, on larvicides. Many refused both.

The algorithm had united once-marginal channels — and then built an audience for them

One of those channels belonged to Mr. Bolsonaro, who had long used the platform to post hoaxes and conspiracies

The conspiracies were not limited to politics. Many Brazilians searching YouTube for health care information found videos that terrified them: some said Zika was being spread by vaccines, or by the insecticides meant to curb the spread of the mosquito-borne disease that has ravaged northeastern Brazi

The videos appeared to rise on the platform in much the same way as extremist political content: by making alarming claims and promising forbidden truths that kept users glued to their screens.

Doctors, social workers and former government officials said the videos had created the foundation of a public health crisis as frightened patients refused vaccines and even anti-Zika insecticides.

Crucially, users who watched one far-right channel would often be shown many more.

Medical providers, she said, were competing “every single day” against “Dr. Google and Dr. YouTube” — and they were losing

Brazil’s medical community had reason to feel outmatched. The Harvard researchers found that YouTube’s systems frequently directed users who searched for information on Zika, or even those who watched a reputable video on health issues, toward conspiracy channels

As the far right rose, many of its leading voices had learned to weaponize the conspiracy videos, offering their vast audiences a target: people to blame

Eventually, the YouTube conspiracists turned their spotlight on Debora Diniz, a women’s rights activist whose abortion advocacy had long made her a target of the far right

Bernardo Küster, a YouTube star whose homemade rants had won him 750,000 subscribers and an endorsement from Mr. Bolsonaro, accused her of involvement in the supposed Zika plots.

As far-right and conspiracy channels began citing one another, YouTube’s recommendation system learned to string their videos together

However implausible any individual rumor might be on its own, joined together, they created the impression that dozens of disparate sources were revealing the same terrifying truth.

When the university where Ms. Diniz taught received a warning that a gunman would shoot her and her students, and the police said they could no longer guarantee her safety, she left Brazil.

“The YouTube system of recommending the next video and the next video,” she said, had created “an ecosystem of hate.

“‘I heard here that she’s an enemy of Brazil. I hear in the next one that feminists are changing family values. And the next one I hear that they receive money from abroad” she said. “That loop is what leads someone to say ‘I will do what has to be done.’

In Brazil, this is a growing online practice known as “linchamento” — lynching. Mr. Bolsonaro was an early pioneer, spreading videos in 2012 that falsely accused left-wing academics of plotting to force schools to distribute “gay kits” to convert children to homosexuality.

Mr. Jordy, his tattooed Niterói protégé, was untroubled to learn that his own YouTube campaign, accusing teachers of spreading communism, had turned their lives upside down.One of those teachers, Valeria Borges, said she and her colleagues had been overwhelmed with messages of hate, creating a climate of fear.

Mr. Jordy, far from disputing this, said it had been his goal. “I wanted her to feel fear,” he said

The group’s co-founder, a man-bunned former rock guitarist name Pedro D’Eyrot, said “we have something here that we call the dictatorship of the like.”

Reality, he said, is shaped by whatever message goes most viral.

Even as he spoke, a two-hour YouTube video was captivating the nation. Titled “1964” for the year of Brazil’s military coup, it argued that the takeover had been necessary to save Brazil from communism.Mr. Dominguez, the teenager learning to play guitar, said the video persuaded him that his teachers had fabricated the horrors of military rule.

THE AMERICAN EXPERIMENT rests on three political ideas—“these truths,” Thomas Jefferson called them—political equality, natural rights, and the sovereignty of the people.

After Benjamin Franklin read Jefferson’s draft, he picked up his quill, scratched out the words “sacred & undeniable,” and suggested that “these truths” were, instead, “self-evident.” This was more than a quibble. Truths that are sacred and undeniable are God-given and divine, the stuff of religion. Truths that are self-evident are laws of nature, empirical and observable, the stuff of science. This divide has nearly rent the Republic apart.

The real dispute is between “these truths” and the course of events: Does American history prove these truths, or does it belie them?

The United States rests on a dedication to equality, which is chiefly a moral idea, rooted in Christianity, but it rests, too, on a dedication to inquiry, fearless and unflinching. Its founders agreed with the Scottish philosopher and historian David Hume, who wrote, in 1748, that “Records of Wars, Intrigues, Factions, and Revolutions are so many Collections of Experiments.”9 They believed that truth is to be found in ideas about morality but also in the study of history.

understanding history as a form of inquiry—not as something easy or comforting but as something demanding and exhausting—was central to the nation’s founding. This, too, was new.

A new kind of historical writing, less memorial and more unsettling, only first emerged in the fourteenth century. “History is a philosophical science,” the North African Muslim scholar Ibn Khaldun wrote in 1377, in the prologue to his history of the world, in which he defined history as the study “of the causes and origins of existing things.”11

Only by fits and starts did history become not merely a form of memory but also a form of investigation, to be disputed, like philosophy, its premises questioned, its evidence examined, its arguments countered.

Declaring independence was itself an argument about the relationship between the present and the past, an argument that required evidence of a very particular kind: historical evidence. That’s why most of the Declaration of Independence is a list of historical claims. “To prove this,” Jefferson wrote, “let facts be submitted to a candid world.”

In an attempt to solve this problem, the earliest historians of the United States decided to begin their accounts with Columbus’s voyage, stitching 1776 to 1492. George Bancroft published his History of the United States from the Discovery of the American Continent to the Present in 1834, when the nation was barely more than a half-century old, a fledgling, just hatched. By beginning with Columbus, Bancroft made the United States nearly three centuries older than it was, a many-feathered old bird.

In 1787, then, when Alexander Hamilton asked “whether societies of men are really capable or not of establishing good government from reflection and choice, or whether they are forever destined to depend for their political constitutions on accident and force,” that was the kind of question a scientist asks before beginning an experiment. Time alone would tell. But time has passed. The beginning has come to an end. What, then, is the verdict of history?

In deciding what to leave in and what to leave out, I’ve confined myself to what, in my view, a people constituted as a nation in the early twenty-first century need to know about their own past, mainly because this book is meant to double as an old-fashioned civics book, an explanation of the origins and ends of democratic institutions, from the town meeting to the party system, from the nominating convention to the secret ballot, from talk radio to Internet polls. This book is chiefly a political

Aside from being a brief history of the United States and a civics primer, this book aims to be something else, too: it’s an explanation of the nature of the past. History isn’t only a subject; it’s also a method.

The truths on which the nation was founded are not mysteries, articles of faith, never to be questioned, as if the founding were an act of God, but neither are they lies, all facts fictions, as if nothing can be known, in a world without truth.

Between reverence and worship, on the one side, and irreverence and contempt, on the other, lies an uneasy path, away from false pieties and petty triumphs over people who lived and died and committed both their acts of courage and their sins and errors long before we committed ours. “We cannot hallow this ground,” Lincoln said at Gettysburg. We are obliged, instead, to walk this ground, dedicating ourselves to both the living and the dead.

studying history is like that, looking into one face and seeing, behind it, another, face after face after face. “Know whence you came,” Baldwin told his nephew.17 The past is an inheritance, a gift and a burden. It can’t be shirked. You carry it everywhere. There’s nothing for it but to get to know it.

Nature takes one toll, malice another. History is the study of what remains, what’s left behind, which can be almost anything, so long as it survives the ravages of time and war: letters, diaries, DNA, gravestones, coins, television broadcasts, paintings, DVDs, viruses, abandoned Facebook pages, the transcripts of congressional hearings, the ruins of buildings. Some of these things are saved by chance or accident, like the one house that, as if by miracle, still stands after a hurricane razes a town. But most of what historians study survives because it was purposely kept—placed

As nation-states emerged, they needed to explain themselves, which they did by telling stories about their origins, tying together ribbons of myths, as if everyone in the “English nation,” for instance, had the same ancestors, when, of course, they did not. Very often, histories of nation-states are little more than myths that hide the seams that stitch the nation to the state.15

When the United States declared its independence in 1776, plainly, it was a state, but what made it a nation? The fiction that its people shared a common ancestry was absurd on its face; they came from all over, and, having waged a war against England, the very last thing they wanted to celebrate was their Englishness.

Facts, knowledge, experience, proof. These words come from the law. Around the seventeenth century, they moved into what was then called “natural history”: astronomy, physics, chemistry, geology. By the eighteenth century they were applied to history and to politics, too. These truths: this was the language of reason, of enlightenment, of inquiry, and of history.

Against conquest, slaughter, and slavery came the urgent and abiding question, “By what right?”

Yet the origins of the United States date to 1492 for another, more troubling reason: the nation’s founding truths were forged in a crucible of violence, the products of staggering cruelty, conquest and slaughter, the assassination of worlds.

Locke, spurred both by a growing commitment to religious toleration and by a desire to distinguish English settlement from Spanish conquest, stressed the lack of cultivation as a better justification for taking the natives’ land than religious difference, an emphasis with lasting consequences.

Unlike Polo and Mandeville, Columbus did not make a catalogue of the ways and beliefs of the people he met (only later did he hire Pané to do that). Instead, he decided that the people he met had no ways and beliefs. Every difference he saw as an absence.22 Insisting that they had no faith and no civil government and were therefore infidels and savages who could not rightfully own anything, he claimed possession of their land, by the act of writing. They were a people without truth; he would make his truth theirs. He would tell them where the dead go.

It became commonplace, inevitable, even, first among the Spanish, and then, in turn, among the French, the Dutch, and the English, to see their own prosperity and good health and the terrible sicknesses suffered by the natives as signs from God. “Touching these savages, there is a thing that I cannot omit to remark to you,” one French settler wrote: “it appears visibly that God wishes that they yield their place to new peoples.” Death convinced them at once of their right and of the truth of their faith. “The natives, they are all dead of small Poxe,” John Winthrop wrote when he arrived in New England in 1630: “the Lord hathe cleared our title to what we possess.”

In much of New Spain, the mixed-race children of Spanish men and Indian women, known as mestizos, outnumbered Indians; an intricate caste system marked gradations of skin color, mixtures of Europeans, Native Americans, and Africans, as if skin color were like dyes made of plants, the yellow of sassafras, the red of beets, the black of carob. Later, the English would recognize only black and white, a fantasy of stark and impossible difference, of nights without twilight and days without dawns. And yet both regimes of race, a culture of mixing or a culture of pretending not to mix, pressed upon the brows of every person of the least curiosity the question of common humanity: Are all peoples one?

Elizabeth’s best defender argued that if God decided “the female should rule and govern,” it didn’t matter that women were “weake in nature, feable in bodie, softe in courage,” because God would make every right ruler strong. In any case, England’s constitution abided by a “rule mixte,” in which the authority of the monarch was checked by the power of Parliament; also, “it is not she that ruleth but the lawes.” Elizabeth herself called on yet another authority: the favor of the people.48 A mixed constitution, the rule of law, the will of the people: these were English ideas that Americans would one day make their own, crying, “Liberty!”

In the brutal, bloody century between Columbus’s voyage and John White’s, an idea was born, out of fantasy, out of violence, the idea that there exists in the world a people who live in an actual Garden of Eden, a state of nature, before the giving of laws, before the forming of government. This imagined history of America became an English book of genesis, their new truth. “In the beginning,” the Englishman John Locke would write, “all the world was America.” In America, everything became a beginning.

England’s empire would have a different character than that of either Spain or France. Catholics could make converts by the act of baptism, but Protestants were supposed to teach converts to read the Bible; that meant permanent settlements, families, communities, schools, and churches. Also, England’s empire would be maritime—its navy was its greatest strength. It would be commercial. And, of greatest significance for the course of the nation that would grow out of those settlements, its colonists would be free men, not vassals, guaranteed their “English liberties.”

Beginning with the Virginia charter, the idea of English liberties for English subjects was planted on American soil and, with it, the king’s claim to dominion, a claim that rested on the idea that people like Powhatan and his people lived in darkness and without government, no matter that the English called their leaders kings.

Twenty Englishmen were elected to the House of Burgesses. Twenty Africans were condemned to the house of bondage. Another chapter opened in the American book of genesis: liberty and slavery became the American Abel and Cain.

To build his case against the king, Coke dusted off a copy of an ancient and almost entirely forgotten legal document, known as Magna Carta (literally, the “great charter”), in which, in the year 1215, King John had pledged to his barons that he would obey the “law of the land.” Magna Carta wasn’t nearly as important as Coke made it out to be, but by arguing for its importance, he made it important, not only for English history, but for American history, too, tying the political fate of everyone in England’s colonies to the strange doings of a very bad king from the Middle Ages.