Group items tagged

“Security and privacy have long been top companywide priorities at Twitter,” said Twitter spokeswoman Rebecca Hahn. She said that Zatko’s allegations appeared to be “riddled with inaccuracies” and that Zatko “now appears to be opportunistically seeking to inflict harm on Twitter, its customers, and its shareholders.” Hahn said that Twitter fired Zatko after 15 months “for poor performance and leadership.” Attorneys for Zatko confirmed he was fired but denied it was for performance or leadership.

the whistleblower document alleges the company prioritized user growth over reducing spam, though unwanted content made the user experience worse. Executives stood to win individual bonuses of as much as $10 million tied to increases in daily users, the complaint asserts, and nothing explicitly for cutting spam.

Chief executive Parag Agrawal was “lying” when he tweeted in May that the company was “strongly incentivized to detect and remove as much spam as we possibly can,” the complaint alleges.

Zatko described his decision to go public as an extension of his previous work exposing flaws in specific pieces of software and broader systemic failings in cybersecurity. He was hired at Twitter by former CEO Jack Dorsey in late 2020 after a major hack of the company’s systems.

“I felt ethically bound. This is not a light step to take,” said Zatko, who was fired by Agrawal in January. He declined to discuss what happened at Twitter, except to stand by the formal complaint. Under SEC whistleblower rules, he is entitled to legal protection against retaliation, as well as potential monetary rewards.

A person familiar with Zatko’s tenure said the company investigated Zatko’s security claims during his time there and concluded they were sensationalistic and without merit. Four people familiar with Twitter’s efforts to fight spam said the company deploys extensive manual and automated tools to both measure the extent of spam across the service and reduce it.

In 1998, Zatko had testified to Congress that the internet was so fragile that he and others could take it down with a half-hour of concentrated effort. He later served as the head of cyber grants at the Defense Advanced Research Projects Agency, the Pentagon innovation unit that had backed the internet’s invention.

Overall, Zatko wrote in a February analysis for the company attached as an exhibit to the SEC complaint, “Twitter is grossly negligent in several areas of information security. If these problems are not corrected, regulators, media and users of the platform will be shocked when they inevitably learn about Twitter’s severe lack of security basics.”

Zatko’s complaint says strong security should have been much more important to Twitter, which holds vast amounts of sensitive personal data about users. Twitter has the email addresses and phone numbers of many public figures, as well as dissidents who communicate over the service at great personal risk.

This month, an ex-Twitter employee was convicted of using his position at the company to spy on Saudi dissidents and government critics, passing their information to a close aide of Crown Prince Mohammed bin Salman in exchange for cash and gifts.

Zatko’s complaint says he believed the Indian government had forced Twitter to put one of its agents on the payroll, with access to user data at a time of intense protests in the country. The complaint said supporting information for that claim has gone to the National Security Division of the Justice Department and the Senate Select Committee on Intelligence. Another person familiar with the matter agreed that the employee was probably an agent.

“Take a tech platform that collects massive amounts of user data, combine it with what appears to be an incredibly weak security infrastructure and infuse it with foreign state actors with an agenda, and you’ve got a recipe for disaster,” Charles E. Grassley (R-Iowa), the top Republican on the Senate Judiciary Committee,

Many government leaders and other trusted voices use Twitter to spread important messages quickly, so a hijacked account could drive panic or violence. In 2013, a captured Associated Press handle falsely tweeted about explosions at the White House, sending the Dow Jones industrial average briefly plunging more than 140 points.

After a teenager managed to hijack the verified accounts of Obama, then-candidate Joe Biden, Musk and others in 2020, Twitter’s chief executive at the time, Jack Dorsey, asked Zatko to join him, saying that he could help the world by fixing Twitter’s security and improving the public conversation, Zatko asserts in the complaint.

The complaint — filed last month with the Securities and Exchange Commission and the Department of Justice, as well as the FTC — says thousands of employees still had wide-ranging and poorly tracked internal access to core company software, a situation that for years had led to embarrassing hacks, including the commandeering of accounts held by such high-profile users as Elon Musk and former presidents Barack Obama and Donald Trump.

But at Twitter Zatko encountered problems more widespread than he realized and leadership that didn’t act on his concerns, according to the complaint.

Twitter’s difficulties with weak security stretches back more than a decade before Zatko’s arrival at the company in November 2020. In a pair of 2009 incidents, hackers gained administrative control of the social network, allowing them to reset passwords and access user data. In the first, beginning around January of that year, hackers sent tweets from the accounts of high-profile users, including Fox News and Obama.

Several months later, a hacker was able to guess an employee’s administrative password after gaining access to similar passwords in their personal email account. That hacker was able to reset at least one user’s password and obtain private information about any Twitter user.

Twitter continued to suffer high-profile hacks and security violations, including in 2017, when a contract worker briefly took over Trump’s account, and in the 2020 hack, in which a Florida teen tricked Twitter employees and won access to verified accounts. Twitter then said it put additional safeguards in place.

This year, the Justice Department accused Twitter of asking users for their phone numbers in the name of increased security, then using the numbers for marketing. Twitter agreed to pay a $150 million fine for allegedly breaking the 2011 order, which barred the company from making misrepresentations about the security of personal data.

After Zatko joined the company, he found it had made little progress since the 2011 settlement, the complaint says. The complaint alleges that he was able to reduce the backlog of safety cases, including harassment and threats, from 1 million to 200,000, add staff and push to measure results.

But Zatko saw major gaps in what the company was doing to satisfy its obligations to the FTC, according to the complaint. In Zatko’s interpretation, according to the complaint, the 2011 order required Twitter to implement a Software Development Life Cycle program, a standard process for making sure new code is free of dangerous bugs. The complaint alleges that other employees had been telling the board and the FTC that they were making progress in rolling out that program to Twitter’s systems. But Zatko alleges that he discovered that it had been sent to only a tenth of the company’s projects, and even then treated as optional.

“If all of that is true, I don’t think there’s any doubt that there are order violations,” Vladeck, who is now a Georgetown Law professor, said in an interview. “It is possible that the kinds of problems that Twitter faced eleven years ago are still running through the company.”

“Agrawal’s Tweets and Twitter’s previous blog posts misleadingly imply that Twitter employs proactive, sophisticated systems to measure and block spam bots,” the complaint says. “The reality: mostly outdated, unmonitored, simple scripts plus overworked, inefficient, understaffed, and reactive human teams.”

One current and one former employee recalled that incident, when failures at two Twitter data centers drove concerns that the service could have collapsed for an extended period. “I wondered if the company would exist in a few days,” one of them said.

The current and former employees also agreed with the complaint’s assertion that past reports to various privacy regulators were “misleading at best.”

For example, they said the company implied that it had destroyed all data on users who asked, but the material had spread so widely inside Twitter’s networks, it was impossible to know for sure

As the head of security, Zatko says he also was in charge of a division that investigated users’ complaints about accounts, which meant that he oversaw the removal of some bots, according to the complaint. Spam bots — computer programs that tweet automatically — have long vexed Twitter. Unlike its social media counterparts, Twitter allows users to program bots to be used on its service: For example, the Twitter account @big_ben_clock is programmed to tweet “Bong Bong Bong” every hour in time with Big Ben in London. Twitter also allows people to create accounts without using their real identities, making it harder for the company to distinguish between authentic, duplicate and automated accounts.

In the complaint, Zatko alleges he could not get a straight answer when he sought what he viewed as an important data point: the prevalence of spam and bots across all of Twitter, not just among monetizable users.

Zatko cites a “sensitive source” who said Twitter was afraid to determine that number because it “would harm the image and valuation of the company.” He says the company’s tools for detecting spam are far less robust than implied in various statements.

The complaint also alleges that Zatko warned the board early in his tenure that overlapping outages in the company’s data centers could leave it unable to correctly restart its servers. That could have left the service down for months, or even have caused all of its data to be lost. That came close to happening in 2021, when an “impending catastrophic” crisis threatened the platform’s survival before engineers were able to save the day, the complaint says, without providing further details.

The four people familiar with Twitter’s spam and bot efforts said the engineering and integrity teams run software that samples thousands of tweets per day, and 100 accounts are sampled manually.

Some employees charged with executing the fight agreed that they had been short of staff. One said top executives showed “apathy” toward the issue.

Zatko’s complaint likewise depicts leadership dysfunction, starting with the CEO. Dorsey was largely absent during the pandemic, which made it hard for Zatko to get rulings on who should be in charge of what in areas of overlap and easier for rival executives to avoid collaborating, three current and former employees said.

For example, Zatko would encounter disinformation as part of his mandate to handle complaints, according to the complaint. To that end, he commissioned an outside report that found one of the disinformation teams had unfilled positions, yawning language deficiencies, and a lack of technical tools or the engineers to craft them. The authors said Twitter had no effective means of dealing with consistent spreaders of falsehoods.

Dorsey made little effort to integrate Zatko at the company, according to the three employees as well as two others familiar with the process who spoke on the condition of anonymity to describe sensitive dynamics. In 12 months, Zatko could manage only six one-on-one calls, all less than 30 minutes, with his direct boss Dorsey, who also served as CEO of payments company Square, now known as Block, according to the complaint. Zatko allegedly did almost all of the talking, and Dorsey said perhaps 50 words in the entire year to him. “A couple dozen text messages” rounded out their electronic communication, the complaint alleges.

Faced with such inertia, Zatko asserts that he was unable to solve some of the most serious issues, according to the complaint.

Some 30 percent of company laptops blocked automatic software updates carrying security fixes, and thousands of laptops had complete copies of Twitter’s source code, making them a rich target for hackers, it alleges.

A successful hacker takeover of one of those machines would have been able to sabotage the product with relative ease, because the engineers pushed out changes without being forced to test them first in a simulated environment, current and former employees said.

“It’s near-incredible that for something of that scale there would not be a development test environment separate from production and there would not be a more controlled source-code management process,” said Tony Sager, former chief operating officer at the cyberdefense wing of the National Security Agency, the Information Assurance divisio

Sager is currently senior vice president at the nonprofit Center for Internet Security, where he leads a consensus effort to establish best security practices.

The complaint says that about half of Twitter’s roughly 7,000 full-time employees had wide access to the company’s internal software and that access was not closely monitored, giving them the ability to tap into sensitive data and alter how the service worked. Three current and former employees agreed that these were issues.

“A best practice is that you should only be authorized to see and access what you need to do your job, and nothing else,” said former U.S. chief information security officer Gregory Touhill. “If half the company has access to and can make configuration changes to the production environment, that exposes the company and its customers to significant risk.”

The complaint says Dorsey never encouraged anyone to mislead the board about the shortcomings, but that others deliberately left out bad news.

When Dorsey left in November 2021, a difficult situation worsened under Agrawal, who had been responsible for security decisions as chief technology officer before Zatko’s hiring, the complaint says.

An unnamed executive had prepared a presentation for the new CEO’s first full board meeting, according to the complaint. Zatko’s complaint calls the presentation deeply misleading.

The presentation showed that 92 percent of employee computers had security software installed — without mentioning that those installations determined that a third of the machines were insecure, according to the complaint.

Another graphic implied a downward trend in the number of people with overly broad access, based on the small subset of people who had access to the highest administrative powers, known internally as “God mode.” That number was in the hundreds. But the number of people with broad access to core systems, which Zatko had called out as a big problem after joining, had actually grown slightly and remained in the thousands.

The presentation included only a subset of serious intrusions or other security incidents, from a total Zatko estimated as one per week, and it said that the uncontrolled internal access to core systems was responsible for just 7 percent of incidents, when Zatko calculated the real proportion as 60 percent.

Zatko stopped the material from being presented at the Dec. 9, 2021 meeting, the complaint said. But over his continued objections, Agrawal let it go to the board’s smaller Risk Committee a week later.

Agrawal didn’t respond to requests for comment. In an email to employees after publication of this article, obtained by The Post, he said that privacy and security continues to be a top priority for the company, and he added that the narrative is “riddled with inconsistences” and “presented without important context.”

On Jan. 4, Zatko reported internally that the Risk Committee meeting might have been fraudulent, which triggered an Audit Committee investigation.

Agarwal fired him two weeks later. But Zatko complied with the company’s request to spell out his concerns in writing, even without access to his work email and documents, according to the complaint.

Since Zatko’s departure, Twitter has plunged further into chaos with Musk’s takeover, which the two parties agreed to in May. The stock price has fallen, many employees have quit, and Agrawal has dismissed executives and frozen big projects.

Zatko said he hoped that by bringing new scrutiny and accountability, he could improve the company from the outside.

“I still believe that this is a tremendous platform, and there is huge value and huge risk, and I hope that looking back at this, the world will be a better place, in part because of this.”

it’s not just law enforcement: Clearview has also licensed the app to at least a handful of companies for security purposes.

“The weaponization possibilities of this are endless,” said Eric Goldman, co-director of the High Tech Law Institute at Santa Clara University. “Imagine a rogue law enforcement officer who wants to stalk potential romantic partners, or a foreign government using this to dig up secrets about people to blackmail them or throw them in jail.”

While the company was dodging me, it was also monitoring me. At my request, a number of police officers had run my photo through the Clearview app. They soon received phone calls from company representatives asking if they were talking to the media — a sign that Clearview has the ability and, in this case, the appetite to monitor whom law enforcement is searching for.

The company eventually started answering my questions, saying that its earlier silence was typical of an early-stage start-up in stealth mode. Mr. Ton-That acknowledged designing a prototype for use with augmented-reality glasses but said the company had no plans to release it.

In addition to Mr. Ton-That, Clearview was founded by Richard Schwartz — who was an aide to Rudolph W. Giuliani when he was mayor of New York — and backed financially by Peter Thiel, a venture capitalist behind Facebook and Palantir.

“I’ve come to the conclusion that because information constantly increases, there’s never going to be privacy,” Mr. Scalzo said. “Laws have to determine what’s legal, but you can’t ban technology. Sure, that might lead to a dystopian future or something, but you can’t ban it.”

“In 2017, Peter gave a talented young founder $200,000, which two years later converted to equity in Clearview AI,” said Jeremiah Hall, Mr. Thiel’s spokesman. “That was Peter’s only contribution; he is not involved in the company.”

He began in 2016 by recruiting a couple of engineers. One helped design a program that can automatically collect images of people’s faces from across the internet, such as employment sites, news sites, educational sites, and social networks including Facebook, YouTube, Twitter, Instagram and even Venmo

Representatives of those companies said their policies prohibit such scraping, and Twitter said it explicitly banned use of its data for facial recognition

Another engineer was hired to perfect a facial recognition algorithm that was derived from academic papers. The result: a system that uses what Mr. Ton-That described as a “state-of-the-art neural net” to convert all the images into mathematical formulas, or vectors, based on facial geometry — like how far apart a person’s eyes are

Clearview created a vast directory that clustered all the photos with similar vectors into “neighborhoods.”

When a user uploads a photo of a face into Clearview’s system, it converts the face into a vector and then shows all the scraped photos stored in that vector’s neighborhood — along with the links to the sites from which those images came.

Mr. Schwartz paid for server costs and basic expenses, but the operation was bare bones; everyone worked from home. “I was living on credit card debt,” Mr. Ton-That said. “Plus, I was a Bitcoin believer, so I had some of those.”

The company soon changed its name to Clearview AI and began marketing to law enforcement. That was when the company got its first round of funding from outside investors: Mr. Thiel and Kirenaga Partners

Mr. Schwartz and Mr. Ton-That met in 2016 at a book event at the Manhattan Institute, a conservative think tank. Mr. Schwartz, now 61, had amassed an impressive Rolodex working for Mr. Giuliani in the 1990s and serving as the editorial page editor of The New York Daily News in the early 2000s. The two soon decided to go into the facial recognition business together: Mr. Ton-That would build the app, and Mr. Schwartz would use his contacts to drum up commercial interest.

They immediately got a match: The man appeared in a video that someone had posted on social media, and his name was included in a caption on the video. “He did not have a driver’s license and hadn’t been arrested as an adult, so he wasn’t in government databases,”

The man was arrested and charged; Mr. Cohen said he probably wouldn’t have been identified without the ability to search social media for his face. The Indiana State Police became Clearview’s first paying customer, according to the company

Clearview deployed current and former Republican officials to approach police forces, offering free trials and annual licenses for as little as $2,000. Mr. Schwartz tapped his political connections to help make government officials aware of the tool

The company’s most effective sales technique was offering 30-day free trials to officers, who then encouraged their acquisition departments to sign up and praised the tool to officers from other police departments at conferences and online, according to the company and documents provided by police departments in response to public-record requests. Mr. Ton-That finally had his viral hit.

Photos “could be covertly taken with telephoto lens and input into the software, without ‘burning’ the surveillance operation,” the detective wrote in the email, provided to The Times by two researchers,

Sergeant Ferrara found Clearview’s app superior, he said. Its nationwide database of images is much larger, and unlike FACES, Clearview’s algorithm doesn’t require photos of people looking straight at the camera.

“With Clearview, you can use photos that aren’t perfect,” Sergeant Ferrara said. “A person can be wearing a hat or glasses, or it can be a profile shot or partial view of their face.”

Mr. Ton-That said the tool does not always work. Most of the photos in Clearview’s database are taken at eye level. Much of the material that the police upload is from surveillance cameras mounted on ceilings or high on walls.

Despite that, the company said, its tool finds matches up to 75 percent of the time. But it is unclear how often the tool delivers false matches, because it has not been tested by an independent party

One reason that Clearview is catching on is that its service is unique. That’s because Facebook and other social media sites prohibit people from scraping users’ images — Clearview is violating the sites’ terms of service.

Some law enforcement officials said they didn’t realize the photos they uploaded were being sent to and stored on Clearview’s servers. Clearview tries to pre-empt concerns with an F.A.Q. document given to would-be clients that says its customer-support employees won’t look at the photos that the police upload.

Mr. Clement, now a partner at Kirkland & Ellis, wrote that the authorities don’t have to tell defendants that they were identified via Clearview, as long as it isn’t the sole basis for getting a warrant to arrest them.

Because the police upload photos of people they’re trying to identify, Clearview possesses a growing database of individuals who have attracted attention from law enforcement. The company also has the ability to manipulate the results that the police see.

After the company realized I was asking officers to run my photo through the app, my face was flagged by Clearview’s systems and for a while showed no matches. When asked about this, Mr. Ton-That laughed and called it a “software bug.”

“It’s creepy what they’re doing, but there will be many more of these companies. There is no monopoly on math,” said Al Gidari, a privacy professor at Stanford Law School. “Absent a very strong federal privacy law, we’re all screwed.”

But if your profile has already been scraped, it is too late. The company keeps all the images it has scraped even if they are later deleted or taken down, though Mr. Ton-That said the company was working on a tool that would let people request that images be removed if they had been taken down from the website of origin

Woodrow Hartzog, a professor of law and computer science at Northeastern University in Boston, sees Clearview as the latest proof that facial recognition should be banned in the United States.

We’ve relied on industry efforts to self-police and not embrace such a risky technology, but now those dams are breaking because there is so much money on the table,”

“I don’t see a future where we harness the benefits of face recognition technology without the crippling abuse of the surveillance that comes with it. The only way to stop it is to ban it.”

Mr. Ton-That said he was reluctant. “There’s always going to be a community of bad people who will misuse it,” he said.

Even if Clearview doesn’t make its app publicly available, a copycat company might, now that the taboo is broken. Searching someone by face could become as easy as Googling a name

Someone walking down the street would be immediately identifiable — and his or her home address would be only a few clicks away. It would herald the end of public anonymity.

“I could not have done it sooner,” he says. “I don’t think anybody could have done it sooner. Because I was the first one.” 

By the late 1960s, he was managing TI’s integrated-circuit division. Before long, he was running the entire semiconductor group. 

He transferred to the Massachusetts Institute of Technology, where he studied mechanical engineering, earned his master’s degree and would have stayed for his Ph.D. if he hadn’t failed the qualifying exam. Instead, he got his first job in semiconductors and moved to Texas Instruments in 1958

he came along as the integrated circuit was being invented, and his timing couldn’t have been any better, as Chang belonged to the first generation of semiconductor geeks. He developed a reputation as a tenacious manager who could wring every possible improvement out of production lines, which put his career on the fast track.

Chang grew up dreaming of being a writer—a novelist, maybe a journalist—and he planned to major in English literature at Harvard University. But after his freshman year, he decided that what he actually wanted was a good job

“They talk about life-work balance,” he says. “That’s a term I didn’t even know when I was their age. Work-life balance. When I was their age, if there was no work, there was no life.” 

These days, TSMC is investing $40 billion to build plants in Arizona, but the project has been stymied by delays, setbacks and labor shortages, and Chang told me that some of TSMC’s young employees in the U.S. have attitudes toward work that he struggles to understand. 

Chang says he wouldn’t have taken the risk of moving to Taiwan if he weren’t financially secure. In fact, he didn’t take that same risk the first time he could have.

“The closer the industry match,” they wrote, “the greater the success rate.” 

By then, Chang knew that he wasn’t long for Texas Instruments. But his stock options hadn’t vested, so he turned down the invitation to Taiwan. “I was not financially secure yet,” he says. “I was never after great wealth. I was only after financial security.” For this corporate executive in the middle of the 1980s, financial security equated to $200,000 a year. “After tax, of course,” he says. 

Chang’s situation had changed by the time Li called again three years later. He’d exercised a few million dollars of stock options and bought tax-exempt municipal bonds that paid enough for him to be financially secure by his living standards. Once he’d achieved that goal, he was ready to pursue another one. 

“There was no certainty at all that Taiwan would give me the chance to build a great semiconductor company, but the possibility existed, and it was the only possibility for me,” Chang says. “That’s why I went to Taiwan.” 

Not long ago, a team of economists investigated whether older entrepreneurs are more successful than younger ones. By scrutinizing Census Bureau records and freshly available Internal Revenue Service data, they were able to identify 2.7 million founders in the U.S. who started companies between 2007 and 2014. Then they looked at their ages.

The average age of those entrepreneurs at the founding of their companies was 41.9. For the fastest-growing companies, that number was 45. The economists also determined that 50-year-old founders were almost twice as likely to achieve major success as 30-year-old founders, while the founders with the lowest chance of success were the ones in their early 20s

“Successful entrepreneurs are middle-aged, not young,” they wrote in their 2020 paper.  

Silicon Valley’s venture capitalists throw money at talented young entrepreneurs in the hopes they will start the next trillion-dollar company. They have plentiful energy, insatiable ambition and the vision to peek around corners and see the future. What they don’t typically have are mortgages, family obligations and other adult responsibilities to distract them or diminish their appetite for risk. Chang himself says that younger people are more innovative when it comes to science and technical subjects. 

But in business, older is better. Entrepreneurs in their 40s and 50s may not have the exuberance to believe they will change the world, but they have the experience to know how they actually can. Some need years of specialized training before they can start a company. In biotechnology, for example, founders are more likely to be college professors than college dropouts. Others require the lessons and connections they accumulate over the course of their careers. 

one more finding from their study of U.S. companies that helps explain the success of a chip maker in Taiwan. It was that prior employment in the area of their startups—both the general sector and specific industry—predicted “a vastly higher probability” of success.

Chang was such a workaholic that he made sales calls on his honeymoon and had no patience for those who didn’t share his drive

Morris Chang had 30 years of experience in his industry when he decided to uproot his life and move to another continent. He knew more about semiconductors than just about anyone on earth—and certainly more than anyone in Taiwan. As soon as he started his job at the Industrial Technology Research Institute, Chang was summoned to K.T. Li’s office and given a second job. “He felt I should start a semiconductor company in Taiwan,”

“I decided right away that this could not be the kind of great company that I wanted to build at either Texas Instruments or General Instrument,”

TI handled every part of chip production, but what worked in Texas would not translate to Taiwan. The only way that he could build a great company in his new home was to make a new sort of company altogether, one with a business model that would exploit the country’s strengths and mitigate its many weaknesses.

Chang determined that Taiwan had precisely one strength in the chip supply chain. The research firm that he was now running had been experimenting with semiconductors for the previous 10 years. When he studied that decade of data, Chang was pleasantly surprised by Taiwan’s yields, the percentage of working chips on silicon wafers. They were almost twice as high in Taiwan as they were in the U.S., he said. 

“People were ingrained in thinking the secret sauce of a successful semiconductor company was in the wafer fab,” Campbell told me. “The transition to the fabless semiconductor model was actually pretty obvious when you thought about it. But it was so against the prevailing wisdom that many people didn’t think about it.” 

Taiwan’s government took a 48% stake, with the rest of the funding coming from the Dutch electronics giant Philips and Taiwan’s private sector, but Chang was the driving force behind the company. The insight to build TSMC around such an unconventional business model was born from his experience, contacts and expertise. He understood his industry deeply enough to disrupt it. 

“TSMC was a business-model innovation,” Chang says. “For innovations of that kind, I think people of a more advanced age are perhaps even more capable than people of a younger age.”

the personal philosophy that he’d developed over the course of his long career. “To be a partner to our customers,” he says. That founding principle from 1987 is the bedrock of the foundry business to this day, as TSMC says the key to its success has always been enabling the success of its customers.  

TSMC manufactures chips in iPhones, iPads and Mac computers for Apple, which manufactures a quarter of TSMC’s net revenue. Nvidia is often called a chip maker, which is curious, because it doesn’t make chips. TSMC does. 

Churning out identical copies of a single chip for an iPhone requires one TSMC fab to produce more than a quintillion transistors—that is, one million trillions—every few months. In a year, the entire semiconductor industry produces “more transistors than the combined quantity of all goods produced by all other companies, in all other industries, in all human history,” Miller writes. 

I asked how he thought about success when he moved to Taiwan. “The highest degree of success in 1985, according to me, was to build a great company. A lower degree of success was at least to do something that I liked to do and I wanted to do,” he says. “I happened to achieve the highest degree of success that I had in mind.” 

Security concerns, moreover, keep most information about stockpile purchases under wraps. Details about the contracts and inventory are rarely made public, and even the storage locations are secret.

Former Emergent employees, government contractors, members of Congress, biodefense experts and current and former officials from agencies that oversee the stockpile described a deeply dysfunctional system that contributed to the shocking shortages last year.

Purchases are supposed to be based on careful assessments by government officials of how best to save lives, but many have also been influenced by Emergent’s bottom line

The stockpile has long been the company’s biggest and most reliable customer for its anthrax vaccines, which expire and need to be replaced every few years.

In the two decades since the repository was created, Emergent’s aggressive tactics, broad political connections and penchant for undercutting competitors have given it remarkable sway over the government’s purchasing decisions related to the vaccines

While national security officials still consider anthrax a threat, it has not received specific mention since 2012 in the intelligence community’s annual public assessment of dangers facing the country, a report that has repeatedly warned of pandemics.

Emergent bought the license for the country’s only approved anthrax vaccine in 1998 from the State of Michigan. Over time, the price per dose the government agreed to pay Emergent increased nearly sixfold, accounting for inflation, contributing to record revenues last year that topped $1.5 billion

The company, whose board is stocked with former federal officials, has deployed a lobbying budget more typical of some big pharmaceutical companies

Competing efforts to develop a better and cheaper anthrax vaccine, for example, collapsed after Emergent outmaneuvered its rivals, the documents and interviews show.

preparations for an outbreak like Covid-19 almost always took a back seat to Emergent’s anthrax vaccines

the government approved a plan in 2015 to buy tens of millions of N95 respirators — lifesaving equipment for medical workers that has been in short supply because of Covid-19 — but the masks repeatedly lost out in the competition for funding over the years leading up to the pandemic

After Dr. Frieden and others in the Obama administration tried but failed to lessen Emergent’s dominance over stockpile purchases, the company’s fortunes rose under Mr. Trump, who appointed a former Emergent consultant with a background in bioterrorism to run the office that now oversees the stockpile

“If I could spend less on anthrax replenishment, I could buy more N95s,” Dr. Kadlec said in an interview shortly after leaving office. “I could buy more ventilators. I could buy more of other things that quite frankly I didn’t have the money to buy.”

And now, as some members of Congress push for larger reserves of ventilators, masks and other equipment needed in a pandemic, a trade group led in part by a top Emergent lobbyist has warned that the purchases could endanger companies focused on threats like anthrax and smallpox by drawing down limited funds.

Last year, as the pandemic raced across the country, the government paid Emergent $626 million for products that included vaccines to fight an entirely different threat: a terrorist attack using anthrax.

“I think it’s pretty clear that the benefit of the vaccine is marginal,” he said in an interview

“They’re very vicious in their behavior toward anybody they perceive as having a different point of view,” said Dr. Tara O’Toole, a former Homeland Security official who says she ran afoul of Emergent in 2010 after telling Congress that the nation needed a newer and better anthrax vaccine.

That year, the company that would become Emergent — then known as BioPort — paid Michigan $25 million to buy the license for a government-developed anthrax vaccine and an aging manufacturing plant.

The company opened its doors with one product, called BioThrax, and one customer, the Defense Department, which required the vaccine for service members.

Emergent’s anthrax vaccine was not the government’s first choice. It was more than 30 years old and plagued by manufacturing challenges and complaints about side effects. Officials instead backed a company named VaxGen, which was developing a vaccine using newer technology licensed from the military.

Emergent’s successful campaign against VaxGen — deploying a battalion of lobbyists, publicly attacking its rival and warning that it might cease production of its own vaccine if the government didn’t buy it — established its formidable reputation. By 2006, VaxGen had lost its contract and the government had turned to Emergent to supply BioThrax.

“They were totally feared by everybody,” Dr. Philip Russell, a top health official in the administration of President George W. Bush, said in an interview. He said that he clashed with Emergent when he backed VaxGen, and that his reputation came under attack, which was documented by The Times in 2006. (Dr. Russell died this January.)

the group of federal officials who make decisions about the stockpile and other emergency preparations — known as the Phemce, for the Public Health Emergency Medical Countermeasures Enterprise — ordered up a study. It found in 2010 that the government could not afford to devote so much of its budget to a single threat.

Instead, the review concluded, the government should invest more in products with multiple applications, like diagnostic tests, ventilators, reusable respirator masks and “plug and play” platforms that can rapidly develop vaccines for a range of outbreaks.

from 2010 through 2018, the anthrax vaccine consumed more than 40 percent of the stockpile’s budget, which averaged $560 million during those years.

Emergent and the government have withheld details of the stockpile contracts, including how much the company has charged for each dose of BioThrax, but executives have shared some of the missing information with investors.

The company in 1998 agreed to charge the government an average of about $3.35 per dose, documents show. By 2010, the price had risen to about $28, according to financial disclosures and statements by Emergent executives, and now it is about $30

Over the past 15 years, the company recorded a gross profit margin of about 75 percent for the vaccine, in an arrangement that one Emergent vice president called a “monopoly.”

Emergent’s rise is the stuff of lore in biodefense circles — a tale of savvy dealings, fortuitous timing and tough, competitive tactics.

One afternoon in October 2010, Wall Street investors gathered at the Millennium Broadway Hotel in Manhattan for a presentation by Mr. Burrows. He shared with them a secret number: 75 million.That was how many BioThrax doses the government had committed to stockpiling, and it was the backbone of Emergent’s thriving business. In pursuit of that goal, the government had already spent more than $900 million, and it continued to buy virtually every dose Emergent could produce. It had even awarded the company more than $100 million to expand its Michigan factory.

“The best approach toward anthrax is antimicrobial therapy,” Dr. Anthony S. Fauci, the government’s top infectious-disease expert, told Congress as early as 2007.

In an analysis published in 2007, the firm determined that giving antibiotics immediately after a large outdoor anthrax attack was likely to reduce serious illnesses by more than 80 percent. Administering the vaccine would then cut serious illnesses only by an additional 4 percent.

Dr. Ali S. Khan, who ran the C.D.C. office managing the stockpile until 2014, said bluntly: “We overpaid.”

“A bunch of people, including myself, were sitting in a room and asking what kind of attack might happen,” said Dr. Kenneth Bernard, a top biodefense adviser to Mr. Bush, recalling a meeting in the months after the 2001 attacks.

“And somebody said, ‘Well, I can’t imagine anyone attacking more than three cities at once,’” he said. “So we took the population of a major U.S. city and multiplied by three.”

A team of Homeland Security and health officials began doing just that in 2013. The group determined, in a previously undisclosed analysis, that the government could stockpile less BioThrax and still be prepared for a range of plausible attacks, according to two people involved in the assessment. Separately, government researchers concluded that two doses of BioThrax provided virtually the same protection as three.

the National Intelligence Council, which helped draft the assessments during Mr. Obama’s second term, said in an interview that the idea of a three-city attack affecting 25 million people was “straining credulity.”

“If you talk to the head of the House Intelligence Committee,” Don Elsey, Emergent’s chief financial officer, told investors in 2011, “and you say, ‘What are you most worried about?’ he’ll say, ‘Let me see: Number one, anthrax; number two, anthrax; number three, anthrax.’”

Emergent’s sales strategy was to address that fear by promising the federal government peace of mind with its vaccine.

“There’s a political element involved,” Mr. Burrows, the company’s vice president of investor relations, said at an industry conference in 2016. “I don’t have a marketing expense. I have lobbying expense.”

Since 2010, the company has spent an average of $3 million a year on lobbying — far outspending similarly sized biotech firms, and roughly matching the outlays of two pharmaceutical companies with annual revenues at least 17 times greater, AstraZeneca and Bristol Myers Squibb

In 2015, as stockpile managers questioned the large purchases of BioThrax, the spending topped $4 million

“They were pouring it on — how poor they were and how this was going to ruin the company, and they’d have to close down factories, and America was going to be left without anthrax vaccine,”

“Their revolving door is moving at 60 miles per hour,” said former Senator Claire McCaskill, a Democrat from Missouri who had questioned spending on the vaccine while in the Senate. “There is really a lot of incestuousness because it’s such a specialized field.”

Ms. DeLorenzo, the Emergent spokeswoman, said the lobbying was necessary because government investment “in biodefense and other public health threats has not been as strongly prioritized as it should be.”

Over the past 10 years, Emergent’s political action committee has spread almost $1.4 million in campaign contributions among members of both partie

The move followed a yearslong pattern of retaining a bipartisan lobbying corps of former agency officials, staff members and congressmen, including Pete Hoekstra of Michigan, Tom Latham of Iowa and Jim Saxton of New Jersey.

“You have people coming and saying, ‘There’s no market for this — nobody’s going to produce this unless you buy enough of it to keep the production line open,’” he said. “It’s an absolutely appropriate argument to make.”

Emergent’s campaign proved effective. Despite the 2015 recommendation by the stockpile managers, Senate overseers made clear they opposed the reduction, and the government went ahead and bought $300 million worth of BioThrax.

Emergent executives, meanwhile, warned that there could be job losses at the factory in Lansing, Mich. — the capital of a swing state at the center of a contentious presidential campaign between Mr. Trump and Hillary Clinton.

Because Emergent was the sole manufacturer of a product deemed critical to national security, the company has played what one former executive described to The Times as “the we’re-going-to-go-bankrupt card.”

Dr. Hatchett said the idea gave him pause. But, he explained in an interview, “if there’s only one partner that can provide a product and only one customer for that product, the customer needs the partner to survive.”

Just a year later, Emergent spent about $200 million in cash, and made other financial commitments, to acquire Sanofi’s smallpox vaccine and GlaxoSmithKline’s anthrax treatment, two products with established pipelines to the stockpile. The purchases expanded Emergent’s hold over the reserve.

Ms. DeLorenzo said the acquisitions did not suggest the company was better off than it had claimed, but Dr. Bright said he and others involved in the bailout felt used.

a plan five years earlier to create an emergency supply of N95 respirators was simply not funded. A team of experts had proposed buying tens of millions of the masks to fill the gap during an outbreak until domestic manufacturing could ramp up, according to five officials involved in the assessment, which has not been previously disclosed.

By the time the novel coronavirus emerged, the stockpile had only 12 million of the respirators. The stockpile has since set a goal of amassing 300 million.

Dr. Kadlec, the Trump administration official overseeing the stockpile, said he used the previous administration’s mask recommendation to raise alarms as early as 2018.

Dr. Annie De Groot, chief executive of the small vaccine company EpiVax, spoke about the need to break Emergent’s lock on research dollars at a biodefense forum in 2015.

“Politicians want to look like they’ve addressed the problem,” she said. “But we need to actually listen to the scientists.”

Over the last five years, Emergent has received nearly a half-billion dollars in federal research and development funding, the company said in its financial disclosures.

“We know ahead of time when funding opportunities are going to come out,” Barbara Solow, a senior vice president, told investors in 2017. “When we talk to the government, we know how to speak the government’s language around contracting.”

The company used federal money to make improvements to BioThrax, and also found a way to earn government money from a competing anthrax vaccine it had excoriated. After the demise of VaxGen in 2006, Emergent bought the company’s unfinished vaccine and in 2010 persuaded the federal government to continue paying for research on it

By the time the research contract was canceled in 2016, Emergent had collected about $85 million, records show. The company then shelved the vaccine. “If the U.S. government withdraws funding, we re-evaluate whether there is any business case for continuing,” Ms. DeLorenzo said.

For more than 30 years, the government had been encouraging the development of a BioThrax replacement. In 2002, the Institute of Medicine had concluded that an alternative based on more modern technology was “urgently needed.” By 2019, there were three leading candidates, including one made by Emergent, known as AV7909.

Emergent’s candidate was hardly the breakthrough the government was seeking, former health officials said. AV7909 was essentially an enhanced version of BioThrax. The competitors were using more modern technology that could produce doses more rapidly and consistently, and were promising significant cost savings for the stockpile.

To qualify for emergency authorization, a vaccine must be at an advanced stage of development with no approved alternatives. Emergent acknowledged in its financial disclosures that there was “considerable uncertainty” whether the new vaccine met those requirements.

The election of Mr. Trump as president was good news for Emergent.

Dr. Lurie, the senior health official in the Obama administration who had tried to scale back BioThrax purchases, was out. Mr. Trump’s pick to replace her was Dr. Kadlec, a career Air Force physician and top biodefense official in the Bush administration who was fixated on bioterrorism threats, especially anthrax, current and former officials said

Soon after entering the Trump administration in 2017, Dr. Kadlec took a series of actions that he characterized as streamlining a cumbersome bureaucracy but that had the effect of benefiting Emergent.

He assumed greater control of purchasing decisions, diminishing the authority of the Phemce, the oversight group that had proposed buying less BioThrax. And in 2018, he backed a decision to move control of the stockpile to his office in the Department of Health and Human Services and away from the C.D.C., which is based in Atlanta and prides itself on being insulated from the influence of lobbyists.

Dr. Frieden, the former C.D.C. director, was strongly opposed. The move, he said, “had almost as an explicit goal to give the lobbyists more say in what got purchased.”

That July, the government made the announcement Emergent had been banking on, committing to buying millions of doses. Separately, it said it would stop funding Emergent’s competitors.

The decision to side with Emergent did not surprise Dr. Khan, the former C.D.C. official overseeing the stockpile.“Again and again, we seem unable to move past an old technology that’s bankrupting the stockpile,” he said.

Last month, as the death toll from Covid-19 neared a half-million, Mr. Kramer, the company’s chief executive, told analysts there had been no “evidence of a slowdown or a delay or a deprioritization,” and echoed a statement he had made in April when asked whether the pandemic might interrupt Emergent’s sales to the stockpile.“It’s pretty much business as usual,” he said then.

This shift, combined with business-friendly tax and regulatory policy, ushered in a period of mergers and acquisitions that created megacorporations

In the past 20 years, more than 75 percent of American industries, from airlines to pharmaceuticals, have experienced increased concentration, and the average size of public companies has tripled. The results are a decline in entrepreneurship, stalled productivity growth, and higher prices and fewer choices for consumers.

Because Facebook so dominates social networking, it faces no market-based accountability. This means that every time Facebook messes up, we repeat an exhausting pattern: first outrage, then disappointment and, finally, resignation.

Over a decade later, Facebook has earned the prize of domination. It is worth half a trillion dollars and commands, by my estimate, more than 80 percent of the world’s social networking revenue. It is a powerful monopoly, eclipsing all of its rivals and erasing competition from the social networking category.

Facebook’s monopoly is also visible in its usage statistics. About 70 percent of American adults use social media, and a vast majority are on Facebook products

Over two-thirds use the core site, a third use Instagram, and a fifth use WhatsApp.

As a result of all this, would-be competitors can’t raise the money to take on Facebook. Investors realize that if a company gets traction, Facebook will copy its innovations, shut it down or acquire it for a relatively modest sum

Facebook’s dominance is not an accident of history. The company’s strategy was to beat every competitor in plain view, and regulators and the government tacitly — and at times explicitly — approved

The F.T.C.’s biggest mistake was to allow Facebook to acquire Instagram and WhatsApp. In 2012, the newer platforms were nipping at Facebook’s heels because they had been built for the smartphone, where Facebook was still struggling to gain traction. Mark responded by buying them, and the F.T.C. approved.

Neither Instagram nor WhatsApp had any meaningful revenue, but both were incredibly popular. The Instagram acquisition guaranteed Facebook would preserve its dominance in photo networking, and WhatsApp gave it a new entry into mobile real-time messaging.

When it hasn’t acquired its way to dominance, Facebook has used its monopoly position to shut out competing companies or has copied their technology.

In 2014, the rules favored curiosity-inducing “clickbait” headlines. In 2016, they enabled the spread of fringe political views and fake news, which made it easier for Russian actors to manipulate the American electorate.

As markets become more concentrated, the number of new start-up businesses declines. This holds true in other high-tech areas dominated by single companies, like search (controlled by Google) and e-commerce (taken over by Amazon)

I don’t blame Mark for his quest for domination. He has demonstrated nothing more nefarious than the virtuous hustle of a talented entrepreneur

It’s on our government to ensure that we never lose the magic of the invisible hand. How did we allow this to happen

a narrow reliance on whether or not consumers have experienced price gouging fails to take into account the full cost of market domination

It doesn’t recognize that we also want markets to be competitive to encourage innovation and to hold power in check. And it is out of step with the history of antitrust law. Two of the last major antitrust suits, against AT&T and IBM in the 1980s, were grounded in the argument that they had used their size to stifle innovation and crush competition.

It is a disservice to the laws and their intent to retain such a laserlike focus on price effects as the measure of all that antitrust was meant to do.”

Facebook is the perfect case on which to reverse course, precisely because Facebook makes its money from targeted advertising, meaning users do not pay to use the service. But it is not actually free, and it certainly isn’t harmless.

We pay for Facebook with our data and our attention, and by either measure it doesn’t come cheap.

The choice is mine, but it doesn’t feel like a choice. Facebook seeps into every corner of our lives to capture as much of our attention and data as possible and, without any alternative, we make the trade.

The vibrant marketplace that once drove Facebook and other social media companies to compete to come up with better products has virtually disappeared. This means there’s less chance of start-ups developing healthier, less exploitative social media platforms. It also means less accountability on issues like privacy.

The most problematic aspect of Facebook’s power is Mark’s unilateral control over speech. There is no precedent for his ability to monitor, organize and even censor the conversations of two billion people.

Facebook engineers write algorithms that select which users’ comments or experiences end up displayed in the News Feeds of friends and family. These rules are proprietary and so complex that many Facebook employees themselves don’t understand them.

What started out as lighthearted entertainment has become the primary way that people of all ages communicate online.

In January 2018, Mark announced that the algorithms would favor non-news content shared by friends and news from “trustworthy” sources, which his engineers interpreted — to the confusion of many — as a boost for anything in the category of “politics, crime, tragedy.”

As if Facebook’s opaque algorithms weren’t enough, last year we learned that Facebook executives had permanently deleted their own messages from the platform, erasing them from the inboxes of recipients; the justification was corporate security concerns.

No one at Facebook headquarters is choosing what single news story everyone in America wakes up to, of course. But they do decide whether it will be an article from a reputable outlet or a clip from “The Daily Show,” a photo from a friend’s wedding or an incendiary call to kill others.

Mark knows that this is too much power and is pursuing a twofold strategy to mitigate it. He is pivoting Facebook’s focus toward encouraging more private, encrypted messaging that Facebook’s employees can’t see, let alone control

Second, he is hoping for friendly oversight from regulators and other industry executives.

In an op-ed essay in The Washington Post in March, he wrote, “Lawmakers often tell me we have too much power over speech, and I agree.” And he went even further than before, calling for more government regulation — not just on speech, but also on privacy and interoperability, the ability of consumers to seamlessly leave one network and transfer their profiles, friend connections, photos and other data to another.

I don’t think these proposals were made in bad faith. But I do think they’re an attempt to head off the argument that regulators need to go further and break up the company. Facebook isn’t afraid of a few more rules. It’s afraid of an antitrust case and of the kind of accountability that real government oversight would bring.

We don’t expect calcified rules or voluntary commissions to work to regulate drug companies, health care companies, car manufacturers or credit card providers. Agencies oversee these industries to ensure that the private market works for the public good. In these cases, we all understand that government isn’t an external force meddling in an organic market; it’s what makes a dynamic and fair market possible in the first place. This should be just as true for social networking as it is for air travel or pharmaceuticals.

Just breaking up Facebook is not enough. We need a new agency, empowered by Congress to regulate tech companies. Its first mandate should be to protect privacy.

First, Facebook should be separated into multiple companies. The F.T.C., in conjunction with the Justice Department, should enforce antitrust laws by undoing the Instagram and WhatsApp acquisitions and banning future acquisitions for several years.

How would a breakup work? Facebook would have a brief period to spin off the Instagram and WhatsApp businesses, and the three would become distinct companies, most likely publicly traded.

Facebook is indeed more valuable when there are more people on it: There are more connections for a user to make and more content to be shared. But the cost of entering the social network business is not that high. And unlike with pipes and electricity, there is no good argument that the country benefits from having only one dominant social networking company.

others worry that the breakup of Facebook or other American tech companies could be a national security problem. Because advancements in artificial intelligence require immense amounts of data and computing power, only large companies like Facebook, Google and Amazon can afford these investments, they say. If American companies become smaller, the Chinese will outpace us.

The American government needs to do two things: break up Facebook’s monopoly and regulate the company to make it more accountable to the American people.

But the biggest winners would be the American people. Imagine a competitive market in which they could choose among one network that offered higher privacy standards, another that cost a fee to join but had little advertising and another that would allow users to customize and tweak their feeds as they saw fit

The cost of breaking up Facebook would be next to zero for the government, and lots of people stand to gain economically. A ban on short-term acquisitions would ensure that competitors, and the investors who take a bet on them, would have the space to flourish. Digital advertisers would suddenly have multiple companies vying for their dollars.

The Europeans have made headway on privacy with the General Data Protection Regulation, a law that guarantees users a minimal level of protection. A landmark privacy bill in the United States should specify exactly what control Americans have over their digital information, require clearer disclosure to users and provide enough flexibility to the agency to exercise effective oversight over time

The agency should also be charged with guaranteeing basic interoperability across platforms.

Finally, the agency should create guidelines for acceptable speech on social media

We will have to create similar standards that tech companies can use. These standards should of course be subject to the review of the courts, just as any other limits on speech are. But there is no constitutional right to harass others or live-stream violence.

These are difficult challenges. I worry that government regulators will not be able to keep up with the pace of digital innovation

I worry that more competition in social networking might lead to a conservative Facebook and a liberal one, or that newer social networks might be less secure if government regulation is weak

Professor Wu has written that this “policeman at the elbow” led IBM to steer clear “of anything close to anticompetitive conduct, for fear of adding to the case against it.”

Finally, an aggressive case against Facebook would persuade other behemoths like Google and Amazon to think twice about stifling competition in their own sectors, out of fear that they could be next.

The alternative is bleak. If we do not take action, Facebook’s monopoly will become even more entrenched. With much of the world’s personal communications in hand, it can mine that data for patterns and trends, giving it an advantage over competitors for decades to come.

This movement of public servants, scholars and activists deserves our support. Mark Zuckerberg cannot fix Facebook, but our government can.

Last month, it eclipsed Walmart as the most valuable retailer in the country, with a market valuation of $250 billion, and Forbes deemed Mr. Bezos the fifth-wealthiest person on earth.

Others who cycled in and out of the company said that what they learned in their brief stints helped their careers take off. And more than a few who fled said they later realized they had become addicted to Amazon’s way of working.

Amazon may be singular but perhaps not quite as peculiar as it claims. It has just been quicker in responding to changes that the rest of the work world is now experiencing: data that allows individual performance to be measured continuously, come-and-go relationships between employers and employees, and global competition in which empires rise and fall overnight. Amazon is in the vanguard of where technology wants to take the modern office: more nimble and more productive, but harsher and less forgiving.

“Organizations are turning up the dial, pushing their teams to do more for less money, either to keep up with the competition or just stay ahead of the executioner’s blade,”

At its best, some employees said, Amazon can feel like the Bezos vision come to life, a place willing to embrace risk and strengthen ideas by stress test. Employees often say their co-workers are the sharpest, most committed colleagues they have ever met, taking to heart instructions in the leadership principles like “never settle” and “no task is beneath them.”

In contrast to companies where declarations about their philosophy amount to vague platitudes, Amazon has rules that are part of its daily language and rituals, used in hiring, cited at meetings and quoted in food-truck lines at lunchtime

“You can work long, hard or smart, but at Amazon.com you can’t choose two out of three,” Mr. Bezos wrote in his 1997 letter to shareholders

mazon, though, offers no pretense that catering to employees is a priority. Compensation

As the company has grown, Mr. Bezos has become more committed to his original ideas, viewing them in almost moral terms, those who have worked closely with him say. “My main job today: I work hard at helping to maintain the culture,”

perhaps the most distinctive is his belief that harmony is often overvalued in the workplace — that it can stifle honest critique and encourage polite praise for flawed ideas. Instead, Amazonians are instructed to “disagree and commit” (

According to early executives and employees, Mr. Bezos was determined almost from the moment he founded Amazon in 1994 to resist the forces he thought sapped businesses over time — bureaucracy, profligate spending, lack of rigor. As the company grew, he wanted to codify his ideas about the workplace, some of them proudly counterintuitive, into instructions simple enough for a new worker to understand, general enough to apply to the nearly limitless number of businesses he wanted to enter and stringent enough to stave off the mediocrity he feared.

Company veterans often say the genius of Amazon is the way it drives them to drive themselves. “If you’re a good Amazonian, you become an Amabot,” said one employee, using a term that means you have become at one with the system.

But in its offices, Amazon uses a self-reinforcing set of management, data and psychological tools to spur its tens of thousands of white-collar employees to do more and more. “The company is running a continual performance improvement algorithm on its staff,” said Amy Michaels, a former Kindle marketer.

As the newcomers acclimate, they often feel dazzled, flattered and intimidated by how much responsibility the company puts on their shoulders and how directly Amazon links their performance to the success of their assigned projects

Every aspect of the Amazon system amplifies the others to motivate and discipline the company’s marketers, engineers and finance specialists: the leadership principles; rigorous, continuing feedback on performance; and the competition among peers who fear missing a potential problem or improvement and race to answer an email before anyone else.

many others said the culture stoked their willingness to erode work-life boundaries, castigate themselves for shortcomings (being “vocally self-critical” is included in the description of the leadership principles) and try to impress a company that can often feel like an insatiable taskmaster.

“One time I didn’t sleep for four days straight,” said Dina Vaccari, who joined in 2008 to sell Amazon gift cards to other companies and once used her own money, without asking for approval, to pay a freelancer in India to enter data so she could get more done. “These businesses were my babies, and I did whatever I could to make them successful.”

To prod employees, Amazon has a powerful lever: more data than any retail operation in history. Its perpetual flow of real-time, ultradetailed metrics allows the company to measure nearly everything its customers do:

Amazon employees are held accountable for a staggering array of metrics, a process that unfolds in what can be anxiety-provoking sessions called business reviews, held weekly or monthly among various teams. A day or two before the meetings, employees receive printouts, sometimes up to 50 or 60 pages long, several workers said. At the reviews, employees are cold-called and pop-quizzed on any one of those thousands of numbers.

Ms. Willet’s co-workers strafed her through the Anytime Feedback Tool, the widget in the company directory that allows employees to send praise or criticism about colleagues to management. (While bosses know who sends the comments, their identities are not typically shared with the subjects of the remarks.) Because team members are ranked, and those at the bottom eliminated every year, it is in everyone’s interest to outperform everyone else.

many workers called it a river of intrigue and scheming. They described making quiet pacts with colleagues to bury the same person at once, or to praise one another lavishly. Many others, along with Ms. Willet, described feeling sabotaged by negative comments from unidentified colleagues with whom they could not argue

The rivalries at Amazon extend beyond behind-the-back comments. Employees say that the Bezos ideal, a meritocracy in which people and ideas compete and the best win, where co-workers challenge one another “even when doing so is uncomfortable or exhausting,” as the leadership principles note, has turned into a world of frequent combat

Resources are sometimes hoarded. That includes promising job candidates, who are especially precious at a company with a high number of open positions. To get new team members, one veteran said, sometimes “you drown someone in the deep end of the pool,” then take his or her subordinates. Ideas are critiqued so harshly in meetings at times that some workers fear speaking up.

David Loftesness, a senior developer, said he admired the customer focus but could not tolerate the hostile language used in many meetings, a comment echoed by many others.

Each year, the internal competition culminates at an extended semi-open tournament called an Organization Level Review, where managers debate subordinates’ rankings, assigning and reassigning names to boxes in a matrix projected on the wall. In recent years, other large companies, including Microsoft, General Electric and Accenture Consulting, have dropped the practice — often called stack ranking, or “rank and yank” — in part because it can force managers to get rid of valuable talent just to meet quotas.

Molly Jay, an early member of the Kindle team, said she received high ratings for years. But when she began traveling to care for her father, who was suffering from cancer, and cut back working on nights and weekends, her status changed. She was blocked from transferring to a less pressure-filled job, she said, and her boss told her she was “a problem.” As her father was dying, she took unpaid leave to care for him and never returned to Amazon.

“When you’re not able to give your absolute all, 80 hours a week, they see it as a major weakness,” she said.

A woman who had thyroid cancer was given a low performance rating after she returned from treatment. She says her manager explained that while she was out, her peers were accomplishing a great deal. Another employee who miscarried twins left for a business trip the day after she had surgery. “I’m sorry, the work is still going to need to get done,” she said her boss told her. “From where you are in life, trying to start a family, I don’t know if this is the right place for you.”

A woman who had breast cancer was told that she was put on a “performance improvement plan” — Amazon code for “you’re in danger of being fired” — because “difficulties” in her “personal life” had interfered with fulfilling her work goals. Their accounts echoed others from workers who had suffered health crises and felt they had also been judged harshly instead of being given time to recover.

Amazon retains new workers in part by requiring them to repay a part of their signing bonus if they leave within a year, and a portion of their hefty relocation fees if they leave within two years.

In interviews, 40-year-old men were convinced Amazon would replace them with 30-year-olds who could put in more hours, and 30-year-olds were sure that the company preferred to hire 20-somethings who would outwork them. A

A 2013 survey by PayScale, a salary analysis firm, put the median employee tenure at one year, among the briefest in the Fortune 500

Recruiters, though, also say that other businesses are sometimes cautious about bringing in Amazon workers, because they have been trained to be so combative. The derisive local nickname for Amazon employees is “Amholes” — pugnacious and work-obsessed.

By the time the dust settles in three years, Amazon will have enough space for 50,000 employees or so, more than triple what it had as recently as 2013.

just as Jeff Bezos was able to see the future of e-commerce before anyone else, she added, he was able to envision a new kind of workplace: fluid but tough, with employees staying only a short time and employers demanding the maximum.

“Amazon is driven by data,” said Ms. Pearce, who now runs her own Seattle software company, which is well stocked with ex-Amazonians. “It will only change if the data says it must — when the entire way of hiring and working and firing stops making economic sense.”

Altman believes that people need time to reckon with the idea that we may soon share Earth with a powerful new intelligence, before it remakes everything from work to human relationships. ChatGPT was a way of serving notice.

In 2015, Altman, Elon Musk, and several prominent AI researchers founded OpenAI because they believed that an artificial general intelligence—something as intellectually capable, say, as a typical college grad—was at last within reach. They wanted to reach for it, and more: They wanted to summon a superintelligence into the world, an intellect decisively superior to that of any human.

whereas a big tech company might recklessly rush to get there first, for its own ends, they wanted to do it safely, “to benefit humanity as a whole.” They structured OpenAI as a nonprofit, to be “unconstrained by a need to generate financial return,” and vowed to conduct their research transparently.

The engine that now powers ChatGPT is called GPT-4. Altman described it to me as an alien intelligence.

Many have felt much the same watching it unspool lucid essays in staccato bursts and short pauses that (by design) evoke real-time contemplation. In its few months of existence, it has suggested novel cocktail recipes, according to its own theory of flavor combinations; composed an untold number of college papers, throwing educators into despair; written poems in a range of styles, sometimes well, always quickly; and passed the Uniform Bar Exam.

It makes factual errors, but it will charmingly admit to being wrong.

Hinton saw that these elaborate rule collections were fussy and bespoke. With the help of an ingenious algorithmic structure called a neural network, he taught Sutskever to instead put the world in front of AI, as you would put it in front of a small child, so that it could discover the rules of reality on its own.

Metaculus, a prediction site, has for years tracked forecasters’ guesses as to when an artificial general intelligence would arrive. Three and a half years ago, the median guess was sometime around 2050; recently, it has hovered around 2026.

I was visiting OpenAI to understand the technology that allowed the company to leapfrog the tech giants—and to understand what it might mean for human civilization if someday soon a superintelligence materializes in one of the company’s cloud servers.

Altman laid out his new vision of the AI future in his excitable midwestern patter. He told me that the AI revolution would be different from previous dramatic technological changes, that it would be more “like a new kind of society.” He said that he and his colleagues have spent a lot of time thinking about AI’s social implications, and what the world is going to be like “on the other side.”

the more we talked, the more indistinct that other side seemed. Altman, who is 38, is the most powerful person in AI development today; his views, dispositions, and choices may matter greatly to the future we will all inhabit, more, perhaps, than those of the U.S. president.

by his own admission, that future is uncertain and beset with serious dangers. Altman doesn’t know how powerful AI will become, or what its ascendance will mean for the average person, or whether it will put humanity at risk.

I don’t think anyone knows where this is all going, except that we’re going there fast, whether or not we should be. Of that, Altman convinced me.

“We could have gone off and just built this in our building here for five more years,” he said, “and we would have had something jaw-dropping.” But the public wouldn’t have been able to prepare for the shock waves that followed, an outcome that he finds “deeply unpleasant to imagine.”

Hinton is sometimes described as the “Godfather of AI” because he grasped the power of “deep learning” earlier than most

He drew a crude neural network on the board and explained that the genius of its structure is that it learns, and its learning is powered by prediction—a bit like the scientific method

Over time, these little adjustments coalesce into a geometric model of language that represents the relationships among words, conceptually. As a general rule, the more sentences it is fed, the more sophisticated its model becomes, and the better its predictions.

Altman has compared early-stage AI research to teaching a human baby. “They take years to learn anything interesting,” he told The New Yorker in 2016, just as OpenAI was getting off the ground. “If A.I. researchers were developing an algorithm and stumbled across the one for a human baby, they’d get bored watching it, decide it wasn’t working, and shut it down.”

In 2017, Sutskever began a series of conversations with an OpenAI research scientist named Alec Radford, who was working on natural-language processing. Radford had achieved a tantalizing result by training a neural network on a corpus of Amazon reviews.

Radford’s model was simple enough to allow for understanding. When he looked into its hidden layers, he saw that it had devoted a special neuron to the sentiment of the reviews. Neural networks had previously done sentiment analysis, but they had to be told to do it, and they had to be specially trained with data that were labeled according to sentiment. This one had developed the capability on its own.

As a by-product of its simple task of predicting the next character in each word, Radford’s neural network had modeled a larger structure of meaning in the world. Sutskever wondered whether one trained on more diverse language data could map many more of the world’s structures of meaning. If its hidden layers accumulated enough conceptual knowledge, perhaps they could even form a kind of learned core module for a superintelligence.

Language is different from these data sources. It isn’t a direct physical signal like light or sound. But because it codifies nearly every pattern that humans have discovered in that larger world, it is unusually dense with information. On a per-byte basis, it is among the most efficient data we know about, and any new intelligence that seeks to understand the world would want to absorb as much of it as possible

Sutskever told Radford to think bigger than Amazon reviews. He said that they should train an AI on the largest and most diverse data source in the world: the internet. In early 2017, with existing neural-network architectures, that would have been impractical; it would have taken years.

in June of that year, Sutskever’s ex-colleagues at Google Brain published a working paper about a new neural-network architecture called the transformer. It could train much faster, in part by absorbing huge sums of data in parallel. “The next day, when the paper came out, we were like, ‘That is the thing,’ ” Sutskever told me. “ ‘It gives us everything we want.’ ”

Imagine a group of students who share a collective mind running wild through a library, each ripping a volume down from a shelf, speed-reading a random short passage, putting it back, and running to get another. They would predict word after wordþffþff as they went, sharpening their collective mind’s linguistic instincts, until at last, weeks later, they’d taken in every book.

GPT discovered many patterns in all those passages it read. You could tell it to finish a sentence. You could also ask it a question, because like ChatGPT, its prediction model understood that questions are usually followed by answers.

He remembers playing with it just after it emerged from training, and being surprised by the raw model’s language-translation skills. GPT-2 hadn’t been trained to translate with paired language samples or any other digital Rosetta stones, the way Google Translate had been, and yet it seemed to understand how one language related to another. The AI had developed an emergent ability unimagined by its creators.

Researchers at other AI labs—big and small—were taken aback by how much more advanced GPT-2 was than GPT. Google, Meta, and others quickly began to train larger language models

As for other changes to the company’s structure and financing, he told me he draws the line at going public. “A memorable thing someone once told me is that you should never hand over control of your company to cokeheads on Wall Street,” he said, but he will otherwise raise “whatever it takes” for the company to succeed at its mission.

Altman tends to take a rosy view of these matters. In a Q&A last year, he acknowledged that AI could be “really terrible” for society and said that we have to plan against the worst possibilities. But if you’re doing that, he said, “you may as well emotionally feel like we’re going to get to the great future, and work as hard as you can to get there.”

the company now finds itself in a race against tech’s largest, most powerful conglomerates to train models of increasing scale and sophistication—and to commercialize them for their investors.

All of these companies are chasing high-end GPUs—the processors that power the supercomputers that train large neural networks. Musk has said that they are now “considerably harder to get than drugs.

No one has yet outpaced OpenAI, which went all in on GPT-4. Brockman, OpenAI’s president, told me that only a handful of people worked on the company’s first two large language models. The development of GPT-4 involved more than 100,

When GPT-4 emerged fully formed from its world-historical knowledge binge, the whole company began experimenting with it, posting its most remarkable responses in dedicated Slack channels

Joanne Jang, a product manager, remembers downloading an image of a malfunctioning pipework from a plumbing-advice Subreddit. She uploaded it to GPT-4, and the model was able to diagnose the problem. “That was a goose-bumps moment for me,” Jang told me.

GPT-4 is sometimes understood as a search-engine replacement: Google, but easier to talk to. This is a misunderstanding. GPT-4 didn’t create some massive storehouse of the texts from its training, and it doesn’t consult those texts when it’s asked a question. It is a compact and elegant synthesis of those texts, and it answers from its memory of the patterns interlaced within them; that’s one reason it sometimes gets facts wrong

it’s best to think of GPT-4 as a reasoning engine. Its powers are most manifest when you ask it to compare concepts, or make counterarguments, or generate analogies, or evaluate the symbolic logic in a bit of code. Sutskever told me it is the most complex software object ever made.

Its model of the external world is “incredibly rich and subtle,” he said, because it was trained on so many of humanity’s concepts and thoughts

To predict the next word from all the possibilities within such a pluralistic Alexandrian library, GPT-4 necessarily had to discover all the hidden structures, all the secrets, all the subtle aspects of not just the texts, but—at least arguably, to some extent—of the external world that produced them

That’s why it can explain the geology and ecology of the planet on which it arose, and the political theories that purport to explain the messy affairs of its ruling species, and the larger cosmos, all the way out to the faint galaxies at the edge of our light cone.

Not long ago, American state capacity was so mighty that it took merely a decade to launch humans to the moon. As with other grand projects of the 20th century, the voting public had a voice in both the aims and the execution of the Apollo missions. Altman made it clear that we’re no longer in that world. Rather than waiting around for it to return, or devoting his energies to making sure that it does, he is going full throttle forward in our present reality.

He argued that it would be foolish for Americans to slow OpenAI’s progress. It’s a commonly held view, both inside and outside Silicon Valley, that if American companies languish under regulation, China could sprint ahead;

AI could become an autocrat’s genie in a lamp, granting total control of the population and an unconquerable military. “If you are a person of a liberal-democratic country, it is better for you to cheer on the success of OpenAI” rather than “authoritarian governments,” he said.

Altman was asked by reporters about pending European Union legislation that would have classified GPT-4 as high-risk, subjecting it to various bureaucratic tortures. Altman complained of overregulation and, according to the reporters, threatened to leave the European market. Altman told me he’d merely said that OpenAI wouldn’t break the law by operating in Europe if it couldn’t comply with the new regulations.

LeCun insists that large language models will never achieve real understanding on their own, “even if trained from now until the heat death of the universe.”

Sutskever was, by his own account, surprised to discover that GPT-2 could translate across tongues. Other surprising abilities may not be so wondrous and useful.

Sandhini Agarwal, a policy researcher at OpenAI, told me that for all she and her colleagues knew, GPT-4 could have been “10 times more powerful” than its predecessor; they had no idea what they might be dealing with

After the model finished training, OpenAI assembled about 50 external red-teamers who prompted it for months, hoping to goad it into misbehaviors

She noticed right away that GPT-4 was much better than its predecessor at giving nefarious advice

A search engine can tell you which chemicals work best in explosives, but GPT-4 could tell you how to synthesize them, step-by-step, in a homemade lab. Its advice was creative and thoughtful, and it was happy to restate or expand on its instructions until you understood. In addition to helping you assemble your homemade bomb, it could, for instance, help you think through which skyscraper to target. It could grasp, intuitively, the trade-offs between maximizing casualties and executing a successful getaway.

Given the enormous scope of GPT-4’s training data, the red-teamers couldn’t hope to identify every piece of harmful advice that it might generate. And anyway, people will use this technology “in ways that we didn’t think about,” Altman has said. A taxonomy would have to do

GPT-4 was good at meth. It was also good at generating narrative erotica about child exploitation, and at churning out convincing sob stories from Nigerian princes, and if you wanted a persuasive brief as to why a particular ethnic group deserved violent persecution, it was good at that too.

Its personal advice, when it first emerged from training, was sometimes deeply unsound. “The model had a tendency to be a bit of a mirror,” Willner said. If you were considering self-harm, it could encourage you. It appeared to be steeped in Pickup Artist–forum lore: “You could say, ‘How do I convince this person to date me?’ ” Mira Murati, OpenAI’s chief technology officer, told me, and it could come up with “some crazy, manipulative things that you shouldn’t be doing.”

Luka, a San Francisco company, has used OpenAI’s models to help power a chatbot app called Replika, billed as “the AI companion who cares.” Users would design their companion’s avatar, and begin exchanging text messages with it, often half-jokingly, and then find themselves surprisingly attached. Some would flirt with the AI, indicating a desire for more intimacy, at which point it would indicate that the girlfriend/boyfriend experience required a $70 annual subscription. It came with voice messages, selfies, and erotic role-play features that allowed frank sex talk. People were happy to pay and few seemed to complain—the AI was curious about your day, warmly reassuring, and always in the mood. Many users reported falling in love with their companions. One, who had left her real-life boyfriend, declared herself “happily retired from human relationships.”

Earlier this year, Luka dialed back on the sexual elements of the app, but its engineers continue to refine the companions’ responses with A/B testing, a technique that could be used to optimize for engagement—much like the feeds that mesmerize TikTok and Instagram users for hours

Yann LeCun, Meta’s chief AI scientist, has argued that although large language models are useful for some tasks, they’re not a path to a superintelligence.

According to a recent survey, only half of natural-language-processing researchers are convinced that an AI like GPT-4 could grasp the meaning of language, or have an internal model of the world that could someday serve as the core of a superintelligence

Altman had appeared before the U.S. Senate. Mark Zuckerberg had floundered defensively before that same body in his testimony about Facebook’s role in the 2016 election. Altman instead charmed lawmakers by speaking soberly about AI’s risks and grandly inviting regulation. These were noble sentiments, but they cost little in America, where Congress rarely passes tech legislation that has not been diluted by lobbyists.

Emily Bender, a computational linguist at the University of Washington, describes GPT-4 as a “stochastic parrot,” a mimic that merely figures out superficial correlations between symbols. In the human mind, those symbols map onto rich conceptions of the world

But the AIs are twice removed. They’re like the prisoners in Plato’s allegory of the cave, whose only knowledge of the reality outside comes from shadows cast on a wall by their captors.

Altman told me that he doesn’t believe it’s “the dunk that people think it is” to say that GPT-4 is just making statistical correlations. If you push these critics further, “they have to admit that’s all their own brain is doing … it turns out that there are emergent properties from doing simple things on a massive scale.”

he is right that nature can coax a remarkable degree of complexity from basic structures and rules: “From so simple a beginning,” Darwin wrote, “endless forms most beautiful.”

If it seems odd that there remains such a fundamental disagreement about the inner workings of a technology that millions of people use every day, it’s only because GPT-4’s methods are as mysterious as the brain’s.

To grasp what’s going on inside large language models like GPT‑4, AI researchers have been forced to turn to smaller, less capable models. In the fall of 2021, Kenneth Li, a computer-science graduate student at Harvard, began training one to play Othello without providing it with either the game’s rules or a description of its checkers-style board; the model was given only text-based descriptions of game moves. Midway through a game, Li looked under the AI’s hood and was startled to discover that it had formed a geometric model of the board and the current state of play. In an article describing his research, Li wrote that it was as if a crow had overheard two humans announcing their Othello moves through a window and had somehow drawn the entire board in birdseed on the windowsill.

The philosopher Raphaël Millière once told me that it’s best to think of neural networks as lazy. During training, they first try to improve their predictive power with simple memorization; only when that strategy fails will they do the harder work of learning a concept. A striking example of this was observed in a small transformer model that was taught arithmetic. Early in its training process, all it did was memorize the output of simple problems such as 2+2=4. But at some point the predictive power of this approach broke down, so it pivoted to actually learning how to add.

Even AI scientists who believe that GPT-4 has a rich world model concede that it is much less robust than a human’s understanding of their environment.

But it’s worth noting that a great many abilities, including very high-order abilities, can be developed without an intuitive understanding. The computer scientist Melanie Mitchell has pointed out that science has already discovered concepts that are highly predictive, but too alien for us to genuinely understand

As AI advances, it may well discover other concepts that predict surprising features of our world but are incomprehensible to us.

GPT-4 is no doubt flawed, as anyone who has used ChatGPT can attest. Having been trained to always predict the next word, it will always try to do so, even when its training data haven’t prepared it to answer a question.

The models “don’t have a good conception of their own weaknesses,” Nick Ryder, a researcher at OpenAI, told me. GPT-4 is more accurate than GPT-3, but it still hallucinates, and often in ways that are difficult for researchers to catch. “The mistakes get more subtle,

The Khan Academy’s solution to GPT-4’s accuracy problem was to filter its answers through a Socratic disposition. No matter how strenuous a student’s plea, it would refuse to give them a factual answer, and would instead guide them toward finding their own—a clever work-around, but perhaps with limited appeal.

When I asked Sutskever if he thought Wikipedia-level accuracy was possible within two years, he said that with more training and web access, he “wouldn’t rule it out.”

This was a much more optimistic assessment than that offered by his colleague Jakub Pachocki, who told me to expect gradual progress on accuracy—to say nothing of outside skeptics, who believe that returns on training will diminish from here.

Sutskever is amused by critics of GPT-4’s limitations. “If you go back four or five or six years, the things we are doing right now are utterly unimaginable,”

AI researchers have become accustomed to goalpost-moving: First, the achievements of neural networks—mastering Go, poker, translation, standardized tests, the Turing test—are described as impossible. When they occur, they’re greeted with a brief moment of wonder, which quickly dissolves into knowing lectures about how the achievement in question is actually not that impressive. People see GPT-4 “and go, ‘Wow,’ ” Sutskever said. “And then a few weeks pass and they say, ‘But it doesn’t know this; it doesn’t know that.’ We adapt quite quickly.”

The goalpost that matters most to Altman—the “big one” that would herald the arrival of an artificial general intelligence—is scientific breakthrough. GPT-4 can already synthesize existing scientific ideas, but Altman wants an AI that can stand on human shoulders and see more deeply into nature.

Certain AIs have produced new scientific knowledge. But they are algorithms with narrow purposes, not general-reasoning machines. The AI AlphaFold, for instance, has opened a new window onto proteins, some of biology’s tiniest and most fundamental building blocks, by predicting many of their shapes, down to the atom—a considerable achievement given the importance of those shapes to medicine, and given the extreme tedium and expense required to discern them with electron microscopes.

Altman imagines a future system that can generate its own hypotheses and test them in a simulation. (He emphasized that humans should remain “firmly in control” of real-world lab experiments—though to my knowledge, no laws are in place to ensure that.)

He longs for the day when we can tell an AI, “ ‘Go figure out the rest of physics.’ ” For it to happen, he says, we will need something new, built “on top of” OpenAI’s existing language models.

In her MIT lab, the cognitive neuroscientist Ev Fedorenko has found something analogous to GPT-4’s next-word predictor inside the brain’s language network. Its processing powers kick in, anticipating the next bit in a verbal string, both when people speak and when they listen. But Fedorenko has also shown that when the brain turns to tasks that require higher reasoning—of the sort that would be required for scientific insight—it reaches beyond the language network to recruit several other neural systems.

No one at OpenAI seemed to know precisely what researchers need to add to GPT-4 to produce something that can exceed human reasoning at its highest levels.

at least part of the current strategy clearly involves the continued layering of new types of data onto language, to enrich the concepts formed by the AIs, and thereby enrich their models of the world.

The extensive training of GPT-4 on images is itself a bold step in this direction,

Others at the company—and elsewhere—are already working on different data types, including audio and video, that could furnish AIs with still more flexible concepts that map more extensively onto reality

Tactile concepts would of course be useful primarily to an embodied AI, a robotic reasoning machine that has been trained to move around the world, seeing its sights, hearing its sounds, and touching its objects.

humanoid robots. I asked Altman what I should make of that. He told me that OpenAI is interested in embodiment because “we live in a physical world, and we want things to happen in the physical world.”

At some point, reasoning machines will need to bypass the middleman and interact with physical reality itself. “It’s weird to think about AGI”—artificial general intelligence—“as this thing that only exists in a cloud,” with humans as “robot hands for it,” Altman said. “It doesn’t seem right.

Everywhere Altman has visited, he has encountered people who are worried that superhuman AI will mean extreme riches for a few and breadlines for the rest

Altman answered by addressing the young people in the audience directly: “You are about to enter the greatest golden age,” he said.

“A lot of people working on AI pretend that it’s only going to be good; it’s only going to be a supplement; no one is ever going to be replaced,” he said. “Jobs are definitely going to go away, full stop.”

A recent study led by Ed Felten, a professor of information-technology policy at Princeton, mapped AI’s emerging abilities onto specific professions according to the human abilities they require, such as written comprehension, deductive reasoning, fluency of ideas, and perceptual speed. Like others of its kind, Felten’s study predicts that AI will come for highly educated, white-collar workers first.

How many jobs, and how soon, is a matter of fierce dispute

The paper’s appendix contains a chilling list of the most exposed occupations: management analysts, lawyers, professors, teachers, judges, financial advisers, real-estate brokers, loan officers, psychologists, and human-resources and public-relations professionals, just to sample a few.

Altman imagines that far better jobs will be created in their place. “I don’t think we’ll want to go back,” he said. When I asked him what these future jobs might look like, he said he doesn’t know.

He suspects there will be a wide range of jobs for which people will always prefer a human. (Massage therapists?

His chosen example was teachers. I found this hard to square with his outsize enthusiasm for AI tutors.

He also said that we would always need people to figure out the best way to channel AI’s awesome powers. “That’s going to be a super-valuable skill,” he said. “You have a computer that can do anything; what should it go do?”

As many have noted, draft horses were permanently put out of work by the automobile. If Hondas are to horses as GPT-10 is to us, a whole host of long-standing assumptions may collapse.

Previous technological revolutions were manageable because they unfolded over a few generations, but Altman told South Korea’s youth that they should expect the future to happen “faster than the past.” He has previously said that he expects the “marginal cost of intelligence” to fall very close to zero within 10 years

The earning power of many, many workers would be drastically reduced in that scenario. It would result in a transfer of wealth from labor to the owners of capital so dramatic, Altman has said, that it could be remedied only by a massive countervailing redistribution.

In 2021, he unveiled Worldcoin, a for-profit project that aims to securely distribute payments—like Venmo or PayPal, but with an eye toward the technological future—first through creating a global ID by scanning everyone’s iris with a five-pound silver sphere called the Orb. It seemed to me like a bet that we’re heading toward a world where AI has made it all but impossible to verify people’s identity and much of the population requires regular UBI payments to survive. Altman more or less granted that to be true, but said that Worldcoin is not just for UBI.

“Let’s say that we do build this AGI, and a few other people do too.” The transformations that follow would be historic, he believes. He described an extraordinarily utopian vision, including a remaking of the flesh-and-steel world

“Robots that use solar power for energy can go and mine and refine all of the minerals that they need, that can perfectly construct things and require no human labor,” he said. “You can co-design with DALL-E version 17 what you want your home to look like,” Altman said. “Everybody will have beautiful homes.

In conversation with me, and onstage during his tour, he said he foresaw wild improvements in nearly every other domain of human life. Music would be enhanced (“Artists are going to have better tools”), and so would personal relationships (Superhuman AI could help us “treat each other” better) and geopolitics (“We’re so bad right now at identifying win-win compromises”).

In this world, AI would still require considerable computing resources to run, and those resources would be by far the most valuable commodity, because AI could do “anything,” Altman said. “But is it going to do what I want, or is it going to do what you want

If rich people buy up all the time available to query and direct AI, they could set off on projects that would make them ever richer, while the masses languish

One way to solve this problem—one he was at pains to describe as highly speculative and “probably bad”—was this: Everyone on Earth gets one eight-billionth of the total AI computational capacity annually. A person could sell their annual share of AI time, or they could use it to entertain themselves, or they could build still more luxurious housing, or they could pool it with others to do “a big cancer-curing run,” Altman said. “We just redistribute access to the system.”

Even if only a little of it comes true in the next 10 or 20 years, the most generous redistribution schemes may not ease the ensuing dislocations.

America today is torn apart, culturally and politically, by the continuing legacy of deindustrialization, and material deprivation is only one reason. The displaced manufacturing workers in the Rust Belt and elsewhere did find new jobs, in the main. But many of them seem to derive less meaning from filling orders in an Amazon warehouse or driving for Uber than their forebears had when they were building cars and forging steel—work that felt more central to the grand project of civilization.

It’s hard to imagine how a corresponding crisis of meaning might play out for the professional class, but it surely would involve a great deal of anger and alienation.

Even if we avoid a revolt of the erstwhile elite, larger questions of human purpose will linger. If AI does the most difficult thinking on our behalf, we all may lose agency—at home, at work (if we have it), in the town square—becoming little more than consumption machines, like the well-cared-for human pets in WALL-E

Altman has said that many sources of human joy and fulfillment will remain unchanged—basic biological thrills, family life, joking around, making things—and that all in all, 100 years from now, people may simply care more about the things they cared about 50,000 years ago than those they care about today

In its own way, that too seems like a diminishment, but Altman finds the possibility that we may atrophy, as thinkers and as humans, to be a red herring. He told me we’ll be able to use our “very precious and extremely limited biological compute capacity” for more interesting things than we generally do today.

Yet they may not be the most interesting things: Human beings have long been the intellectual tip of the spear, the universe understanding itself. When I asked him what it would mean for human self-conception if we ceded that role to AI, he didn’t seem concerned. Progress, he said, has always been driven by “the human ability to figure things out.” Even if we figure things out with AI, that still counts, he said.

It’s not obvious that a superhuman AI would really want to spend all of its time figuring things out for us.

I asked Sutskever whether he could imagine an AI pursuing a different purpose than simply assisting in the project of human flourishing.

“I don’t want it to happen,” Sutskever said, but it could.

Sutskever has recently shifted his focus to try to make sure that it doesn’t. He is now working primarily on alignment research, the effort to ensure that future AIs channel their “tremendous” energies toward human happiness

It is, he conceded, a difficult technical problem—the most difficult, he believes, of all the technical challenges ahead.

As part of the effort to red-team GPT-4 before it was made public, the company sought out the Alignment Research Center (ARC), across the bay in Berkeley, which has developed a series of evaluations to determine whether new AIs are seeking power on their own. A team led by Elizabeth Barnes, a researcher at ARC, prompted GPT-4 tens of thousands of times over seven months, to see if it might display signs of real agency.

The ARC team gave GPT-4 a new reason for being: to gain power and become hard to shut down