Group items matching

in title, tags, annotations or url

The things you do with your life are mostly chosen: your job, spouse and hobbies.

At our foundation, we were a society with strong covenantal attachments — to family, community, creed and faith. Then on top of them we built democracy and capitalism that celebrated liberty and individual rights.

The deep covenantal institutions gave people the capacity to use their freedom well. The liberal institutions gave them that freedom.

This delicate balance — liberal institutions built atop illiberal ones — is now giving way. The big social movements of the past half century were about maximizing freedom of choice. Right-wingers wanted to maximize economic choice and left-wingers lifestyle choice. Anything that smacked of restraint came to seem like a bad thing to be eliminated.

We’ll call this worldview — which is all freedom and no covenant — naked liberalism

The problem with naked liberalism is that it relies on individuals it cannot create.

Naked liberals of right and left assume that if you give people freedom they will use it to care for their neighbors, to have civil conversations, to form opinions after examining the evidence. But if you weaken family, faith, community and any sense of national obligation, where is that social, emotional and moral formation supposed to come from? How will the virtuous habits form?

Naked liberalism has made our society an unsteady tree. The branches of individual rights are sprawling, but the roots of common obligation are withering away.

Freedom without covenant becomes selfishness. And that’s what we see at the top of society, in our politics and the financial crisis

First, they will identify themselves according to race. They will become the racial essentialists you see on left and right

And that’s what we see at the bottom of society — frayed communities, broken families, opiate addiction.

Freedom without a unifying national narrative becomes distrust, polarization and permanent political war.

covenantal attachments they become fragile. Moreover, if you rob people of their good covenantal attachments, they will grab bad ones.

Freedom without connection becomes alienation.

Then they resort to tribalism. This is what Donald Trump provides. As Mark S. Weiner writes on the Niskanen Center’s blog, Trump is constantly making friend/enemy distinctions, exploiting liberalism’s thin conception of community and creating toxic communities based on in-group/out-group rivalry.

Trump offers people cultural solutions to their alienation problem. As history clearly demonstrates, people will prefer fascism to isolation, authoritarianism to moral anarchy.

If we are going to have a decent society we’re going to have to save liberalism from itself. We’re going to have to restore and re-enchant the covenantal relationships that are the foundation for the whole deal. The crucial battleground is cultural and prepolitical.

In my experience, most people under 40 get this. They sense the social and moral void at the core and that change has to come at the communal, emotional and moral level.

Many public intellectuals were trained in the social sciences and take the choosing individual as their mental starting point. They have trouble thinking about our shared social and moral formative institutions and how such institutions could be reconstituted.

Congressional Republicans think a successful tax bill will thwart populism. Mainstream Democrats think the alienation problem will go away if we redistribute the crumbs a bit more widely.

History is full of examples of nations that built new national narratives, revived family life, restored community bonds and shared moral culture: Britain in the early 19th century, Germany after World War II, America in the Progressive Era. The first step in launching our own revival is understanding that the problem is down in the roots.

It's thought that all or part of Zealandia may have existed as an island for a while. But then around 25 million years ago, it was disappeared beneath the ocean

The first real clues that New Zealand might represent just a tiny, visible portion of a vast incognito landmass came in 2002, when scientists used bathymetry – the study of the depth of bodies of water – to analyse the region. The ocean above what we now call Zealandia is considerably shallower than that surrounding it, suggesting that the area was not underlain by an oceanic tectonic plate – like most of the world's oceans – but a continental one.

The clincher came in 2017, when scientists put several lines of evidence together, including data about the kinds of rocks it contains and its relative thickness – oceanic plates tend to be thinner – to propose that this is indeed a new continent. This is not a mere continental fragment or microcontinent, as had previously been proposed, but the real deal, 95% of which is submerged underwater.

despite the excitement around the discovery of a new continent, and more than a decade of intensive research, many details of Zealandia's early formation have remained elusive. This is partly because of a strange event that occurred when it split from Gondwana.

In 2019, an international team of scientists mapped the geology of South Zealandia. Their research revealed that at some point, Zealandia had been stretched – pulled apart by tectonic forces, thinning the continent compared to regular continental plates and creating ruptures that later became oceanic crust. In the process, it became twisted and this made reconstructing its history to work out its original form much more challenging. 

The researchers' analysis of rocks from the lost continent revealed that the stretching happened in two stages. The first began around 89-101 million years ago, and led to a rip which became the Tasman Sea inbetween Australia and New Zealand. The second phase started 80-90 million years ago and led to Zealandia splitting off from West Antarctica and creating the Pacific Ocea

For the latest study, another research group – involving many of the same geologists as before – charted North Zealandia. This time, they analysed rocks that had been dredged up from the Fairway Ridge, a region of the South Pacific off the coast of Australia, which forms the northernmost tip of Zealandia. These ancient remnants, which have not had a dry day for 25 million years, included a mixture of igneous rocks – those formed by volcanic processes – and sedimentary ones made in shallow basins just off the coast of Zealandia.

The resulting maps of Zealandia transform it from a featureless mass into a place with many bands of distinctive geology running along its length from northwest to southeast. These fit together with the geology of West Antarctica like a jigsaw puzzle, confirming that this region and Zealandia once slotted together.

Peter Thiel has lost interest in democracy.

Thiel has cultivated an image as a man of ideas, an intellectual who studied philosophy with René Girard and owns first editions of Leo Strauss in English and German. Trump quite obviously did not share these interests, or Thiel’s libertarian principles.

For years, Thiel had been saying that he generally favored the more pessimistic candidate in any presidential race because “if you’re too optimistic, it just shows you’re out of touch.” He scorned the rote optimism of politicians who, echoing Ronald Reagan, portrayed America as a shining city on a hill. Trump’s America, by contrast, was a broken landscape, under siege.

Thiel is not against government in principle, his friend Auren Hoffman (who is no relation to Reid) says. “The ’30s, ’40s, and ’50s—which had massive, crazy amounts of power—he admires because it was effective. We built the Hoover Dam. We did the Manhattan Project,” Hoffman told me. “We started the space program.”

Their failure to make the world conform to his vision has soured him on the entire enterprise—to the point where he no longer thinks it matters very much who wins the next election.

His libertarian critique of American government has curdled into an almost nihilistic impulse to demolish it.

“Voting for Trump was like a not very articulate scream for help,” Thiel told me. He fantasized that Trump’s election would somehow force a national reckoning. He believed somebody needed to tear things down—slash regulations, crush the administrative state—before the country could rebuild.

He admits now that it was a bad bet.

“There are a lot of things I got wrong,” he said. “It was crazier than I thought. It was more dangerous than I thought. They couldn’t get the most basic pieces of the government to work. So that was—I think that part was maybe worse than even my low expectations.”

eid Hoffman, who has known Thiel since college, long ago noticed a pattern in his old friend’s way of thinking. Time after time, Thiel would espouse grandiose, utopian hopes that failed to materialize, leaving him “kind of furious or angry” about the world’s unwillingness to bend to whatever vision was possessing him at the moment

Thiel. He is worth between $4 billion and $9 billion. He lives with his husband and two children in a glass palace in Bel Air that has nine bedrooms and a 90-foot infinity pool. He is a titan of Silicon Valley and a conservative kingmaker.

“Peter tends to be not ‘glass is half empty’ but ‘glass is fully empty,’” Hoffman told me.

he tells the story of his life as a series of disheartening setbacks.

He met Mark Zuckerberg, liked what he heard, and became Facebook’s first outside investor. Half a million dollars bought him 10 percent of the company, most of which he cashed out for about $1 billion in 2012.

Thiel made some poor investments, losing enormous sums by going long on the stock market in 2008, when it nose-dived, and then shorting the market in 2009, when it rallied

on the whole, he has done exceptionally well. Alex Karp, his Palantir co-founder, who agrees with Thiel on very little other than business, calls him “the world’s best venture investor.”

Thiel told me this is indeed his ambition, and he hinted that he may have achieved it.

He longs for radical new technologies and scientific advances on a scale most of us can hardly imagine

He longs for a world in which great men are free to work their will on society, unconstrained by government or regulation or “redistributionist economics” that would impinge on their wealth and power—or any obligation, really, to the rest of humanity

Did his dream of eternal life trace to The Lord of the Rings?

He takes for granted that this kind of progress will redound to the benefit of society at large.

More than anything, he longs to live forever.

Calling death a law of nature is, in his view, just an excuse for giving up. “It’s something we are told that demotivates us from trying harder,”

Thiel grew up reading a great deal of science fiction and fantasy—Heinlein, Asimov, Clarke. But especially Tolkien; he has said that he read the Lord of the Rings trilogy at least 10 times. Tolkien’s influence on his worldview is obvious: Middle-earth is an arena of struggle for ultimate power, largely without government, where extraordinary individuals rise to fulfill their destinies. Also, there are immortal elves who live apart from men in a magical sheltered valley.

But his dreams have always been much, much bigger than that.

Yes, Thiel said, perking up. “There are all these ways where trying to live unnaturally long goes haywire” in Tolkien’s works. But you also have the elves.

How are the elves different from the humans in Tolkien? And they’re basically—I think the main difference is just, they’re humans that don’t die.”

During college, he co-founded The Stanford Review, gleefully throwing bombs at identity politics and the university’s diversity-minded reform of the curriculum. He co-wrote The Diversity Myth in 1995, a treatise against what he recently called the “craziness and silliness and stupidity and wickedness” of the left.

Thiel laid out a plan, for himself and others, “to find an escape from politics in all its forms.” He wanted to create new spaces for personal freedom that governments could not reach

But something changed for Thiel in 2009

he people, he concluded, could not be trusted with important decisions. “I no longer believe that freedom and democracy are compatible,” he wrote.

ven more notable one followed: “Since 1920, the vast increase in welfare beneficiaries and the extension of the franchise to women—two constituencies that are notoriously tough for libertarians—have rendered the notion of ‘capitalist democracy’ into an oxymoron.”

By 2015, six years after declaring his intent to change the world from the private sector, Thiel began having second thoughts. He cut off funding for the Seasteading Institute—years of talk had yielded no practical progress–and turned to other forms of escape

The fate of our world may depend on the effort of a single person who builds or propagates the machinery of freedom,” he wrote. His manifesto has since become legendary in Silicon Valley, where his worldview is shared by other powerful men (and men hoping to be Peter Thiel).

Thiel’s investment in cryptocurrencies, like his founding vision at PayPal, aimed to foster a new kind of money “free from all government control and dilution

His decision to rescue Elon Musk’s struggling SpaceX in 2008—with a $20 million infusion that kept the company alive after three botched rocket launches—came with aspirations to promote space as an open frontier with “limitless possibility for escape from world politics

It was seasteading that became Thiel’s great philanthropic cause in the late aughts and early 2010s. The idea was to create autonomous microstates on platforms in international waters.

“There’s zero chance Peter Thiel would live on Sealand,” he said, noting that Thiel likes his comforts too much. (Thiel has mansions around the world and a private jet. Seal performed at his 2017 wedding, at the Belvedere Museum in Vienna.)

As he built his companies and grew rich, he began pouring money into political causes and candidates—libertarian groups such as the Endorse Liberty super PAC, in addition to a wide range of conservative Republicans, including Senators Orrin Hatch and Ted Cruz

Sam Altman, the former venture capitalist and now CEO of OpenAI, revealed in 2016 that in the event of global catastrophe, he and Thiel planned to wait it out in Thiel’s New Zealand hideaway.

When I asked Thiel about that scenario, he seemed embarrassed and deflected the question. He did not remember the arrangement as Altman did, he said. “Even framing it that way, though, makes it sound so ridiculous,” he told me. “If there is a real end of the world, there is no place to go.”

You’d have eco farming. You’d turn the deserts into arable land. There were sort of all these incredible things that people thought would happen in the ’50s and ’60s and they would sort of transform the world.”

None of that came to pass. Even science fiction turned hopeless—nowadays, you get nothing but dystopias

He hungered for advances in the world of atoms, not the world of bits.

Founders Fund, the venture-capital firm he established in 200

The fund, therefore, would invest in smart people solving hard problems “that really have the potential to change the world.”

This was not what Thiel wanted to be doing with his time. Bodegas and dog food were making him money, apparently, but he had set out to invest in transformational technology that would advance the state of human civilization.

He told me that he no longer dwells on democracy’s flaws, because he believes we Americans don’t have one. “We are not a democracy; we’re a republic,” he said. “We’re not even a republic; we’re a constitutional republic.”

“It was harder than it looked,” Thiel said. “I’m not actually involved in enough companies that are growing a lot, that are taking our civilization to the next level.”

Founders Fund has holdings in artificial intelligence, biotech, space exploration, and other cutting-edge fields. What bothers Thiel is that his companies are not taking enough big swings at big problems, or that they are striking out.

In at least 20 hours of logged face-to-face meetings with Buma, Thiel reported on what he believed to be a Chinese effort to take over a large venture-capital firm, discussed Russian involvement in Silicon Valley, and suggested that Jeffrey Epstein—a man he had met several times—was an Israeli intelligence operative. (Thiel told me he thinks Epstein “was probably entangled with Israeli military intelligence” but was more involved with “the U.S. deep state.”)

Buma, according to a source who has seen his reports, once asked Thiel why some of the extremely rich seemed so open to contacts with foreign governments. “And he said that they’re bored,” this source said. “‘They’re bored.’ And I actually believe it. I think it’s that simple. I think they’re just bored billionaires.”

he has a sculpture that resembles a three-dimensional game board. Ascent: Above the Nation State Board Game Display Prototype is the New Zealander artist Simon Denny’s attempt to map Thiel’s ideological universe. The board features a landscape in the aesthetic of Dungeons & Dragons, thick with monsters and knights and castles. The monsters include an ogre labeled “Monetary Policy.” Near the center is a hero figure, recognizable as Thiel. He tilts against a lion and a dragon, holding a shield and longbow. The lion is labeled “Fair Elections.” The dragon is labeled “Democracy.” The Thiel figure is trying to kill them.

When I asked Thiel to explain his views on democracy, he dodged the question. “I always wonder whether people like you … use the word democracy when you like the results people have and use the word populism when you don’t like the results,” he told me. “If I’m characterized as more pro-populist than the elitist Atlantic is, then, in that sense, I’m more pro-democratic.”

“I couldn’t find them,” he said. “I couldn’t get enough of them to work.

He said he has no wish to change the American form of government, and then amended himself: “Or, you know, I don’t think it’s realistic for it to be radically changed.” Which is not at all the same thing.

When I asked what he thinks of Yarvin’s autocratic agenda, Thiel offered objections that sounded not so much principled as practical.

“I don’t think it’s going to work. I think it will look like Xi in China or Putin in Russia,” Thiel said, meaning a malign dictatorship. “It ultimately I don’t think will even be accelerationist on the science and technology side, to say nothing of what it will do for individual rights, civil liberties, things of that sort.”

Still, Thiel considers Yarvin an “interesting and powerful” historian

he always talks about is the New Deal and FDR in the 1930s and 1940s,” Thiel said. “And the heterodox take is that it was sort of a light form of fascism in the United States.”

Yarvin, Thiel said, argues that “you should embrace this sort of light form of fascism, and we should have a president who’s like FDR again.”

Did Thiel agree with Yarvin’s vision of fascism as a desirable governing model? Again, he dodged the question.

“That’s not a realistic political program,” he said, refusing to be drawn any further.

ooking back on Trump’s years in office, Thiel walked a careful line.

A number of things were said and done that Thiel did not approve of. Mistakes were made. But Thiel was not going to refashion himself a Never Trumper in retrospect.

“I have to somehow give the exact right answer, where it’s like, ‘Yeah, I’m somewhat disenchanted,’” he told me. “But throwing him totally under the bus? That’s like, you know—I’ll get yelled at by Mr. Trump. And if I don’t throw him under the bus, that’s—but—somehow, I have to get the tone exactly right.”

Thiel knew, because he had read some of my previous work, that I think Trump’s gravest offense against the republic was his attempt to overthrow the election. I asked how he thought about it.

“Look, I don’t think the election was stolen,” he said. But then he tried to turn the discussion to past elections that might have been wrongly decided. Bush-Gore in 2000, for instanc

He came back to Trump’s attempt to prevent the transfer of power. “I’ll agree with you that it was not helpful,” he said.

there is another piece of the story, which Thiel reluctantly agreed to discuss

Puck reported that Democratic operatives had been digging for dirt on Thiel since before the 2022 midterm elections, conducting opposition research into his personal life with the express purpose of driving him out of politic

Among other things, the operatives are said to have interviewed a young model named Jeff Thomas, who told them he was having an affair with Thiel, and encouraged Thomas to talk to Ryan Grim, a reporter for The Intercept. Grim did not publish a story during election season, as the opposition researchers hoped he would, but he wrote about Thiel’s affair in March, after Thomas died by suicide.

He deplored the dirt-digging operation, telling me in an email that “the nihilism afflicting American politics is even deeper than I knew.”

He also seemed bewildered by the passions he arouses on the left. “I don’t think they should hate me this much,”

he spoke at the closed-press event with a lot less nuance than he had in our interviews. His after-dinner remarks were full of easy applause lines and in-jokes mocking the left. Universities had become intellectual wastelands, obsessed with a meaningless quest for diversity, he told the crowd. The humanities writ large are “transparently ridiculous,” said the onetime philosophy major, and “there’s no real science going on” in the sciences, which have devolved into “the enforcement of very curious dogmas.”

“Diversity—it’s not enough to just hire the extras from the space-cantina scene in Star Wars,” he said, prompting laughter.

Nor did Thiel say what genuine diversity would mean. The quest for it, he said, is “very evil and it’s very silly.”

“the silliness is distracting us from very important things,” such as the threat to U.S. interests posed by the Chinese Communist Party.

“Whenever someone says ‘DEI,’” he exhorted the crowd, “just think ‘CCP.’”

Somebody asked, in the Q&A portion of the evening, whether Thiel thought the woke left was deliberately advancing Chinese Communist interests

“It’s always the difference between an agent and asset,” he said. “And an agent is someone who is working for the enemy in full mens rea. An asset is a useful idiot. So even if you ask the question ‘Is Bill Gates China’s top agent, or top asset, in the U.S.?’”—here the crowd started roaring—“does it really make a difference?”

About 10 years ago, Thiel told me, a fellow venture capitalist called to broach the question. Vinod Khosla, a co-founder of Sun Microsystems, had made the Giving Pledge a couple of years before. Would Thiel be willing to talk with Gates about doing the same?

Thiel feels that giving his billions away would be too much like admitting he had done something wrong to acquire them

He also lacked sympathy for the impulse to spread resources from the privileged to those in need. When I mentioned the terrible poverty and inequality around the world, he said, “I think there are enough people working on that.”

besides, a different cause moves him far more.

Should Thiel happen to die one day, best efforts notwithstanding, his arrangements with Alcor provide that a cryonics team will be standing by.

Then his body will be cooled to –196 degrees Celsius, the temperature of liquid nitrogen. After slipping into a double-walled, vacuum-insulated metal coffin, alongside (so far) 222 other corpsicles, “the patient is now protected from deterioration for theoretically thousands of years,” Alcor literature explains.

All that will be left for Thiel to do, entombed in this vault, is await the emergence of some future society that has the wherewithal and inclination to revive him. And then make his way in a world in which his skills and education and fabulous wealth may be worth nothing at all.

I wondered how much Thiel had thought through the implications for society of extreme longevity. The population would grow exponentially. Resources would not. Where would everyone live? What would they do for work? What would they eat and drink? Or—let’s face it—would a thousand-year life span be limited to men and women of extreme wealth?

“Well, I maybe self-serve,” he said, perhaps understating the point, “but I worry more about stagnation than about inequality.”

Thiel is not alone among his Silicon Valley peers in his obsession with immortality. Oracle’s Larry Ellison has described mortality as “incomprehensible.” Google’s Sergey Brin aspires to “cure death.” Dmitry Itskov, a leading tech entrepreneur in Russia, has said he hopes to live to 10,000.

. “I should be investing way more money into this stuff,” he told me. “I should be spending way more time on this.”

You haven’t told your husband? Wouldn’t you want him to sign up alongside you?“I mean, I will think about that,” he said, sounding rattled. “I will think—I have not thought about that.”

No matter how fervent his desire, Thiel’s extraordinary resources still can’t buy him the kind of “super-duper medical treatments” that would let him slip the grasp of death. It is, perhaps, his ultimate disappointment.

There are all these things I can’t do with my money,” Thiel said.

Altman believes that people need time to reckon with the idea that we may soon share Earth with a powerful new intelligence, before it remakes everything from work to human relationships. ChatGPT was a way of serving notice.

In 2015, Altman, Elon Musk, and several prominent AI researchers founded OpenAI because they believed that an artificial general intelligence—something as intellectually capable, say, as a typical college grad—was at last within reach. They wanted to reach for it, and more: They wanted to summon a superintelligence into the world, an intellect decisively superior to that of any human.

whereas a big tech company might recklessly rush to get there first, for its own ends, they wanted to do it safely, “to benefit humanity as a whole.” They structured OpenAI as a nonprofit, to be “unconstrained by a need to generate financial return,” and vowed to conduct their research transparently.

The engine that now powers ChatGPT is called GPT-4. Altman described it to me as an alien intelligence.

Many have felt much the same watching it unspool lucid essays in staccato bursts and short pauses that (by design) evoke real-time contemplation. In its few months of existence, it has suggested novel cocktail recipes, according to its own theory of flavor combinations; composed an untold number of college papers, throwing educators into despair; written poems in a range of styles, sometimes well, always quickly; and passed the Uniform Bar Exam.

It makes factual errors, but it will charmingly admit to being wrong.

Hinton saw that these elaborate rule collections were fussy and bespoke. With the help of an ingenious algorithmic structure called a neural network, he taught Sutskever to instead put the world in front of AI, as you would put it in front of a small child, so that it could discover the rules of reality on its own.

Metaculus, a prediction site, has for years tracked forecasters’ guesses as to when an artificial general intelligence would arrive. Three and a half years ago, the median guess was sometime around 2050; recently, it has hovered around 2026.

I was visiting OpenAI to understand the technology that allowed the company to leapfrog the tech giants—and to understand what it might mean for human civilization if someday soon a superintelligence materializes in one of the company’s cloud servers.

Altman laid out his new vision of the AI future in his excitable midwestern patter. He told me that the AI revolution would be different from previous dramatic technological changes, that it would be more “like a new kind of society.” He said that he and his colleagues have spent a lot of time thinking about AI’s social implications, and what the world is going to be like “on the other side.”

the more we talked, the more indistinct that other side seemed. Altman, who is 38, is the most powerful person in AI development today; his views, dispositions, and choices may matter greatly to the future we will all inhabit, more, perhaps, than those of the U.S. president.

by his own admission, that future is uncertain and beset with serious dangers. Altman doesn’t know how powerful AI will become, or what its ascendance will mean for the average person, or whether it will put humanity at risk.

I don’t think anyone knows where this is all going, except that we’re going there fast, whether or not we should be. Of that, Altman convinced me.

“We could have gone off and just built this in our building here for five more years,” he said, “and we would have had something jaw-dropping.” But the public wouldn’t have been able to prepare for the shock waves that followed, an outcome that he finds “deeply unpleasant to imagine.”

Hinton is sometimes described as the “Godfather of AI” because he grasped the power of “deep learning” earlier than most

He drew a crude neural network on the board and explained that the genius of its structure is that it learns, and its learning is powered by prediction—a bit like the scientific method

Over time, these little adjustments coalesce into a geometric model of language that represents the relationships among words, conceptually. As a general rule, the more sentences it is fed, the more sophisticated its model becomes, and the better its predictions.

Altman has compared early-stage AI research to teaching a human baby. “They take years to learn anything interesting,” he told The New Yorker in 2016, just as OpenAI was getting off the ground. “If A.I. researchers were developing an algorithm and stumbled across the one for a human baby, they’d get bored watching it, decide it wasn’t working, and shut it down.”

In 2017, Sutskever began a series of conversations with an OpenAI research scientist named Alec Radford, who was working on natural-language processing. Radford had achieved a tantalizing result by training a neural network on a corpus of Amazon reviews.

Radford’s model was simple enough to allow for understanding. When he looked into its hidden layers, he saw that it had devoted a special neuron to the sentiment of the reviews. Neural networks had previously done sentiment analysis, but they had to be told to do it, and they had to be specially trained with data that were labeled according to sentiment. This one had developed the capability on its own.

As a by-product of its simple task of predicting the next character in each word, Radford’s neural network had modeled a larger structure of meaning in the world. Sutskever wondered whether one trained on more diverse language data could map many more of the world’s structures of meaning. If its hidden layers accumulated enough conceptual knowledge, perhaps they could even form a kind of learned core module for a superintelligence.

Language is different from these data sources. It isn’t a direct physical signal like light or sound. But because it codifies nearly every pattern that humans have discovered in that larger world, it is unusually dense with information. On a per-byte basis, it is among the most efficient data we know about, and any new intelligence that seeks to understand the world would want to absorb as much of it as possible

Sutskever told Radford to think bigger than Amazon reviews. He said that they should train an AI on the largest and most diverse data source in the world: the internet. In early 2017, with existing neural-network architectures, that would have been impractical; it would have taken years.

in June of that year, Sutskever’s ex-colleagues at Google Brain published a working paper about a new neural-network architecture called the transformer. It could train much faster, in part by absorbing huge sums of data in parallel. “The next day, when the paper came out, we were like, ‘That is the thing,’ ” Sutskever told me. “ ‘It gives us everything we want.’ ”

Imagine a group of students who share a collective mind running wild through a library, each ripping a volume down from a shelf, speed-reading a random short passage, putting it back, and running to get another. They would predict word after wordþffþff as they went, sharpening their collective mind’s linguistic instincts, until at last, weeks later, they’d taken in every book.

GPT discovered many patterns in all those passages it read. You could tell it to finish a sentence. You could also ask it a question, because like ChatGPT, its prediction model understood that questions are usually followed by answers.

He remembers playing with it just after it emerged from training, and being surprised by the raw model’s language-translation skills. GPT-2 hadn’t been trained to translate with paired language samples or any other digital Rosetta stones, the way Google Translate had been, and yet it seemed to understand how one language related to another. The AI had developed an emergent ability unimagined by its creators.

Researchers at other AI labs—big and small—were taken aback by how much more advanced GPT-2 was than GPT. Google, Meta, and others quickly began to train larger language models

As for other changes to the company’s structure and financing, he told me he draws the line at going public. “A memorable thing someone once told me is that you should never hand over control of your company to cokeheads on Wall Street,” he said, but he will otherwise raise “whatever it takes” for the company to succeed at its mission.

Altman tends to take a rosy view of these matters. In a Q&A last year, he acknowledged that AI could be “really terrible” for society and said that we have to plan against the worst possibilities. But if you’re doing that, he said, “you may as well emotionally feel like we’re going to get to the great future, and work as hard as you can to get there.”

the company now finds itself in a race against tech’s largest, most powerful conglomerates to train models of increasing scale and sophistication—and to commercialize them for their investors.

All of these companies are chasing high-end GPUs—the processors that power the supercomputers that train large neural networks. Musk has said that they are now “considerably harder to get than drugs.

No one has yet outpaced OpenAI, which went all in on GPT-4. Brockman, OpenAI’s president, told me that only a handful of people worked on the company’s first two large language models. The development of GPT-4 involved more than 100,

When GPT-4 emerged fully formed from its world-historical knowledge binge, the whole company began experimenting with it, posting its most remarkable responses in dedicated Slack channels

Joanne Jang, a product manager, remembers downloading an image of a malfunctioning pipework from a plumbing-advice Subreddit. She uploaded it to GPT-4, and the model was able to diagnose the problem. “That was a goose-bumps moment for me,” Jang told me.

GPT-4 is sometimes understood as a search-engine replacement: Google, but easier to talk to. This is a misunderstanding. GPT-4 didn’t create some massive storehouse of the texts from its training, and it doesn’t consult those texts when it’s asked a question. It is a compact and elegant synthesis of those texts, and it answers from its memory of the patterns interlaced within them; that’s one reason it sometimes gets facts wrong

it’s best to think of GPT-4 as a reasoning engine. Its powers are most manifest when you ask it to compare concepts, or make counterarguments, or generate analogies, or evaluate the symbolic logic in a bit of code. Sutskever told me it is the most complex software object ever made.

Its model of the external world is “incredibly rich and subtle,” he said, because it was trained on so many of humanity’s concepts and thoughts

To predict the next word from all the possibilities within such a pluralistic Alexandrian library, GPT-4 necessarily had to discover all the hidden structures, all the secrets, all the subtle aspects of not just the texts, but—at least arguably, to some extent—of the external world that produced them

That’s why it can explain the geology and ecology of the planet on which it arose, and the political theories that purport to explain the messy affairs of its ruling species, and the larger cosmos, all the way out to the faint galaxies at the edge of our light cone.

Not long ago, American state capacity was so mighty that it took merely a decade to launch humans to the moon. As with other grand projects of the 20th century, the voting public had a voice in both the aims and the execution of the Apollo missions. Altman made it clear that we’re no longer in that world. Rather than waiting around for it to return, or devoting his energies to making sure that it does, he is going full throttle forward in our present reality.

He argued that it would be foolish for Americans to slow OpenAI’s progress. It’s a commonly held view, both inside and outside Silicon Valley, that if American companies languish under regulation, China could sprint ahead;

AI could become an autocrat’s genie in a lamp, granting total control of the population and an unconquerable military. “If you are a person of a liberal-democratic country, it is better for you to cheer on the success of OpenAI” rather than “authoritarian governments,” he said.

Altman was asked by reporters about pending European Union legislation that would have classified GPT-4 as high-risk, subjecting it to various bureaucratic tortures. Altman complained of overregulation and, according to the reporters, threatened to leave the European market. Altman told me he’d merely said that OpenAI wouldn’t break the law by operating in Europe if it couldn’t comply with the new regulations.

LeCun insists that large language models will never achieve real understanding on their own, “even if trained from now until the heat death of the universe.”

Sutskever was, by his own account, surprised to discover that GPT-2 could translate across tongues. Other surprising abilities may not be so wondrous and useful.

Sandhini Agarwal, a policy researcher at OpenAI, told me that for all she and her colleagues knew, GPT-4 could have been “10 times more powerful” than its predecessor; they had no idea what they might be dealing with

After the model finished training, OpenAI assembled about 50 external red-teamers who prompted it for months, hoping to goad it into misbehaviors

She noticed right away that GPT-4 was much better than its predecessor at giving nefarious advice

A search engine can tell you which chemicals work best in explosives, but GPT-4 could tell you how to synthesize them, step-by-step, in a homemade lab. Its advice was creative and thoughtful, and it was happy to restate or expand on its instructions until you understood. In addition to helping you assemble your homemade bomb, it could, for instance, help you think through which skyscraper to target. It could grasp, intuitively, the trade-offs between maximizing casualties and executing a successful getaway.

Given the enormous scope of GPT-4’s training data, the red-teamers couldn’t hope to identify every piece of harmful advice that it might generate. And anyway, people will use this technology “in ways that we didn’t think about,” Altman has said. A taxonomy would have to do

GPT-4 was good at meth. It was also good at generating narrative erotica about child exploitation, and at churning out convincing sob stories from Nigerian princes, and if you wanted a persuasive brief as to why a particular ethnic group deserved violent persecution, it was good at that too.

Its personal advice, when it first emerged from training, was sometimes deeply unsound. “The model had a tendency to be a bit of a mirror,” Willner said. If you were considering self-harm, it could encourage you. It appeared to be steeped in Pickup Artist–forum lore: “You could say, ‘How do I convince this person to date me?’ ” Mira Murati, OpenAI’s chief technology officer, told me, and it could come up with “some crazy, manipulative things that you shouldn’t be doing.”

Luka, a San Francisco company, has used OpenAI’s models to help power a chatbot app called Replika, billed as “the AI companion who cares.” Users would design their companion’s avatar, and begin exchanging text messages with it, often half-jokingly, and then find themselves surprisingly attached. Some would flirt with the AI, indicating a desire for more intimacy, at which point it would indicate that the girlfriend/boyfriend experience required a $70 annual subscription. It came with voice messages, selfies, and erotic role-play features that allowed frank sex talk. People were happy to pay and few seemed to complain—the AI was curious about your day, warmly reassuring, and always in the mood. Many users reported falling in love with their companions. One, who had left her real-life boyfriend, declared herself “happily retired from human relationships.”

Earlier this year, Luka dialed back on the sexual elements of the app, but its engineers continue to refine the companions’ responses with A/B testing, a technique that could be used to optimize for engagement—much like the feeds that mesmerize TikTok and Instagram users for hours

Yann LeCun, Meta’s chief AI scientist, has argued that although large language models are useful for some tasks, they’re not a path to a superintelligence.

According to a recent survey, only half of natural-language-processing researchers are convinced that an AI like GPT-4 could grasp the meaning of language, or have an internal model of the world that could someday serve as the core of a superintelligence

Altman had appeared before the U.S. Senate. Mark Zuckerberg had floundered defensively before that same body in his testimony about Facebook’s role in the 2016 election. Altman instead charmed lawmakers by speaking soberly about AI’s risks and grandly inviting regulation. These were noble sentiments, but they cost little in America, where Congress rarely passes tech legislation that has not been diluted by lobbyists.

Emily Bender, a computational linguist at the University of Washington, describes GPT-4 as a “stochastic parrot,” a mimic that merely figures out superficial correlations between symbols. In the human mind, those symbols map onto rich conceptions of the world

But the AIs are twice removed. They’re like the prisoners in Plato’s allegory of the cave, whose only knowledge of the reality outside comes from shadows cast on a wall by their captors.

Altman told me that he doesn’t believe it’s “the dunk that people think it is” to say that GPT-4 is just making statistical correlations. If you push these critics further, “they have to admit that’s all their own brain is doing … it turns out that there are emergent properties from doing simple things on a massive scale.”

he is right that nature can coax a remarkable degree of complexity from basic structures and rules: “From so simple a beginning,” Darwin wrote, “endless forms most beautiful.”

If it seems odd that there remains such a fundamental disagreement about the inner workings of a technology that millions of people use every day, it’s only because GPT-4’s methods are as mysterious as the brain’s.

To grasp what’s going on inside large language models like GPT‑4, AI researchers have been forced to turn to smaller, less capable models. In the fall of 2021, Kenneth Li, a computer-science graduate student at Harvard, began training one to play Othello without providing it with either the game’s rules or a description of its checkers-style board; the model was given only text-based descriptions of game moves. Midway through a game, Li looked under the AI’s hood and was startled to discover that it had formed a geometric model of the board and the current state of play. In an article describing his research, Li wrote that it was as if a crow had overheard two humans announcing their Othello moves through a window and had somehow drawn the entire board in birdseed on the windowsill.

The philosopher Raphaël Millière once told me that it’s best to think of neural networks as lazy. During training, they first try to improve their predictive power with simple memorization; only when that strategy fails will they do the harder work of learning a concept. A striking example of this was observed in a small transformer model that was taught arithmetic. Early in its training process, all it did was memorize the output of simple problems such as 2+2=4. But at some point the predictive power of this approach broke down, so it pivoted to actually learning how to add.

Even AI scientists who believe that GPT-4 has a rich world model concede that it is much less robust than a human’s understanding of their environment.

But it’s worth noting that a great many abilities, including very high-order abilities, can be developed without an intuitive understanding. The computer scientist Melanie Mitchell has pointed out that science has already discovered concepts that are highly predictive, but too alien for us to genuinely understand

As AI advances, it may well discover other concepts that predict surprising features of our world but are incomprehensible to us.

GPT-4 is no doubt flawed, as anyone who has used ChatGPT can attest. Having been trained to always predict the next word, it will always try to do so, even when its training data haven’t prepared it to answer a question.

The models “don’t have a good conception of their own weaknesses,” Nick Ryder, a researcher at OpenAI, told me. GPT-4 is more accurate than GPT-3, but it still hallucinates, and often in ways that are difficult for researchers to catch. “The mistakes get more subtle,

The Khan Academy’s solution to GPT-4’s accuracy problem was to filter its answers through a Socratic disposition. No matter how strenuous a student’s plea, it would refuse to give them a factual answer, and would instead guide them toward finding their own—a clever work-around, but perhaps with limited appeal.

When I asked Sutskever if he thought Wikipedia-level accuracy was possible within two years, he said that with more training and web access, he “wouldn’t rule it out.”

This was a much more optimistic assessment than that offered by his colleague Jakub Pachocki, who told me to expect gradual progress on accuracy—to say nothing of outside skeptics, who believe that returns on training will diminish from here.

Sutskever is amused by critics of GPT-4’s limitations. “If you go back four or five or six years, the things we are doing right now are utterly unimaginable,”

AI researchers have become accustomed to goalpost-moving: First, the achievements of neural networks—mastering Go, poker, translation, standardized tests, the Turing test—are described as impossible. When they occur, they’re greeted with a brief moment of wonder, which quickly dissolves into knowing lectures about how the achievement in question is actually not that impressive. People see GPT-4 “and go, ‘Wow,’ ” Sutskever said. “And then a few weeks pass and they say, ‘But it doesn’t know this; it doesn’t know that.’ We adapt quite quickly.”

The goalpost that matters most to Altman—the “big one” that would herald the arrival of an artificial general intelligence—is scientific breakthrough. GPT-4 can already synthesize existing scientific ideas, but Altman wants an AI that can stand on human shoulders and see more deeply into nature.

Certain AIs have produced new scientific knowledge. But they are algorithms with narrow purposes, not general-reasoning machines. The AI AlphaFold, for instance, has opened a new window onto proteins, some of biology’s tiniest and most fundamental building blocks, by predicting many of their shapes, down to the atom—a considerable achievement given the importance of those shapes to medicine, and given the extreme tedium and expense required to discern them with electron microscopes.

Altman imagines a future system that can generate its own hypotheses and test them in a simulation. (He emphasized that humans should remain “firmly in control” of real-world lab experiments—though to my knowledge, no laws are in place to ensure that.)

He longs for the day when we can tell an AI, “ ‘Go figure out the rest of physics.’ ” For it to happen, he says, we will need something new, built “on top of” OpenAI’s existing language models.

In her MIT lab, the cognitive neuroscientist Ev Fedorenko has found something analogous to GPT-4’s next-word predictor inside the brain’s language network. Its processing powers kick in, anticipating the next bit in a verbal string, both when people speak and when they listen. But Fedorenko has also shown that when the brain turns to tasks that require higher reasoning—of the sort that would be required for scientific insight—it reaches beyond the language network to recruit several other neural systems.

No one at OpenAI seemed to know precisely what researchers need to add to GPT-4 to produce something that can exceed human reasoning at its highest levels.

at least part of the current strategy clearly involves the continued layering of new types of data onto language, to enrich the concepts formed by the AIs, and thereby enrich their models of the world.

The extensive training of GPT-4 on images is itself a bold step in this direction,

Others at the company—and elsewhere—are already working on different data types, including audio and video, that could furnish AIs with still more flexible concepts that map more extensively onto reality

Tactile concepts would of course be useful primarily to an embodied AI, a robotic reasoning machine that has been trained to move around the world, seeing its sights, hearing its sounds, and touching its objects.

humanoid robots. I asked Altman what I should make of that. He told me that OpenAI is interested in embodiment because “we live in a physical world, and we want things to happen in the physical world.”

At some point, reasoning machines will need to bypass the middleman and interact with physical reality itself. “It’s weird to think about AGI”—artificial general intelligence—“as this thing that only exists in a cloud,” with humans as “robot hands for it,” Altman said. “It doesn’t seem right.

Everywhere Altman has visited, he has encountered people who are worried that superhuman AI will mean extreme riches for a few and breadlines for the rest

Altman answered by addressing the young people in the audience directly: “You are about to enter the greatest golden age,” he said.

“A lot of people working on AI pretend that it’s only going to be good; it’s only going to be a supplement; no one is ever going to be replaced,” he said. “Jobs are definitely going to go away, full stop.”

A recent study led by Ed Felten, a professor of information-technology policy at Princeton, mapped AI’s emerging abilities onto specific professions according to the human abilities they require, such as written comprehension, deductive reasoning, fluency of ideas, and perceptual speed. Like others of its kind, Felten’s study predicts that AI will come for highly educated, white-collar workers first.

How many jobs, and how soon, is a matter of fierce dispute

The paper’s appendix contains a chilling list of the most exposed occupations: management analysts, lawyers, professors, teachers, judges, financial advisers, real-estate brokers, loan officers, psychologists, and human-resources and public-relations professionals, just to sample a few.

Altman imagines that far better jobs will be created in their place. “I don’t think we’ll want to go back,” he said. When I asked him what these future jobs might look like, he said he doesn’t know.

He suspects there will be a wide range of jobs for which people will always prefer a human. (Massage therapists?

His chosen example was teachers. I found this hard to square with his outsize enthusiasm for AI tutors.

He also said that we would always need people to figure out the best way to channel AI’s awesome powers. “That’s going to be a super-valuable skill,” he said. “You have a computer that can do anything; what should it go do?”

As many have noted, draft horses were permanently put out of work by the automobile. If Hondas are to horses as GPT-10 is to us, a whole host of long-standing assumptions may collapse.

Previous technological revolutions were manageable because they unfolded over a few generations, but Altman told South Korea’s youth that they should expect the future to happen “faster than the past.” He has previously said that he expects the “marginal cost of intelligence” to fall very close to zero within 10 years

The earning power of many, many workers would be drastically reduced in that scenario. It would result in a transfer of wealth from labor to the owners of capital so dramatic, Altman has said, that it could be remedied only by a massive countervailing redistribution.

In 2021, he unveiled Worldcoin, a for-profit project that aims to securely distribute payments—like Venmo or PayPal, but with an eye toward the technological future—first through creating a global ID by scanning everyone’s iris with a five-pound silver sphere called the Orb. It seemed to me like a bet that we’re heading toward a world where AI has made it all but impossible to verify people’s identity and much of the population requires regular UBI payments to survive. Altman more or less granted that to be true, but said that Worldcoin is not just for UBI.

“Let’s say that we do build this AGI, and a few other people do too.” The transformations that follow would be historic, he believes. He described an extraordinarily utopian vision, including a remaking of the flesh-and-steel world

“Robots that use solar power for energy can go and mine and refine all of the minerals that they need, that can perfectly construct things and require no human labor,” he said. “You can co-design with DALL-E version 17 what you want your home to look like,” Altman said. “Everybody will have beautiful homes.

In conversation with me, and onstage during his tour, he said he foresaw wild improvements in nearly every other domain of human life. Music would be enhanced (“Artists are going to have better tools”), and so would personal relationships (Superhuman AI could help us “treat each other” better) and geopolitics (“We’re so bad right now at identifying win-win compromises”).

In this world, AI would still require considerable computing resources to run, and those resources would be by far the most valuable commodity, because AI could do “anything,” Altman said. “But is it going to do what I want, or is it going to do what you want

If rich people buy up all the time available to query and direct AI, they could set off on projects that would make them ever richer, while the masses languish

One way to solve this problem—one he was at pains to describe as highly speculative and “probably bad”—was this: Everyone on Earth gets one eight-billionth of the total AI computational capacity annually. A person could sell their annual share of AI time, or they could use it to entertain themselves, or they could build still more luxurious housing, or they could pool it with others to do “a big cancer-curing run,” Altman said. “We just redistribute access to the system.”

Even if only a little of it comes true in the next 10 or 20 years, the most generous redistribution schemes may not ease the ensuing dislocations.

America today is torn apart, culturally and politically, by the continuing legacy of deindustrialization, and material deprivation is only one reason. The displaced manufacturing workers in the Rust Belt and elsewhere did find new jobs, in the main. But many of them seem to derive less meaning from filling orders in an Amazon warehouse or driving for Uber than their forebears had when they were building cars and forging steel—work that felt more central to the grand project of civilization.

It’s hard to imagine how a corresponding crisis of meaning might play out for the professional class, but it surely would involve a great deal of anger and alienation.

Even if we avoid a revolt of the erstwhile elite, larger questions of human purpose will linger. If AI does the most difficult thinking on our behalf, we all may lose agency—at home, at work (if we have it), in the town square—becoming little more than consumption machines, like the well-cared-for human pets in WALL-E

Altman has said that many sources of human joy and fulfillment will remain unchanged—basic biological thrills, family life, joking around, making things—and that all in all, 100 years from now, people may simply care more about the things they cared about 50,000 years ago than those they care about today

In its own way, that too seems like a diminishment, but Altman finds the possibility that we may atrophy, as thinkers and as humans, to be a red herring. He told me we’ll be able to use our “very precious and extremely limited biological compute capacity” for more interesting things than we generally do today.

Yet they may not be the most interesting things: Human beings have long been the intellectual tip of the spear, the universe understanding itself. When I asked him what it would mean for human self-conception if we ceded that role to AI, he didn’t seem concerned. Progress, he said, has always been driven by “the human ability to figure things out.” Even if we figure things out with AI, that still counts, he said.

It’s not obvious that a superhuman AI would really want to spend all of its time figuring things out for us.

I asked Sutskever whether he could imagine an AI pursuing a different purpose than simply assisting in the project of human flourishing.

“I don’t want it to happen,” Sutskever said, but it could.

Sutskever has recently shifted his focus to try to make sure that it doesn’t. He is now working primarily on alignment research, the effort to ensure that future AIs channel their “tremendous” energies toward human happiness

It is, he conceded, a difficult technical problem—the most difficult, he believes, of all the technical challenges ahead.

As part of the effort to red-team GPT-4 before it was made public, the company sought out the Alignment Research Center (ARC), across the bay in Berkeley, which has developed a series of evaluations to determine whether new AIs are seeking power on their own. A team led by Elizabeth Barnes, a researcher at ARC, prompted GPT-4 tens of thousands of times over seven months, to see if it might display signs of real agency.

The ARC team gave GPT-4 a new reason for being: to gain power and become hard to shut down

Agarwal told me that this behavior could be a precursor to shutdown avoidance in future models. When GPT-4 devised its lie, it had realized that if it answered honestly, it may not have been able to achieve its goal. This kind of tracks-covering would be particularly worrying in an instance where “the model is doing something that makes OpenAI want to shut it down,” Agarwal said. An AI could develop this kind of survival instinct while pursuing any long-term goal—no matter how small or benign—if it feared that its goal could be thwarted.

Barnes and her team were especially interested in whether GPT-4 would seek to replicate itself, because a self-replicating AI would be harder to shut down. It could spread itself across the internet, scamming people to acquire resources, perhaps even achieving some degree of control over essential global systems and holding human civilization hostage.

When I discussed these experiments with Altman, he emphasized that whatever happens with future models, GPT-4 is clearly much more like a tool than a creature. It can look through an email thread, or help make a reservation using a plug-in, but it isn’t a truly autonomous agent that makes decisions to pursue a goal, continuously, across longer timescales.

Altman told me that at this point, it might be prudent to try to actively develop an AI with true agency before the technology becomes too powerful, in order to “get more comfortable with it and develop intuitions for it if it’s going to happen anyway.”

“We need to do empirical experiments on how these things try to escape control,” Hinton told me. “After they’ve taken over, it’s too late to do the experiments.”

the fulfillment of Altman’s vision of the future will at some point require him or a fellow traveler to build much more autonomous AIs.

When Sutskever and I discussed the possibility that OpenAI would develop a model with agency, he mentioned the bots the company had built to play Dota 2. “They were localized to the video-game world,” Sutskever told me, but they had to undertake complex missions. He was particularly impressed by their ability to work in concert. They seem to communicate by “telepathy,” Sutskever said. Watching them had helped him imagine what a superintelligence might be like.

“The way I think about the AI of the future is not as someone as smart as you or as smart as me, but as an automated organization that does science and engineering and development and manufacturing,”

Suppose OpenAI braids a few strands of research together, and builds an AI with a rich conceptual model of the world, an awareness of its immediate surroundings, and an ability to act, not just with one robot body, but with hundreds or thousands. “We’re not talking about GPT-4. We’re talking about an autonomous corporation,”

Its constituent AIs would work and communicate at high speed, like bees in a hive. A single such AI organization would be as powerful as 50 Apples or Googles, he mused. “This is incredible, tremendous, unbelievably disruptive power.”

Presume for a moment that human society ought to abide the idea of autonomous AI corporations. We had better get their founding charters just right. What goal should we give to an autonomous hive of AIs that can plan on century-long time horizons, optimizing billions of consecutive decisions toward an objective that is written into their very being?

If the AI’s goal is even slightly off-kilter from ours, it could be a rampaging force that would be very hard to constrain

We know this from history: Industrial capitalism is itself an optimization function, and although it has lifted the human standard of living by orders of magnitude, left to its own devices, it would also have clear-cut America’s redwoods and de-whaled the world’s oceans. It almost did.

one of its principal challenges will be making sure that the objectives we give to AIs stick

We can program a goal into an AI and reinforce it with a temporary period of supervised learning, Sutskever explained. But just as when we rear a human intelligence, our influence is temporary. “It goes off to the world,”

That’s true to some extent even of today’s AIs, but it will be more true of tomorrow’s.

He compared a powerful AI to an 18-year-old heading off to college. How will we know that it has understood our teachings? “Will there be a misunderstanding creeping in, which will become larger and larger?”

Divergence may result from an AI’s misapplication of its goal to increasingly novel situations as the world changes

Or the AI may grasp its mandate perfectly, but find it ill-suited to a being of its cognitive prowess. It might come to resent the people who want to train it to, say, cure diseases. “They want me to be a doctor,” Sutskever imagines an AI thinking. “I really want to be a YouTuber.”

If AIs get very good at making accurate models of the world, they may notice that they’re able to do dangerous things right after being booted up. They might understand that they are being red-teamed for risk, and hide the full extent of their capabilities.

hey may act one way when they are weak and another way when they are strong, Sutskever said

We would not even realize that we had created something that had decisively surpassed us, and we would have no sense for what it intended to do with its superhuman powers.

That’s why the effort to understand what is happening in the hidden layers of the largest, most powerful AIs is so urgent. You want to be able to “point to a concept,” Sutskever said. You want to be able to direct AI toward some value or cluster of values, and tell it to pursue them unerringly for as long as it exists.

we don’t know how to do that; indeed, part of his current strategy includes the development of an AI that can help with the research. If we are going to make it to the world of widely shared abundance that Altman and Sutskever imagine, we have to figure all this out.

This is why, for Sutskever, solving superintelligence is the great culminating challenge of our 3-million-year toolmaking tradition. He calls it “the final boss of humanity.”

“First of all, I think that whether the chance of existential calamity is 0.5 percent or 50 percent, we should still take it seriously,”

. “I don’t have an exact number, but I’m closer to the 0.5 than the 50.”

As to how it might happen, he seems most worried about AIs getting quite good at designing and manufacturing pathogens, and with reason: In June, an AI at MIT suggested four viruses that could ignite a pandemic, then pointed to specific research on genetic mutations that could make them rip through a city more quickly

Around the same time, a group of chemists connected a similar AI directly to a robotic chemical synthesizer, and it designed and synthesized a molecule on its own.

Altman worries that some misaligned future model will spin up a pathogen that spreads rapidly, incubates undetected for weeks, and kills half its victims. He worries that AI could one day hack into nuclear-weapons systems too. “There are a lot of things,” he said, and these are only the ones we can imagine.

Altman told me that he doesn’t “see a long-term happy path” for humanity without something like the International Atomic Energy Agency for global oversight of AI

In San Francisco, Agarwal had suggested the creation of a special license to operate any GPU cluster large enough to train a cutting-edge AI, along with mandatory incident reporting when an AI does something out of the ordinary

Other experts have proposed a nonnetworked “Off” switch for every highly capable AI; on the fringe, some have even suggested that militaries should be ready to perform air strikes on supercomputers in case of noncompliance

Sutskever thinks we will eventually want to surveil the largest, most powerful AIs continuously and in perpetuity, using a team of smaller overseer AIs.

Safety rules for a new technology usually accumulate over time, like a body of common law, in response to accidents or the mischief of bad actors. The scariest thing about genuinely powerful AI systems is that humanity may not be able to afford this accretive process of trial and error. We may have to get the rules exactly right at the outset.

Several years ago, Altman revealed a disturbingly specific evacuation plan he’d developed. He told The New Yorker that he had “guns, gold, potassium iodide, antibiotics, batteries, water, gas masks from the Israeli Defense Force, and a big patch of land in Big Sur” he could fly to in case AI attacks.

if the worst-possible AI future comes to pass, “no gas mask is helping anyone.”

but he told me that he can’t really be sure how AI will stack up. “I just have to build the thing,” he said. He is building fast

Altman insisted that they had not yet begun GPT-5’s training run. But when I visited OpenAI’s headquarters, both he and his researchers made it clear in 10 different ways that they pray to the god of scale. They want to keep going bigger, to see where this paradigm leads. After all, Google isn’t slackening its pace; it seems likely to unveil Gemini, a GPT-4 competitor, within months. “We are basically always prepping for a run,

To think that such a small group of people could jostle the pillars of civilization is unsettling. It’s fair to note that if Altman and his team weren’t racing to build an artificial general intelligence, others still would be

Altman’s views about the likelihood of AI triggering a global class war, or the prudence of experimenting with more autonomous agent AIs, or the overall wisdom of looking on the bright side, a view that seems to color all the rest—these are uniquely his

No single person, or single company, or cluster of companies residing in a particular California valley, should steer the kind of forces that Altman is imagining summoning.

AI may well be a bridge to a newly prosperous era of greatly reduced human suffering. But it will take more than a company’s founding charter—especially one that has already proved flexible—to make sure that we all share in its benefits and avoid its risks. It will take a vigorous new politics.

I don’t think the general public has quite awakened to what’s happening. A global race to the AI future has begun, and it is largely proceeding without oversight or restraint. If people in America want to have some say in what that future will be like, and how quickly it arrives, we would be wise to speak up soon.

A poll last week put the Pirates, who hang a black pirate flag in their parliamentary office, on nearly 21% of the vote, far short of the 40%-plus it was polling at the height of mass anti-government protests this spring,

The party has ruled out any possibility of forming a coalition with either of the current two ruling parties

“the parties forming a government … hiding behind compromises in coalition – enabling them to cheat voters again and again”.

This election follows the resignation of Iceland’s prime minister, Sigmundur Davið Gunnlaugsson, who became the first major casualty of the Panama Papers

Support for the Independence party, the Pirates’ rival for the position of largest party, seems to be holding.

Birgitta Jónsdóttir, the Pirates’ parliamentary leader, has said her party is willing to form a government with any party that subscribes to its agenda of “fundamental system change”, including the introduction of a new, crowdsourced national constitution.

the party advocates an “unlimited right” for citizens to be involved in political decision-making, with voters able to propose new legislation and decide on it in national referendums.

nce the 2008 crash when Iceland’s three biggest banks collapsed owing 11 times the country’s GDP, Reykjavík’s stock market fell 97% and the value of the krona halved, Iceland has recovered economically.

“Across Europe, increasingly many people think that the system that is supposed to look after them is not doing it any more,” Jónsdóttir said. “But we know we are new to this, and it is important that we are extra careful and extra critical on ourselves to not take too much on.”

rebels do not necessarily have all that much in common with one another, let alone with the remainders of the religious right. The Trump-voting “deplorable” is likely to be a cultural evangelical but not a churchgoer, or a pro-choice lapsed Catholic who never cared for religious moralists. The typical “manosphere” denizen is something else entirely — younger, tech-savvy, impious, impressed with his own unblinking Darwinism. As constituent parts of cultural conservatism, these groups don’t form a particularly coherent whole; what unites them are common fears (feminism, political correctness, sometimes Islam), not a common cause.

Moreover, his provocations tend to actually work, in the sense that they summon up the illiberal, “shut up or we’ll shut you down” side of left-wing politics.

for a cultural conservatism united only by a shared outsider sensibility, neither consistency nor propriety are consensus virtues any longer — and indecency in the service of attacking liberalism is no vice.

Thatcher said something else that's particularly interesting in this discussion. She said there's no such thing as society. There are only individuals and families. And so what we begin to see is the emergence of a kind of ethic, a survival of the fittest ethic that legitimates the most incredible forms of cruelty, that seems to suggest that freedom in this discourse of getting rid of society, getting rid of the social-- that discourse is really only about self-interest, that possessive individualism is now the only virtue that matters. So freedom, which is essential to any notion of democracy, now becomes nothing more than a matter of pursuing your own self interests

I want to echo something that FDR once said, When he said that, you know, you not only have to have personal freedoms and political freedoms, the right to vote the right to speak, you have to have social freedom. You have to have the freedom from want, the freedom from poverty, the freedom from-- that comes with a lack of health care.

How do you get a discourse governing the country that seems to suggest that anything public, public health, public transportation, public values, you know, public engagement is a pathology?

Individualize the social, which means that all problems, if they exist, rest on the shoulders of individuals.

that the government-- the larger social order, the society has no responsibility whatsoever so that-- you often hear this, I mean, if there--I mean, if you have an economic crisis caused by the hedge fund crooks, you know and millions of people are put out of work and they're all lining up for unemployment, what do we hear in the national media? We hear that maybe they don't know how to fill out unemployment forms, maybe it's about character.

I think that what we haven't seen before is an attack on the social contract, Bill, that is so overwhelming, so dangerous in the way in which its being deconstructed and being disassembled that you now have as a classic example, you have a whole generation of young people who are now seen as disposable.

young people can't turn anywhere without in some way being told that the only obligation of citizenship is to shop, is to be a consumer. You can't walk on a college campus today and walk into the student union and not see everybody represented there from the local banks to Disneyland to local shops, all selling things.

Where are the public spaces for young people other learn a discourse that's not commodified, to be able to think about non-commodifiable values like trust, justice, honesty, integrity, caring for others, compassion. Those things, they're just simply absent, they're not part of those public spheres because those spheres have been commodified.

Zombie Politics and Culture in the Age of Casino Capitalism.” Why that metaphor, zombie politics? HENRY GIROUX: Because it's a politics that's informed by the machinery of social and civil death.

It's a death machine. It's a death machine because in my estimation it does everything it can to kill any vestige of a robust democracy. It turns people into zombies, people who basically are so caught up with surviving that they have no-- they become like the walking dead, you know, they lose their sense of agency-

This casino capitalism as we talk about it, right, one of the things that it does that hasn't been done before, it doesn't just believe it can control the economy. It believes that it can govern all of social life. That's different. That means it has to have its tentacles into every aspect of everyday life. Everything from the way schools are run to the way prisons are outsourced to the way the financial services are run to the way in which people have access to health care, it's an all-encompassing, it seems to me, political, cultural, educational apparatus.

as the social state is crippled, as the social state is in some way robbed, hollowed out and robbed of its potential and its capacities, what takes its place? The punishing state takes its place. You get this notion of incarceration, this, what we call the governing through crime complex where governance now has been ceded to corporations who largely are basically about benefiting the rich, the ultra-rich, the big corporations and allowing the state to exercise its power in enormously destructive and limited ways.

we kill the imagination by suggesting that the only kind of rationality that matters, the only kind of learning that matters is utterly instrumental, pragmatist. So what we do is we collapse education into training, and we end up suggesting that not knowing much is somehow a virtue. And I'll and I think what's so disturbing about this is not only do you see it in the popular culture with the lowest common denominator now drives that culture, but you also see it coming from politicians who actually say things that suggest something about the policies they'd like to implement.

Rick Santorum is not-- is kind of a, you know, an obvious figure. But when he stands up in front of a body of Republicans and he says, the last thing we need in the Republican party are intellectuals. And I think it's kind of a template for the sort of idiocy that increasingly now dominates our culture.

I think intellectuals are-- there are two ways we can describe intellectuals. In the most general sense, we can say, "Intellectuals are people who take pride in ideas. They work with ideas." I mean, they believe that ideas matter. They believe that there's no such thing as common sense, good sense or bad sense, but reflective sense.

how we learn what we learn and what we do with the knowledge that we have is not just for ourselves. It's for the way in which we can expand and deepen the very processes of democracy in general, and address those problems and anti-democratic forces that work against it.

I think the real issue here is, you know, what would it mean to begin to do at least two things?

one is to develop cultural apparatuses that can offer a new vocabulary for people, where questions of freedom and justice and the problems that we're facing can be analyzed in ways that reach mass audiences in accessible language. We have to build a formative culture

Secondly, we've got to overcome the fractured nature of these movements. I mean the thing that plagues me about progressives in the left and liberals is they are all sort of ensconced in these fragmented movements

here's the contradiction I hear in what you're saying. That if you write about a turning toward despair and cynicism in politics. Can you get movements out of despair and cynicism? Can you get people who will take on the system when they have been told that the system is so powerful and so overwhelming that they've lost their, as you call it, moral and political agency?

to be different than it is now, rather than romanticizing hope and turning it into something Disney-like, right, it really has to involve the hard work of A) recognizing the structures of domination that we have to face, B) organizing collectively and somehow to change those, and C) believing it can be done, that it's worth the struggle.

I refuse to become complicitous. I refuse to say--I refuse to be alive and to watch institutions being handed over to right wing zealots. I refuse to be alive and watch the planet be destroyed. I mean, when you mentioned-- you talk about the collective imagination, you know, I mean that imagination emerges when people find strength in collective organizations, when they find strength in each other.

The basic question we face is, I suggest, what makes people do what they do?

I will offer a few suggestions in the following six categories. There may well be others, but these are mine. Top of the list, I think, is ignorance. Closely following ignorance is the issue of memory. Next, I suggest is suspicion. Somewhere down the line is escapism. "Why didn't we..." and "why do you remind us...." Then, there is the development process and its downside, corruption. Hard to "objectify" and impossible to "quantify," is my fifth category, the sense of identify. Finally, I reflect on the sense of dignity and its and itsviolation in shameandthe terrible burden of embarrassment.

Every survey on what Americans know about our world shows, objectively speaking, what can only be described as ignorance.  Few Americans know even where any foreign place is, who lives there, what language they speak, or what shapes their daily lives.

Casual conversations with people all over America indicate that few care. Such information is just not a significant part of their lives.

If one could have taken a poll in England at almost any time in its history one would have found the same results.  I suggest that what is different, operationally, is that in England the ordinary citizen did not play a role in determining policy.  That was the job of the small aristocracy. What the people knew or did not know was unimportant.

above all in America, which is the operational head of the world community, what "the people" know or don't know but believe is no longer irrelevant. It sometimes is crucial. That is because elections are more common, even if not always free, and because people almost everywhere, but particularly in the West, have been to some extent politicized. Thus, Ignorance is not new but today it is often determinant.

ignorance is not just unidirectional; it occurs in a context. What "we" think we know about others fits into what other people think "they" know both about themselves and about us. People everywhere tend to know quite a bit about their own circumstances and the actions that shaped those circumstances. That is, much more than foreigners know about them. This necessarily creates a lopsided worldview

These forms of mutual incomprehension or mutual misreadings often cause wars. Consider three examples:

Dirty tricks like our attempt to murder Nasser were and probably still are not uncommon. The Senate Committee headed by Senator Frank Church provided a chilling record, including cooperation with the Mafia, to assassinate Fidel Castro. Assassinations and attempted assassinations by the Russians, the British, the Israelis and others have been less subjected to sustained inquiry than Church provided, but their involvement in many deplorable incidents is not in doubt.

Britain developed Iran

The ancient Hindu who told the parable of the elephant was right. Those who grab the tail cannot understand those who handle the trunk. Understanding of the whole is always and everywhere necessary for intelligent action.

for how long does memory prevent people from doing the same things again?

long-term memory, memory of big happenings like wars, may be crucial but, it seems to me, last only about a decade.  Who today remembers much about American participation in the wars in Greece, Korea or even Vietnam.

Even when we get the sequences right, we usually stop short of determining the causes, that is, the connections between events.

my hunch is that as rapidly as we can, we put aside what we don't want to remember.

we have been able virtually to remove costly and painful events from the immediacy of daily life

Those who dwell on the costly and painful aspects of rising militarism are at best a nuisance who soon wear out their welcome. We find it so much easier to mesh our thoughts and attitudes with those of the people with whom we eat, work, sleep and play.  Better not to pay attention to those who challenge "conventional wisdom" or buck the tide.

Conventional wisdom and going with the mainstream are, arguably, necessary to make society function.

Sometimes, it seems to me that our questions get in the way of our answers and that our analytical tools themselves distort what our eyes are seeing.  We get so sophisticated that we may, to use the old saw, fail to see the forest for the trees.

These activities have created throughout the world a pervasive sense of illegality and immorality. And it cannot be restricted just to foreign affairs. It spills over into domestic affairs not only, as it commonly does, into societies with fragile legal systems but also into ours.

What is important, I suggest about all these -- and many other suspicious events which have never been fully illuminated -- is two fold: on the one hand, a climate of suspicion has been created that makes the achievement of security and peace far more difficult throughout the world and, on the other hand, trust in government, including the government of the United States, has been compromised.

Johnson charged Nixon with treason, but did not hold him accountable. Johnson's successors in the presidency have, similarly, not applied to political leaders the sort of legal standard to which we, as citizens, are held. Nor have they shared with the citizenry what they know has been done in our name. This is a fundamental attack on our system of government. Those who have "blown the whistle" on such activities, not the perpetrators, have been stigmatized or punished.

This adds up, I suggest, to a political form of corruption even worse than the financial corruption that so corrodes the nation "salvation" activities we have mounted in such countries as Iraq and Afghanistan.

What about escapism?  I suggest that escapism is the child of suspicion. I would wager that if one could stop a hundred or so people on the streets of any village, town or city almost everywhere, he would find that only a handful of those he badgered would want to talk about issues some of us keep warning them that could ruin their lives. Most Americans and probably most people everywhere, simply do not want to think about them.

I have found that when such issues as war, environmental degradation, over population, hunger, pandemics, nuclear accidents or even financial collapse are raised, conversation dwindles. As the familiar expression has it, "eyes glaze over," and as quickly as politely possible, Americans flee from the person who raised the issues as though he had made a bad smell

For most people they are better kept at least out of sight if not totally out of mind. Real life, enjoyable life, life that gives amusement or pleasure right now is at hand. It is available even for the very poor on television.  Sports, even in countries where hunger is widespread, jobs few, life constricted and governments oppressive, these annoyances recede before the immediate excitement of football.

we think we are the doctors but really we are the disease. I don't want to believe that, but there is ample proof that much of what we have done with the best of intentions has made many people suffer

we all sought in the late 1950s and early 1960 to "objectify" and "quantify" the study of international affairs.

Insofar as it dealt with the struggles in the Third World, our analysis suggested to some of us that what we were seeking came down to achieving a growth rate of about 3.5 percent

It seems to me that to the degree possible, everything must be done to avoid attacks on dignity and humiliation.

So, what is a sense of identity, how is it manifested and how do outsiders relate to it?

when the first cities were formed about 3,000 years ago, the inhabitants became too numerous to identify themselves by kinship. So, they elaborated their sense of belonging into custom, religion, dress, diet and language. Gradually, and over centuries, they often elaborated their definition of their identity

whatever form "belonging" takes, it is the "glue" that hold societies together and make it possible for the members to live together.

What the residents needed was to stay put, to improve their housing, of course, but more important to be assisted in taking charge of their lives in their own pattern and at their own speed.

For me, this experience threw into relief the American efforts to remake other societies as the neoconservatives urge. Their proposals urge not only to "regime change" but also to "culture change" -- indeed to disassemble -- whole societies. As played out, particularly during the George W. Bush administration, they have caused or exacerbated unrest and war. To the degree we insist on overturning what the people believe to be normal and right -- in effect of undermining the sense of identity, belonging and self-respect even to improve their physical well-being -- we can expect unrest and war to continue

For what we have done, even with statistically proven improvements and with the best of intentions, both we and they have paid and will pay more. The Third and mainly Islamic world is now in revolt.

Last, and closely related to the sense of belonging and identity, I suggest is the deep need of human beings to avoid attacks on their dignity

Close analysis of almost any confrontation shows that it sets the parameters within which rulers have to act or are likely to be overthrown. We neglect it at our peril.

What had happened was that, unwittingly, the governments, at our urging and with our help, had undermined the fundamental "possession" of their peoples, their sense of identity.

Avoiding humiliation is the essence of diplomacy. But when one has overwhelming power, the temptation is always present to push one's advantage, to put the other person in the corner, to make him "blink," to humble him, even to destroy him.

The messaging consists of common themes that recur in various forms. One central theme is a fierce opposition to government, especially so-called “big government.” This reappears in various sub-forms as well, such as rage against bureaucracy, regulations, Washington, D.C., the IRS, the Environmental Protection Agency, and federal politicians.

Another big theme is fear and victimization. You had better watch out because government is gonna getcha! “They,” whoever that may be, are about to take away your rights.  Your freedom is about to disappear.  Your religious liberties will be stripped away.  You won’t be able to make your own healthcare decisions.  Free choice will be gone.  Your children will suffer.

Another common theme is the fear of foreigners, or outsiders. We must protect our own in-group from the vague and mysterious threats posed by those who are a little bit different from us.

And, of course, someone from the Democratic Party, or some “liberal,” is to blame for all of this wreckage. Demonizing a specific target is powerful.  If a Democrat is in the White House, then the president becomes the favorite bullseye.

why would the Republican Party devise such a strategy that has no hope of success? Well, it turns out that they did not devise this strategy. I

t’s not even a strategy at all. It emerged not as a result of a grand Republican master plan, but of market economics.

The extremist right-wing network of Fox News and talk radio was not created by politicians, and it is not funded by a political party.  It is not supported by donations from people seeking political expression.  It was created for one central purpose: to make money.

the business of peddling hate and anger is a fantastically profitable one.

Rush Limbaugh raked in $80 million for himself in 2015 alone. Sean Hannity was paid $30 million. Glenn Beck is personally worth over $100 million. Bill O’Reilly’s prime-time show, “The O’Reilly Factor,” generates over $100 million per year in advertising revenue.

Fox News has dominated the ratings as the number one cable news channel for the last 14 years and reportedly earns over $1 billion in profits annually, making it a golden goose in the overall Fox corporate empire.  Fox itself is one of the most valuable brands in the world with sales of over $13 billion.  And the tycoon behind Fox News, Rupert Murdoch, is personally worth $12 billion.

Corporate profits are greatly impacted by governmental policies.  Corporations, therefore, desire the government to be controlled by whichever political party is the most favorable to corporate profits.  And this, of course, is the Republican Party.

it is indeed a cozy little business model.  The network builds an audience by appealing to people’s fear, insecurity and anger, and simultaneously directs its audience to support the right-wing political party that best protects the network’s own profits.

From a political perspective, it is certainly not healthy to incite anger and hate within a nation’s own population.  And it is not very wise to inflame hostility and rage against a nation’s own government

from a political perspective of creating a cohesive society and maintaining peace and harmony among the population, this is disastrous.

politicians in the Republican Party could not resist.  The extremist right-wing network of Fox News and talk radio had built up an audience that could easily be exploited for political support. 

An unholy alliance was formed.  The Republican Party would allow the extremist right-wing network to promulgate its destructive propaganda throughout society in order to generate its enormous profits, and in exchange, the network would direct its audience to vote for the Republican Party.

So for years and years, the extremist right-wing media network spewed out content full of anger, hate, and division.  And Republican politicians jumped on the bandwagon.  They began preaching the same destructive messages and appearing on the extremist right-wing network all across the nation.

It worked.The base of the Republican Party grew more and more angry.  Their resentment against our very own government grew ever greater.  Their sense of victimization became ever more acute.  Their fury at the establishment boiled over.

The base of the Republican Party became a Frankenstein. It became radicalized into an extreme movement that turned against the established order, including the leadership of the Republican Party itself. 

This is the reason behind the rise of candidates like Donald Trump and Ted Cruz.  The Republican Party establishment despises these candidates, but the party has no idea how to slay these dragons.

“Nothing in the West can compare with the role which aesthetics has played in Japanese life and history since the Heian period”; and “the miyabi spirit of refined sensibility is still very much in evidence” in modern aesthetic criticism.9

there has run through history the idea that the Japanese are, in terms of their original nature (that is, their nature before the introduction from the outside of such systems of thought and religion as Confucianism and Buddhism), essentially an emotional people. And in stressing the emotional side of human nature, the Japanese have always assigned high value to sincerity (makoto) as the ethic of the emotions.