He had been invited by the Vatican to a conclave-style meeting of academics, entrepreneurs and human-rights advocates to draft a policy appeal on AI safety to present to Pope Leo. Tegmark had been trying to get the group to oppose uncontrollable superintelligence in the text. Now it was time for a Hail Mary.
He handed fellow attendee Marco Trombetti, chief executive of the AI-translation company Translated, one of the papers, printed with a call to stall the development of AI with humanlike intelligence until it could be done safely.
Walking between Renaissance-era buildings, Tegmark said in a low voice, “Marco, would you sign a statement like this?”
Trombetti hesitated at first. For more than a decade, Tegmark’s organization, the Future of Life Institute, had been trying to raise the alarm that advanced AI could be catastrophic to humanity. Shortly after the launch of OpenAI’s ChatGPT, the Future of Life Institute made a famous call for a pause in the most advanced AI development.
Now, he was asking for something even more audacious: a potential halt in the race to build superintelligence that was powering the stock market.
Trombetti ultimately signed, along with more than 130,000 other people, including AI luminary Geoffrey Hinton, MAGA die-hard Steve Bannon, former government official Susan Rice and celebrities will.i.am, Prince Harry and Meghan Markle. Tegmark’s language even found its way into the appeal to the pope.
At 58, with his trademark motorcycle jacket, mop of brown hair and black T-shirt decrying “AGI” (AI with humanlike intelligence), Tegmark presents a bit like a nerdy Ramone. “This is by far the most successful thing we’ve ever done,” he said during a recent interview near his home in the Boston suburbs, referring to the number of signatures the statement nabbed compared with the organization’s past open letters.
The politics of AI are changing. Concerns about risks that were once confined to debates among geeks have gone mainstream and increasingly bipartisan, fueled by fears about what AI is going to do to the job market.
About half of Americans say they are more concerned than excited about AI, according to a recent Pew Research Center study, up from 37% in 2021, with concerns on both sides of the aisle.
One marker of this trend: Tegmark’s recent appearances on Bannon’s “War Room” podcast, where the host said his message resonates with the far-right audience worried about jobs, despite the Trump administration’s accelerationist approach to AI.
“Our show is the biggest supporter of President Trump, but on this one, people want to slow down, number one, and they want guardrails,” Bannon said in an interview.
On the other side of the aisle, Virginia Democrat Rep. Don Beyer, co-chair of the congressional AI caucus, agrees with Tegmark. “The people I represent are way more concerned about AI than excited about it,” said Beyer.
Critics of the push for regulation, such as Trump AI czar David Sacks, argue that the U.S. cannot afford to lose the AI race to China.
Buttonholing Musk
Born in Sweden to a Swedish psychologist mother and American mathematician father, Tegmark studied economics before shifting to physics, earning his Ph.D. in the subject from the University of California at Berkeley. He joined MIT’s physics department in 2004.
A physics conference first brought him into the orbit of Jaan Tallinn, the Skype co-founder who has become one of the biggest philanthropists funding AI safety efforts. Tegmark pitched Tallinn on his vision for an organization that would help with this fight, and he agreed to fund it with an initial $100,000.
Tegmark buttonholed Elon Musk at a 2014 MIT conference on space, after one of his students asked a question that prompted the billionaire’s now-famous quip that, with AI, “we are summoning the demon.” Musk, who would later found xAI, attended a Puerto Rico conference Tegmark organized in 2015 and announced a $10 million donation to Tegmark’s nonprofit. Musk didn’t respond to a request for comment.
Tegmark’s organization received a windfall in 2021 when Ethereum co-founder Vitalik Buterin donated more than $665 million in cryptocurrency as an endowment.
Months after ChatGPT’s 2022 launch, the Future of Life Institute made global headlines with an open letter, signed by more than 30,000 people, including Musk and Apple co-founder Steve Wozniak, calling for a six-month pause in the most advanced AI development.
Nobody paused, but Tegmark saw the exercise as a victory because it destigmatized critiquing AI development.
‘Hope you’ve got savings’
Tegmark’s latest campaign is more pointed — and more populist.
He says humanity can benefit from AI tools that help with specific missions such as curing cancer, without building a “general” AI that could economically replace humans or take over governments.
“AGI is the only kind of AI that will destroy all jobs,” Tegmark said.
To make its point, the Future of Life Institute backed a satirical billboard campaign across San Francisco from a fake AI company, Replacement AI, that promises to replace humans in the job market. “Our AI can automate all your work. No more Mondays, bosses, or paychecks! Hope you’ve got savings [LOL emoji],” one billboard read.
The organization also funds policy, research, education and grant programs, including an AI journalism fellowship.
Tegmark’s views on AI have made him controversial to some in Silicon Valley.
This summer he was subpoenaed by OpenAI, which sought information on the Future of Life Institute’s ties to Musk.
An OpenAI spokesman referred to past statements on X from Jason Kwon, OpenAI’s chief strategy officer. Kwon said subpoenas sent out to various nonprofit groups were related to a lawsuit Musk brought against the startup — challenging its ability to convert to a more traditional company structure. Tegmark has been a critic of OpenAI’s conversion, but denied any current ties to Musk. The Future of Life Institute said the billionaire neither currently funds nor has any role there.
The organization’s call for a halt in AI development has revealed cracks in the AI safety community. The fissures are by design, according to Tegmark.
Up to now, organizations backed by Tallinn and those supported by Facebook co-founder Dustin Moskovitz and his wife, Cari Tuna, have been among the biggest funders of AI safety, often overlapping in their goals and methods.
But Tegmark said his appeals were met with rejection or silence from their charity Coefficient Giving — recently renamed from Open Philanthropy — despite its having given more than $1 million to his organization over the years. Leaders from Anthropic, whose leadership has deep personal and philosophical ties to their charity, also didn’t sign on to his appeal, Tegmark said.
Anthropic declined to comment. A spokesman for Coefficient Giving said that, because there is disagreement about which AI threats are most pressing, it avoids “taking institutional positions that might imply a settled view about the right approach.”
Tegmark sees this as evidence of two camps emerging in AI safety: one that wants to build superintelligence and try to make sure it is safe, and another that doesn’t want to build AI that is smarter than humans at all unless it can be proven safe. He puts himself in the latter category.
“It’s going to get nasty,” Bannon said. “You are talking about the leading source of power in the 21 century.”
Write to Keach Hagey at [email protected] and Sam Schechner at [email protected]
Read more on The Wall Street Journal
