OpenAI Safety Leader Jan Leike Joins Rival Firm Anthropic
Topline
Former OpenAI executive and key researcher Jan Leike on Tuesday announced he has joined artificial intelligence firm Anthropic, defecting to a rival less than two weeks after he resigned over safety concerns, as the ChatGPT maker fends off criticism it is favoring profits over long-term safety.
The former co-head of OpenAI’s superalignment safety team has joined rival AI company Anthropic, … [+]
Key Facts
Leike, who until recently jointly led OpenAI’s long-term safety program alongside company cofounder Ilya Sutskever, said he is “excited” to join AI firm Anthropic “to continue the superalignment mission!”
Superalignment is an area of research dedicated to ensuring the long-term safety of AI by making sure a potential superintelligence — a yet-to-be-realized form of artificial intelligence that can outsmart humans — is controllable, acts in line with human values and will ultimately not go rogue and harm humanity.
Leike said his new team “will work on scalable oversight, weak-to-strong generalization, and automated alignment research,” similar topics he had worked on at OpenAI.
He encouraged like minded researchers to get in touch and said “if you’re interested in joining, my dms are open.”
It is not clear what role Leike will hold at Anthropic and the company has yet to make an official statement on his appointment.
Get Forbes Breaking News Text Alerts: We’re launching text message alerts so you’ll always know the biggest stories shaping the day’s headlines. Text “Alerts” to (201) 335-0739 or sign up here.
News Peg
From the start, OpenAI has sought to position itself as an AI company dedicated to benefiting humanity and ensuring the long-term safety of artificial intelligence. In recent years, OpenAI’s commitment to this mission has come under increasing scrutiny from both outsiders worried about the risks of advanced AI and those intimately involved in the company. This latter cohort includes tech billionaire and company cofounder Elon Musk, who has publicly criticized CEO Sam Altman for allegedly abandoning its founding open source mission in favor of pursuing lucrative deals with the likes of Microsoft. It also includes the seven founders of Anthropic, a group of researchers who defected from OpenAI in 2021 that includes siblings Dario Amodei and Daniela Amodei, who respectively lead the company as CEO and president. Anthropic, which has received backing from the likes of Google and Amazon, is now a major competitor to OpenAI and has sought to style itself and its flagship AI model and chatbot, Claude, as a more safety-oriented venture.
Crucial Quote
Leike was openly critical of OpenAI’s approach to safety after leaving the company and said he felt like he and his team had been “sailing against the wind” and struggling to secure resources for safety work. “Over the past years, safety culture and processes have taken a backseat to shiny products,” Leike said, urging the company to prioritize safety research and get serious about the implications of AGI.
What We Don’t Know
Leike left OpenAI shortly after the departure of cofounder Sutskever and afterwards the company effectively dissolved the superalignment team they co-led (it has, however, launched a new safety and security committee). It is not clear what Sutskever plans to do next or where he will do it but industry insiders will be sure to watch the movements of such a high profile figure closely. Sutskever, who was also OpenAI’s chief scientist and played a role in ousting CEO Altman, was less openly critical of his former employer than Leike. In a post on X, he said he is “confident that OpenAI will build AGI” — short for artificial general intelligence, a theoretical system capable of performing broadly across a range of domains as humans do — “that is both safe and beneficial” and praised the company’s leaders. Sutskever said he is “excited for what comes next” and said he “will share details in due time,” indicating only that it is “a project that is very personally meaningful to me.”
