Anthropic’s position on AI safety is helping it lure talent away from OpenAI
The leadership shakeup at top AI companies continued this week. Jan Leike, one of OpenAI’s lead safety researchers who resigned earlier this month stating that safety at the company “has taken a backseat to shiny products,” joined rival firm Anthropic.
“I’m excited to join @AnthropicAI to continue the superalignment mission! My new team will work on scalable oversight, weak-to-strong generalization, and automated alignment research,” Leike posted on X.
The new role sounds almost identical to the one Leike had at OpenAI, where he co-led the “superalignment” team tasked with understanding the risks of future powerful AI systems and ensuring they could be controlled. The OpenAI superalignment team—which Leike ran alongside Ilya Sutskever, OpenAI cofounder and chief science officer who also resigned last month—lasted less than a year before its co-leads stepped down and OpenAI disbanded the effort. Having your top AI Safety folks all jump ship is a bad look, to say the least. The company once positioned the team as vital, but OpenAI never gave the team the resources it promised them in order to do the job, as my Eye on AI coauthor Jeremy Kahn exclusively reported in Fortune.
Leike’s safety-related departure comes alongside a whirlwind of recent scandals that have chipped away at public trust in OpenAI—from the intense non-disparagement agreements the company made employees sign to using a voice that sounds eerily similar to Scarlett Johansson’s after she denied the company’s offer to voice ChatGPT. (OpenAI has denied it deliberately tried to imitate Johansson’s voice and has shown the Washington Post documents to back up its position.)
Former OpenAI board member Helen Toner this week also broke her silence about why the board abruptly fired Sam Altman last November, saying he didn’t notify the board about ChatGPT’s release and gave inaccurate information about the company’s safety processes “on multiple occasions,” among other reasons. (Current OpenAI board members Bret Taylor and Larry Summers responded to an editorial that Toner and her fellow former OpenAI board member Tasha McCauley wrote in The Economist calling for regulation of AI companies with an editorial of their own. Summers and Taylor said that in six months of near-daily contact with OpenAI, they had found Altman to be “highly forthcoming on all relevant issues and consistently collegial with his management team.” They also noted that Altman and OpenAI had repeatedly called for governments to regulate AI and AI companies.)
So far, discussions around incentives in the intensifying war for top AI talent have mostly revolved around multimillion-dollar compensation packages, and the ability of Big Tech companies such as Meta to offer AI experts access to thousands of GPUs, the specialized computer chips needed to build and run AI models. With their skills in such high demand, experienced AI researchers and engineers have reportedly fetched pay packages as high as $10 million. Lekie’s jump to rival Anthropic to do basically the same job, however, suggests that AI companies’ demonstrated commitment to safety may become an increasingly important factor in the AI talent war. There are a lot of folks who want to see AI developed and deployed responsibly, and the stream of resignations from OpenAI over safety concerns shows that top talent will go elsewhere if they feel the company they’re working for is on a dangerous path.
The stakes are getting higher by the day as AI (and its harmful effects) advance rapidly, with insufficient guardrails. For AI professionals, whether they are concerned with existing harms, like deepfakes, or long-term risks, like AI destroying humanity, as Leike is, there’s no time to waste at a company that won’t take their work or the risks seriously. Engineers who are developing AI systems might also think twice about where they bring their talents if they believe certain companies are acting recklessly, fearing they’ll be blamed for unleashing harmful AI and all the consequences that come with it.
So far, Anthropic has been the main beneficiary of this. The company has largely positioned itself as more safety-focused than OpenAI, and Leike isn’t the first leader to defect from OpenAI to Anthropic over safety concerns. In fact, Anthropic was founded by two OpenAI executives who left the firm in 2020 over concerns it was prioritizing commercializing the technology over safety.
Since its launch, Anthropic has continued to take steps that point toward more conscientious development of AI. The company’s “constitutional AI” research, for example, involved polling American adults about what sort of principles they think would be important for an AI model to abide by and then training a smaller version of Claude—the company’s ChatGPT competitor—based on their suggestions. Last week, Anthropic published a research paper detailing its latest efforts to understand the inner workings of large language models. And while OpenAI has moved away from its nonprofit beginnings and seen its board splinter (and even oust CEO Sam Altman) over safety concerns, Anthropic has taken sharp moves in the other direction. Last year, Anthropic registered as a Long-Term Benefit Trust (LTBT), which bakes in a governance mechanism that allows an independent body of financially disinterested members to dismiss board members if they fail to uphold the company’s stated mission to “responsibly develop and maintain advanced AI for the long-term benefit of humanity.” Though to be fair, it’s not clear exactly how this process would play out. A profile of the company published today in Time further dives into Antrhopic’s approach and hope to create “a race to the top” on safety.
Besides rival AI companies, the lure of doing right by humanity could also boost another entity in its AI recruiting: the federal government. With the urgent need for AI expertise across government agencies—including ones that will impact how AI is deployed—AI professionals who have felt burned by the tech industry’s pursuits for endless profits might come to believe they could make a stronger impact in the public sector. Seeing as the government is unable to compete with sky-high private sector salaries, it may be its best sales pitch—and some government officials have made it, though maybe not in so many words. Of course, without passing measures to regulate AI, and by continuing to give companies like OpenAI free reign, the government’s message that it’s a place to ensure AI is developed safely may start to ring hollow.
And with that, here’s more AI news.
Sage Lazzaro
sage.lazzaro@consultant.fortune.com
sagelazzaro.com
AI IN THE NEWS
Meta identifies a network pushing deceptive, “likely AI-generated” content around Israel-Palestine. That’s according to Reuters. The Israel-based network posed as locals in the countries they targeted and shared likely AI-generated comments on posts of public figures and news organizations. These comments—which ranged from praising Israel’s military actions to claiming that “radical Islam” poses a threat to liberal values in Canada—often included links to the operation’s websites and were met with critical responses from authentic users calling them propaganda. Meta also found the comments on X and YouTube and has since removed the network from its platforms, according to its Q1 2024 Adversarial Threat Report.
OpenAI also says it has found several government-affiliated groups and companies using its tools to automate political disinformation campaigns. The company, in its first report of this kind, said it had uncovered covert efforts by government-affiliated groups and private companies in Russia, China, Iran, and Israel pushing disinformation and propaganda, the New York Times reported. The company found groups using its models to create social media posts and translate them between languages. The Israeli company had used the AI tools to generate fake personas that it could use to create fraudulent social media accounts that it then used to spread anti-Islamic content in the U.S. The OpenAI researchers involved in the report said that many of the AI-generated posts received few “likes” or reposts, indicating that the campaigns were likely not very effective.
Anthropic gives its Claude AI model the ability to use other software tools in a step towards more agent-like AI. The San Francisco-based AI company announced in a blog post that it had updated its Claude AI model to allow the system to use other software tools. The move is a step towards AI agents that will not just generate text, images, sound and video, but perform actions for users across the web. In this case, Anthropic said Claude could now extract information from documents and use it to create tabular data in spreadsheets and other kinds of databases, or to automatically fill in forms. It can also convert natural language instructions into API calls, so that an instruction like “cancel this subscription” would enable the AI model to actually go ahead and perform the cancellation. It said it could also manage a number of other Claude AI models each assigned a specific task in order to handle more complex actions such as looking across multiple people’s calendars in order to find the optimal time to schedule a meeting. Some of these capabilities were already available through rival AI chatbots, such as OpenAI’s GPT-4 and GPT-4o, and some have been previewed by Google as coming soon to its Gemini models. But some of the tool-using abilities Anthropic announced seem to go further than those currently available from any other AI company.
Apple plans to process data from AI apps in a “virtual black box.” That’s according to The Information. Typical cloud computing involves decrypting data in order to process it, but Apple is pursuing a technique that would allow it to keep data private. This would also make it impossible for employees to access the data, sources told The Information. How Apple would introduce AI features, such as a revamped Siri, into its products without compromising its pledge to user privacy has been a top-of-mind question. The company is expected to finally shed light on its AI plans at its upcoming developer conference next month.
Kenyan AI workers speak out about being exploited by U.S. tech companies in open letter to President Biden. The letter comes ahead of Biden’s upcoming official state visit with Kenya President William Ruto. “US Big Tech companies are systemically abusing and exploiting African workers. In Kenya, these US companies are undermining the local labor laws, the country’s justice system and violating international labor standards. Our working conditions amount to modern day slavery,” they write. The letter specifically names OpenAI, Meta, and Scale AI and describes how their work labeling videos and photos for AI systems often involves watching videos of murder, child abuse, rape, and other horrific acts of violence for more than eight hours a day. They make less than $2 per hour, and many suffer from PTSD, they write.
FORTUNE ON AI
Sam Altman’s reputation gets dragged further into the mud —David Meyer
Generative AI is in the crosshairs at Meta and Alphabet annual meetings as shareholders vote for detailed reports —Sharon Goldman
I’ve read over 100 AI requests for proposals from major companies. Here’s the matrix of guardrails and obligations that is emerging —May Habib (Commentary)
How companies can keep adding tech like AI without suffering ‘change fatigue’ —Sheryl Estrada
AI CALENDAR
June 5: FedScoop’s FedTalks 2024 in Washington, D.C.
June 10-14: Apple WWDC in Cupertino
June 25-27: 2024 IEEE Conference on Artificial Intelligence in Singapore
July 15-17: Fortune Brainstorm Tech in Park City, Utah (register here)
July 30-31: Fortune Brainstorm AI Singapore (register here)
Aug. 12-14: Ai4 2024 in Las Vegas
EYE ON AI NUMBERS
52%
That’s the percentage of answers ChatGPT gave in response to programming questions from Stack Overflow that contained incorrect information, according to a paper by Purdue researchers that was presented this month at the Conference on Human Factors in Computing Systems. They also found that participants in the user study portion of their research overlooked the errors in the ChatGPT answers 39% of the time.
The analysis comes as more companies invest in and adopt AI coding assistants. Of course, ChatGPT wasn’t designed specifically to function as a coding assistant, as tools like GitHub’s CoPilot were. Yet, people have still been using it for coding questions and OpenAI has made clear its interest in bolstering capabilities within ChatGPT. Just recently, the company inked a deal with Stack Overflow that will allow it to scrape user posts to train ChatGPT. Additionally, nearly two-thirds of organizations have deployed or are piloting AI coding tools, according to a Gartner survey of 598 global respondents from Q3 2023.
