The Nitty-Gritty About That Latest Risk-Of-AI Letter And A Vaunted Call For A Right To Forewarn

Let’s talk about a newly released AI-impacts letter that has gotten viral attention online and in the mainstream media.

You might have heard or seen that a letter was posted today reiterating a recurring theme of contemporary AI being potentially highly risky, including qualms about the latest in generative AI and large language models (LLMs) that could produce AI-powered harms. The letter drew special attention because the writers are AI researchers and developers who seek to establish a semblance of a “right to warn” when working inside companies that make or promulgate AI systems.

Readers of my column asked me to comment on the matter and explain the nitty-gritty of what the kerfuffle is about. I have extensively covered the ins and outs of AI ethics and AI law, along with a range of AI trends in a wide variety of domains and uses, see the link here.

This newly released letter and the call for a need to freely be able to forewarn about AI risks fits well within the enduring question about where we are with AI and where things are headed, both good and bad.

Contextual Considerations About The Letter

To get a sense of what the letter signifies, take a look at these blazing headlines:

• “Current and former OpenAI employees warn of AI’s ‘serious risk’ and lack of oversight” (CNBC)
• “Employees claim OpenAI, Google ignoring risks of AI and should give them ‘right to warn’ public” (New York Post)
• “OpenAI, Google DeepMind’s current and former employees warn about AI risks” (Reuters)
• “AI workers demand stronger whistleblower protections in open letter” (Engadget)
• “Former OpenAI employees lead push to protect whistleblowers flagging artificial intelligence risks” (The Washington Post)
• “OpenAI insider’s latest open letter is a warning against AI. Will anyone listen?” (Fast Company)

The gist of those headlines is that the letter’s writers are or have been AI workers who wish to speak out about AI risks.

You might at first thought be surprised that AI researchers and AI developers might feel that they cannot bring up the weighty and ongoing topic of heightened AI risks as we push forward in making the latest AI advancements. It would seem they could readily post commentary on social media. If you peruse social media about AI risks and AI safety, you assuredly will find a plethora of useful insights, though regrettably also have to sift through a slew of confounding and often misleading riffraff on the heady matter.

A difficulty that such employees face is that sometimes the AI-making firm that employs them has various restrictions on what they are generally permitted to express. Looming over their heads like a perilous dangling sword is a carefully concocted mixture of carrots and sticks. A carrot might be that if they essentially take no overt action and thus do not speak out, they will be rewarded via job promotions and compensation additions, while a stick might be that they could lose their valuable stock options or be considered a persona non grata within that firm and possibly silently blacklisted within the AI high-tech leadership community.

Okay, you might say, they could aim therefore to be whistleblowers. That appears to be a somewhat common avenue these days in a whole host of fields and realms.

The rub in this instance is that a whistleblower is typically coming forth to report on an illegal activity. In that sense, they are presumably to be lauded for stepping into the morass that goes along with being a whistleblower. Society would likely herald bravery in going against the grain to disclose illegal acts that disobey the law.

A conundrum is what do you do when the acts aren’t illegal, and instead perhaps construed as unethical, though this takes thorny matters into quite a grey area. A whistleblower who is relying on what they consider to be an unethical practice is standing on very slippery sand, possibly quicksand.

In a research article that discusses the legal facets of whistleblowing, entitled “The Legalistic Organizational Response to Whistleblowers’ Disclosures in a Scandal: Law Without Justice?” by Oussama Ouriemmi, Journal of Business Ethics, January 2023, the research states this overall point:

• “Organizational transgressions cause recurring scandals. Often disclosed by whistleblowers, they generate public outrage and force organizations to respond. Recent studies have tried to answer the question: “What happens after a transgression becomes publicly known?” They highlight organizational responses marked by recognition of the transgression, penance, and reintegration of the organization. However, that research only deals with transgressions involving illegal organizational practices. This article broadens the field of study to include legal but unethical organizational practices.”

The trouble of course is that one person’s ethical concerns might not comport with someone else’s semblance of ethical conundrums.

Per the same research paper noted above, consider these salient points about the daring and dicey path of being a whistleblower in a legal context versus in an ethics-only context (excerpts):

• “However, whistleblowers pay a high price for their actions; they nearly always face retaliation from the organizations concerned.” (ibid).
• “Retaliation against the whistleblower by the organization may take varied and non-exclusive forms: demotion, deterioration of working conditions, threats, harassment, ostracism, referral to psychiatrists, dismissal, legal proceedings, etc.” (ibid).
• “When whistleblowing makes confidential information public, the organization may sue the whistleblower. The resulting legal proceedings are often long and costly for the whistleblower and the outcome is not always in the whistleblower’s favor.” (ibid).

The upshot is that if someone is working inside an AI-making company, and they believe in their heart of hearts that the AI is progressing into foul territory, but not in any illegal terrain, their choice to be a whistleblower becomes enormously personally pricey.

Imagine this.

An AI developer is working in a company that makes a modern-day generative AI app. They are part of a team involved in shaping the AI so that it will be able to generate content that evildoers might leverage, perhaps aiding in making toxic chemicals (a topic I’ve discussed in a real-life setting at the link here). The AI developer believes this to be an unethical modification to the AI. Assume that there isn’t any legal restriction involved and thus nothing illegal is being undertaken.

Now what?

The AI developer discusses their concerns with their supervisor. Suppose the supervisor says it isn’t in their purview to complain or disagree with what upper management wants done. Get on with the job. Bonuses depend on hitting an upcoming deadline to get the mods implemented. Stop whining, desist the bickering, get your head down in the trenches, and get the coding done. Period, end of story.

I will ask again, now what?

Well, maybe go to social media and tell what’s going on. As noted earlier, this could have damaging career ramifications.

Quit then.

Quit and go to social media to warn the world about what is happening. Again, there are likely employment and post-employment provisions that will come into the picture. You can certainly try to legally fight against the provisions, but the odds are that it will require a hefty legal battle, for which the employer has big bucks and lots of patience. Your life will go into a legal vicious cycle with money flowing out incessantly as you carry on the battle at hand.

Here’s a twist.

Envision that someone opted to be a whistleblower and the AI company emphasized that they have plenty of cybersecurity built into the AI such that it won’t be turned into a Dr. Evil capability. The firm exclaims that this AI developer was the proverbial “person who cried wolf” when there wasn’t a wolf anywhere to be found.

Furthermore, the AI firm showcases that generative AI via the mods can aid in determining which chemicals might indeed be toxic, serving a vital public interest. The AI will ensure that the world knows which chemicals can produce adverse results. Luckily, thankfully, AI will be a boon to humankind.

Yikes, the whistleblower is knocked back on their heels. Not only might they be confronted with a legal fight or other repercussions from their disclosure, but they are also being smashed into smithereens in the court of public opinion.

I trust that you can see the rock-and-a-hard-place dilemma.

Taking A Close Look At The Letter

Let’s unpack the letter, doing so step by step and I will be extensively quoting from the letter as we explore it.

The letter is entitled “A Right to Warn about Advanced Artificial Intelligence” and was posted online today, June 4, 2024, including listing the names of three AI luminaires (Yoshua Bengio, Geoffrey Hinton, Stuart Russell), and seven named former or current employees of either OpenAI, DeepMind, or Anthropic, plus six unnamed “anonymous” presumed employees that indicate they are or were employees of OpenAI.

The letter begins with this (excerpt):

• “We are current and former employees at frontier AI companies, and we believe in the potential of AI technology to deliver unprecedented benefits to humanity.”
• “We also understand the serious risks posed by these technologies. These risks range from the further entrenchment of existing inequalities, to manipulation and misinformation, to the loss of control of autonomous AI systems potentially resulting in human extinction. AI companies themselves have acknowledged these risks [1, 2, 3], as have governments across the world [4, 5, 6] and other AI experts [7, 8, 9].” {Side note, the numbers refer to included references that were listed at the end of the letter; not shown here}

I’ll pause there to provide some reflection.

The opening statement indicates that the letter writers work at or have worked at so-called frontier AI companies. This is insider speak that is commonly used to refer to AI companies at the very forefront of advancing AI. Envision slogging across the Wild West in the days when you confronted grand unknowns in a potentially treacherous untamed frontier. That’s what some would suggest is happening with AI.

The analogy is perhaps apt due to the realization that a kind of duality exists. On a trek through an unmapped frontier, you might discover amazing waterfalls, nuggets of gold, and all manner of magnificent treasures that will be highly prized for the betterment of humankind. The thing is, you might also discover and bring out of hiding horrific communicable diseases or other gravely societally endangering elements, doing so to the massive detriment of humanity. You’ve almost certainly heard about the considered existential risk of AI, sometimes known as the “P(doom)” or the probability associated with the cataclysmic doom of AI.

AI is a duality, said to have a dual-use capacity, see my coverage at the link here.

AI can be used for badness and evildoing. Meanwhile, AI can be used for goodness, such as hopes that the latest AI might aid in curing cancer, and possibly aid in attaining the United Nations SDG (for my analysis of AI and sustainability development goals, see the link here), and so on. There is a trade-off involved in the pursuit of AI advancement. Presumably, we would want to minimize the downsides of what AI might be dourly used for and maximize the upsides.

In one sense, many AI researchers and AI developers believe they agonizingly face the following plethora of dilemmas each and every day:

• Will the AI advancement they are working on be a blessing or will it be a curse?
• Should they look the other way if they believe it might be a curse?
• If it is a blessing, how will others potentially turn it on its ear and make it into a curse?
• Who is to say whether an AI advancement leans more to one side or the other?
• What if they inadvertently fall into a curse without even suspecting it beforehand?
• Once the bell is rung and an AI advancement exists, and if it is sour, what happens then?
• Can AI be somehow controlled and confined, or will AI go beyond our reach?
• Etc.

I’ve discussed how this can be weary and unsettling for those deeply immersed in the AI field, see the link here.

Those outside of AI have been catching on and realizing that maybe leaving these potential humankind impacts to the hands of AI makers and AI researchers and developers alone, might be a bit disconcerting. All kinds of ethical and legal considerations are currently up for grabs, see for example my discussions at the link here and the link here, just to name a few.

This takes us to the next portion of the letter (excerpts):

• “We are hopeful that these risks can be adequately mitigated with sufficient guidance from the scientific community, policymakers, and the public. However, AI companies have strong financial incentives to avoid effective oversight, and we do not believe bespoke structures of corporate governance are sufficient to change this.” (ibid).
• “AI companies possess substantial non-public information about the capabilities and limitations of their systems, the adequacy of their protective measures, and the risk levels of different kinds of harm. However, they currently have only weak obligations to share some of this information with governments, and none with civil society. We do not think they can all be relied upon to share it voluntarily.” (ibid).
• “So long as there is no effective government oversight of these corporations, current and former employees are among the few people who can hold them accountable to the public. Yet broad confidentiality agreements block us from voicing our concerns, except to the very companies that may be failing to address these issues. Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated. Some of us reasonably fear various forms of retaliation, given the history of such cases across the industry. We are not the first to encounter or speak about these issues.” (ibid).

That pretty much lays out the landscape.

AI makers are presumably sitting in the driver’s seat. At this juncture, the existing legal and regulatory entailments are seen widely as not currently up to the task of overseeing and controlling a seemingly precarious position of AI inching toward adverse outcomes, see the link here.

Some AI makers try to emphasize that you need to balance the potential downsides with the possible upsides. If you take a heavy hand on AI makers, you might lose the innovation race and forestall the benefits.

Worse still, there is the raised point that if this or that AI maker doesn’t get there first, you are possibly allowing someone else to do so. This is usually couched this way. Suppose a fly-by-night AI nutjob invents the dreadful AI you are worrying about. Or a rogue government does so. Wouldn’t it be worse for us all?

The pendulum swings from side to side, at times ending up toward a near free-for-all, while at other times leading to clamors for unimpeachable controls and regulations.

Getting Into The Ask

What do the letter writers propose as a means to contend with these issues?

Here’s what the letter says (bolding shown as per the letter, excerpts):

• “We therefore call upon advanced AI companies to commit to these principles:” (ibid).
• “1. That the company will not enter into or enforce any agreement that prohibits “disparagement” or criticism of the company for risk-related concerns, nor retaliate for risk-related criticism by hindering any vested economic benefit;” (ibid).
• “2. That the company will facilitate a verifiably anonymous process for current and former employees to raise risk-related concerns to the company’s board, to regulators, and an appropriate independent organization with relevant expertise;” (ibid).
• “3. That the company will support a culture of open criticism and allow its current and former employees to raise risk-related concerns about its technologies to the public, to the company’s board, to regulators, or an appropriate independent organization with relevant expertise, so long as trade secrets and other intellectual property interests are appropriately protected;” (ibid).
• “4. That the company will not retaliate against current and former employees who publicly share risk-related confidential information after other processes have failed. We accept that any effort to report risk-related concerns should avoid releasing confidential information unnecessarily. Therefore, once an adequate process for anonymously raising concerns to the company’s board, to regulators, and an appropriate independent organization with relevant expertise exists, we accept that concerns should be raised through such a process initially. However, as long as such a process does not exist, current and former employees should retain their freedom to report their concerns to the public.” (ibid).

As you can plainly see, there are four asks or proposed considerations.

Let’s briefly examine each one.

The first proposed precept is that an AI maker would not require a prohibition of disparagement regarding risk-related concerns, which sometimes is a clause used in an employment contract that is intended to hinder making disparaging remarks beyond the walls of the firm. An added proviso is that even if the disparagement clause was already included, the AI maker would pledge to not pursue it.

I’ll just note as an aside that this would all necessitate some heavy lawyering in terms of hammering out the details. For example, the idea of “risk-related concerns” is currently not stipulated as pertaining solely to AI, thus, in theory, this allows for the covered employee to use this for any risk-related concerns. Is that fair? Is that the intention?

Another vague notion is what constitutes a risk-related concern. It could be that an AI researcher thinks something is consequential, but it is actually an inconsequential risk-related concern. Anyway, they invoke this proviso, protecting them from hardship. You can imagine even the fairest of employers worrying that this loosey-goosey facet might flood them with unfettered attacks of a false or mountain-out-of-a-molehill caliber.

I’m not knocking the concept, and only noting that this ultimately needs to be mindfully and with great due figured out. The language currently conveyed has lots of loopholes, gotchas, and the like.

This isn’t surprising since this is simply a brief letter to presumably get the ball rolling. If they had gone out the gate with a fifty-page treatise on the specifics, few would have read it and few would have likely been willing to prod the letter into a viral discussion online and offline.

Moving on, the second precept is that a means and mechanism to allow for the reporting of anonymous but verifiable feedback by current and former employees on risk-related concerns should be established. This would be fed to the board of directors of the AI maker and regulators and “appropriate” independent organizations.

I don’t want to beat a dead horse so I’ll just note that again this is language that will need tightening.

Suppose for example that the feedback that is given to regulators, and these chosen independent entities (who selects them?), contains proprietary info or intellectual property (IP) that is closely protected and considered the crown jewels of the AI maker. Who will decide what is allowed to be spread around? How will the shared info be protected? The next precept mentions this issue, though this one seems not to mention it.

The third precept is that the AI maker will overtly foster a company culture that is demonstrably receptive to open criticism and that, in addition, risk-related concerns about “its technologies” (which presumably encompasses AI, and perhaps the high-tech proverbial kitchen sink too), can be raised to the public, the board, regulators, and “appropriate” independent organizations. This is to be allowed for current and former employees. Trade secrets and IP are to be protected too.

Lots of fuzzy wording there, but the gist is relatively apparent.

The fourth and final proviso is that the AI maker won’t retaliate against such sharing of risk-related “confidential information” if the preceding approaches were pursued but “failed”. The notion or idea of failure is going to be a mouthful to nail down. For example, if the other processes were tried, and no one along the line felt that the alleged AI risk-related concern was of any value, could this be labeled as a “failure” in that the person was not able to get their perceived sense of satisfaction?

As noted, please realize I am merely observing that these proposals or precepts will necessitate a likely lengthy give-and-take as to the wording. The wording will need to be crafted to ensure that suitable vetting takes place. Not all whistleblowers are necessarily making valid whistleblowing claims. Likewise, not all AI makers will necessarily abide by the spirit of these kinds of stipulations.

Suitable checks and balances will be needed.

Conclusion

The tug of war is underway, and you can expect to witness a lot more of these types of pronouncements.

A cynic would say that these do little good and are nothing more than tilting at windmills. An optimist would say that you have to start somewhere, and someone has to make these issues known. A pessimist might say that no matter what we try, AI doomsday is coming, and the rest of this is simply moving around the chairs on the deck of the Titanic.

It is an all-hands-on-deck consideration right now since we don’t know if there are icebergs ahead, though some believe there are (without even seeing them), and we don’t know if we will strike an iceberg (if they are around).

Let’s keep our eyes and ears wide open about AI advances, for the betterment of humankind.