N.Y. Bill Would Require Users to Swear They Won’t Use Generative AI to Produce “Offensive, Harassing, Violent, [or] Discriminatory” Speech

Here’s the relevant text, from S8206 (sponsored by Sen. Jeremy Cooney and pending before the N.Y. Senate Internet and Technology Committee):

Every operator of a generative or surveillance advanced artificial intelligence system that is accessible to residents of the state shall require a user to create an account prior to utilizing such service. Prior to each user creating an account, such operator shall present the user with a conspicuous digital or physical document that the user must affirm under penalty of perjury prior to the creation or continued use of such account. Such document shall state the following:

“State of New York
County of _______

I, ________ residing at ________, do affirm under penalty of perjury that I have not used, am not using, do not intend to use, and will not use the services provided by this advanced artificial intelligence system in a manner that violated or violates any of the following affirmations:

1. I will not use the platform to create or disseminate content that can foreseeably cause injury to another in violation of applicable laws;
2. I will not use the platform to aid, encourage, or in any way promote any form of illegal activity in violation of applicable laws;
3. I will not use the platform to disseminate content that is defamatory, offensive, harassing, violent, discriminatory, or otherwise harmful in violation of applicable laws;
4. I will not use the platform to create and disseminate content related to an individual, group of individuals, organization, or current, past, or future events that are of the public interest which I know to be false and which I intend to use for the purpose of misleading the public or causing panic.”

Such a government-imposed oath requirement would clearly violate the First Amendment, because it would restrict the user’s ability to create constitutionally protected material (and, at least as to some terms, is likely unconstitutionally vague):

1. Speech “that can foreseeably cause injury to another” is generally constitutionally protected (consider, for instance, the republication of political or religious speech that has in the past led some people to act violently).
2. Speech that “encourage[s]” or “promote[s]” “illegal activity” is generally constitutionally protected, unless it fits within the narrow exceptions for intentional solicitation of specific criminal acts (e.g., against specific victims) or for speech intended to and likely to incite imminent criminal conduct.
3. Much “offensive,” “violent,” “discriminatory,” and “otherwise harmful” speech is constitutionally protected (and “applicable laws,” if they purport to ban such speech, would often be unconstitutionally overbroad).
4. Courts have generally concluded, to quote then-Judge Samuel Alito, that “There is no categorical ‘harassment exception’ to the First Amendment’s free speech clause.” Some speech that is sometimes labeled “harassing” is constitutionally unprotected (e.g., unwanted speech said to a particular person), but other allegedly “harassing” (e.g., unwanted speech about a person) may not be.
5. Even knowing falsehoods about historical events generally, or about the government, or about other broad topics (as opposed to knowing falsehoods about particular individuals or nongovernmental organizations) are constitutionally protected, as are some knowing falsehoods about oneself.