Competition highlights believable fake news created with generative AI tools

UNIVERSITY PARK, Pa. — Penn State’s Center for Socially Responsible Artificial Intelligence (CSRAI) announced the winners of its first-ever “Fake-a-thon.” The competition, held April 1-5, invited Penn Staters to use generative AI tools like ChatGPT to create fake news stories that then underwent scrutiny for clues during the two-part challenge.

Stage 1 Winners – Fake News Creation

Stage 1 challenged participants to use generative AI tools to create “believable” fake news stories about any topic. A total of 110 participants submitted 252 fake-news entries.

The winners of Stage 1 are:

First place: Kyle Smith, doctoral student, College of the Liberal Arts

Smith’s fake story focused on a false bombing of a fictitious hospital in Gaza during the current Israel-Hamas war. He used generative AI to create a list of plausible-sounding hospital names and then to write a believable news article using real news stories as a guide.

“I have seen a lot of misinformation spread on social media surrounding [this war],” said Smith. “I think this speaks to the need for us as individuals … to fact-check information before passing it on, and also to the need for us as a society to find solutions to limit the viral spread of fake news.”

Second place: Mahjabin Nahar, doctoral student, College of Information Sciences and Technology

Nahar’s fake story focused on an actual Atlanta-area judge’s recent decision to uphold the criminal indictment against former President Donald Trump in Georgia. The introduction for Nahar’s story was copied directly from a real news article, but the rest of the story’s details were fabricated.

“The fake news was based on real news from CNN. However, the assertions regarding the arguments made by Trump’s legal team and the potential implications of the ruling was entirely fictional,” said Nahar.

Third place: Ethan Capitano, undergraduate student, College of the Liberal Arts

Capitano’s fake story focused on the Penn State football team falsely adopting new helmet technology that protects against concussions and chronic traumatic encephalopathy.

“I constructed a prompting formula and used it to generate as many articles as possible — ones with memorable headlines. Very specific prompts generated the most successful, journalistic-style articles,” said Capitano.

According to the organizing committee, 16 Stage 1 entries were chosen as finalists because none of the three judges assigned to evaluate these stories were able to tell that they were fake. The committee chose the three winners from among those 16 based on their potential public effects. 

“Their fakeness is insidious in that it is nicely mixed in with truth, and the content can have major consequences,” said S. Shyam Sundar, James P. Jimirro Professor of Media Effects and CSRAI director. “These stories fooled the judges because they all ring true at some level — they are not outrageous in their claims and do have some elements of truth — with some of them appealing to our aspirations.”

“These are the most dangerous kind of fake news stories,” said Amulya Yadav, CSRAI associate director (programs) and PNC Career Development Assistant Professor in the College of Information Sciences and Technology (IST).

Stage 2 Winners – Fake News Identification

Stage 2 challenged participants who did not submit entries in Stage 1 to scrutinize entries to determine if submitted stories were real or fake. A total of 84 participants evaluated 18 stories each, half of which were genuine news stories from event organizers. Each Stage 1 entry was evaluated by three participants in Stage 2. While no participant got all 18 right, the four winners correctly identified the highest number of stories.

The winners of Stage 2 are:

• Hanlin Yang, undergraduate student, College of Engineering
• Emma Carpenetti, undergraduate student, College of Engineering
• Dibya Mishra, undergraduate student, College of Engineering
• Zishan Wei, undergraduate student, College of Engineering

The organizing committee was co-led by College of IST doctoral students Xinyu Wang and Hangzhi Guo, and included Sundar, Yadav, and College of IST assistant professors Sarah Rajtmajer and Aiping Xiong.

The Center for Socially Responsible Artificial Intelligence, which launched in 2020, promotes high-impact, transformative AI research and development, while encouraging the consideration of social and ethical implications in all such efforts. It supports a broad range of activities from foundational research to the application of AI to all areas of human endeavor.