Study Looks to Distinguish Between Human and AI-Generated Content

A recent study conducted by Stony Brook University researchers looked at the reviews of 20 hotels in Chicago, trying to distinguish between the experiences shared by humans on TripAdvisor and those generated by OpenAI’s ChatGPT.

Even though Generative AI tools like ChatGPT and Claude have only been around for a couple of years, the content they create is so good in quality that it’s often indistinguishable from text written by humans. What’s more, apart from being used for daily tasks and language translation, these AI models are also learning about human nature by comparing their own outputs to human-generated content.

So how do we differentiate between human and AI-generated content? According to Vasudha Varadarajan, PhD student and research assistant at Stony Brook, the approach needs to be different from current methods. Instead of trying to understand how the AI model was designed, we should focus on its output. “This approach,” she adds, “allows us to study the language differences between human and AI texts rather than get into the complexities of how the AI model works.”

Varadarajan explored this idea alongside fellow PhD students Siddharth Mangalik and Nikita Soni and H. Andrew Schwartz, Director of HLAB (Human Language Analysis Beings) and an associate professor in the Department of Computer Science. They looked at studies that propose using emotional, personality, and demographic cues (like age and gender), to differentiate text generated by humans vs advanced language models.

According to Siddharth, “Our logic is based on studies of bots which show that they have very little variation in emotionality, personality, and demographics. Bots only show positive emotional sentiments, when compared to human authors.”

Read the full story by Ankita Nagpal at the Institute for AI-Driven Discovery and Innovation website.