I think I solved the biggest educational challenge of our time, namely:

How do we recognize AI generated text from human-created ones?

Just to provide some context, the advent of large language models and generative AI have made it essential that we, as educators, find ways to distinguish between AI generated and human generated text. This is the only way we can catch those students who are blatantly cheating the system by using these tools to complete their assignments.

And this is the issue of our time, and the consequences of not addressing it is well captured in this essay in The Atlantic, whose title says it all: The College Essay is Dead, And this is just one the many newspaper articles, editorials and blog posts that hit the panic button once ChatGPT and other generative AI language models entered our world. And of course, we have a range of responses to this perceived crisis. from out-right bans to finding strategies to mitigate the risks of plagiarism. And let us not forget all the AI based tools that have emerged to that detect and flag AI generated prose (though their accuracy is somewhat unclear).

Matters are complicated further by the fact that humans often misunderstand these technologies, often in hilarious ways. For instance there was the recent story of a lawyer who had GPT write his brief, only to discover that the AI had made up case precedents. These things hallucinate.

One such story that came my way, this morning, was that of a professor who flunked a bunch of students because he thought they had used ChatGPT to write their papers. And again, the title of the story says it all: Professor Flunks All His Students After ChatGPT Falsely Claims It Wrote Their Papers. Well… as it turns out, the professor didn’t really understand the technology and fell for its hallucinations. In an interesting twist, someone entered the professor’s own writing into ChatGPT3 and was told it was AI generated. As the Rolling Stone story reports it:

In an amusing wrinkle, Mumm’s claims appear to be undercut by a simple experiment using ChatGPT. On Tuesday, redditor Delicious_Village112 found an abstract of Mumm’s doctoral dissertation on pig farming and submitted a section of that paper to the bot, asking if it might have written the paragraph. “Yes, the passage you shared could indeed have been generated by a language model like ChatGPT, given the right prompt,” the program answered.

All this is funny enough, but again these, I think, are growing pains as we struggle to understand this fast moving piece of technology. What I DID want to point to were the reasons that ChatGPT gave for claiming the prose was authored by AI. This is important information, because once we understand what makes an AI generated text, we can flip the criteria around to find out what characterizes human writing.

Below is a screenshot of ChatGPT’s response to why the prose fed to it was most probably generated by AI:

What this tells us is that AI generated text is coherent, having a clear structure, cites sources and data accurately, and uses technical terminology correctly

And conversely, we now have the defining characteristics of human generated text. Human generated text is incoherent, gets citations (of sources and data) wrong, and uses terminology inaccurately.

There we go. I am so glad we figured THAT out.

And yes, you are welcome!