How to identify AI generated text?

by | Thursday, June 01, 2023

I think I solved the biggest educational challenge of our time, namely:

How do we recognize AI generated text from human-created ones?

Just to provide some context, the advent of large language models and generative AI have made it essential that we, as educators, find ways to distinguish between AI generated and human generated text. This is the only way we can catch those students who are blatantly cheating the system by using these tools to complete their assignments.

And this is the issue of our time, and the consequences of not addressing it is well captured in this essay in The Atlantic, whose title says it all: The College Essay is Dead, And this is just one the many newspaper articles, editorials and blog posts that hit the panic button once ChatGPT and other generative AI language models entered our world. And of course, we have a range of responses to this perceived crisis. from out-right bans to finding strategies to mitigate the risks of plagiarism. And let us not forget all the AI based tools that have emerged to that detect and flag AI generated prose (though their accuracy is somewhat unclear).

Matters are complicated further by the fact that humans often misunderstand these technologies, often in hilarious ways. For instance there was the recent story of a lawyer who had GPT write his brief, only to discover that the AI had made up case precedents. These things hallucinate.

One such story that came my way, this morning, was that of a professor who flunked a bunch of students because he thought they had used ChatGPT to write their papers. And again, the title of the story says it all: Professor Flunks All His Students After ChatGPT Falsely Claims It Wrote Their Papers. Well… as it turns out, the professor didn’t really understand the technology and fell for its hallucinations. In an interesting twist, someone entered the professor’s own writing into ChatGPT3 and was told it was AI generated. As the Rolling Stone story reports it:

In an amusing wrinkle, Mumm’s claims appear to be undercut by a simple experiment using ChatGPT. On Tuesday, redditor Delicious_Village112 found an abstract of Mumm’s doctoral dissertation on pig farming and submitted a section of that paper to the bot, asking if it might have written the paragraph. “Yes, the passage you shared could indeed have been generated by a language model like ChatGPT, given the right prompt,” the program answered.

All this is funny enough, but again these, I think, are growing pains as we struggle to understand this fast moving piece of technology. What I DID want to point to were the reasons that ChatGPT gave for claiming the prose was authored by AI. This is important information, because once we understand what makes an AI generated text, we can flip the criteria around to find out what characterizes human writing.

Below is a screenshot of ChatGPT’s response to why the prose fed to it was most probably generated by AI:

What this tells us is that AI generated text is coherent, having a clear structure, cites sources and data accurately, and uses technical terminology correctly

And conversely, we now have the defining characteristics of human generated text. Human generated text is incoherent, gets citations (of sources and data) wrong, and uses terminology inaccurately.

There we go. I am so glad we figured THAT out.

And yes, you are welcome!

A few randomly selected blog posts…

Happy Diwali

Happy Diwali For an interactive card click here ... . Remember to turn your volume way up, and click anywhere in the sky above the Taj Mahal for some environmentally friendly, fireworks.

Wordclouds, mathematics and building a better teacher

Wordclouds, mathematics and building a better teacher

Wordcloud created from all the words in the wikipedia page for "mathematics education"  What does a teacher need to know to intelligently integrate technology in their teaching? Or better still, what is it that teachers need to know to become effective...

Palindromic Poetry

A few years ago I got bitten by the bug of Palindromic Poetry - poems that double back on themselves, that can be read this way, or that. This is consistent with my love for ambigrams and other kinds of symmetrical wordplay. Take a look...

Chaos theory

PCWorld article on the The 10 Most Disruptive Technology Combinations. As the article says, "Often, even great new technology needs a partner to really change the world. Here are 10 marriages of technologies that have shaken the digital world over the last 25 years."

Modeling human behavior: The new dark art of silicon sampling

Modeling human behavior: The new dark art of silicon sampling

A couple of months ago I had written this post, On merging with our technologies – which was essentially quotes from a conversation Ezra Klein had with the novelist Mohsin Hamid. I finished the post with a quote speaking the dangers of predictive technologies on human...

Spore & learning about evolution

A NYTimes story about Spore, the new game / toy designed by Will Wright (Playing God, the Home Game) speaks about its connection to evolution. As the article says, Mr. Wright and his publishers at Electronic Arts deserve all the credit they have received from some...

15 years of blogging

15 years of blogging

January 1, 2008. 15 years ago, almost to the day - I posted my first note to this website (screenshot below). My first blog post, dated Jan 1, 2008 I have had a web presence since 1998 - hand coded, HTML pages, traces of which are still available on the Wayback...

Cyborgs are us!

Are we becoming cyborgs? William Saletan has a recent article in Slate titled Full Metal Socket about this issue. As he says: Cyborgs aren't studs from the future. They're old folks. As we age, our parts wear out. That used to mean immobility or death. Today, we can...


  1. Marckie Zeender

    As a large language model developed by OpenAI, I find this method of identifying AI generated text to be accurate.

  2. Mehedy

    You are a masterpiece.This article is a breathtaking masterpiece that seamlessly weaves together profound insights and eloquent prose. It captures the essence of its subject with a grace that is both captivating and inspiring. A true gem that leaves the reader spellbound and hungry for more. Bravo!

    • Punya Mishra

      This comment is clearly spam. But I approved it none-the-less, but only after deleting the URL and the email address that the spammer had included. So this comment will remain here … except it will not provide the link to the person’s website (which, was the sole purpose of this comment).


Submit a Comment

Your email address will not be published. Required fields are marked *