How to spot AI-generated text?

When you receive an email or document written by somebody it can be hard to work out if they use generative AI. There can be giveaways. For example, if that individual has written a document in a different style from their usual writing. Occasionally I have received messages with the prompts left in, such as “I really enjoyed working with you on [insert name of project]”. Around the world, students are finding themselves accused of using AI to write their dissertations, and finding themselves in quasi-legal proceedings in their university where it is impossible to prove conclusively whether generative AI was used to write a document.

This has led me to wonder if there is any way of knowing for certain if a text is AI generated.

Introduction to forensic stylometry

Before we talk about detecting AI written text, let’s go back in time to 2018, before generative AI took over the world, and discuss how we can detect which human wrote a document. Identifying the written style of a document and attributing the author is a field called forensic stylometry.

I first came across forensic stylometry when I was preparing a workshop at the Digital Humanities Summer School in Oxford University in 2018. At that time, I met a number of academics who were researching questions such as whether a particular Greek or Roman poet was likely to have authored a set of unattributed poems, and JK Rowling had recently been in the news for her “unmasking” as Robert Galbraith, the nom de plume she chose for her Cormoran Strike series of detective novels.

The best known formula for author attribution is an algorithm called Burrows’ Delta [2]. The Burrows’ Delta formula is very simple and predates any kind of neural network, and has long been used for author attribution in the humanities. It relies on taking common function words such as “if”, “the”, “over”, and removing content words, and constructing a ‘fingerprint’ of a given author’s style. This fingerprint can be compared to that of an unknown text to identify the most likely author.

Given enough text by an author, Burrow’s Delta can be pretty accurate. For example, when you feed it entire books from Project Gutenberg, it can reliably identify which of the Brontë sisters authored a book that it hasn’t seen before.

Since most of the tools to use Burrow’s Delta were in R, I felt we needed a Python library to do this. I also found that there was no library which gave me a simple probability so that I could say “we are 60% sure that JK Rowling wrote this text”. So I developed a Python library called Fast Stylometry, which can be fed a set of documents by different authors, and identifies who is the most likely author of a new, unseen, document. It calculates the Burrows’ Delta score of the document for each candidate author, and then converts these to probabilities in the range between 0 and 1. You can try Fast Stylometry here in your browser.

Since developing Fast Stylometry, a number of times I have been asked by people and organisations to identify candidate authors of texts. Sometimes these are historic documents of academic interest. These are invariably short documents, perhaps emails, or at the most they are documents consisting of a couple of pages. I have found that the Burrows’ Delta analysis is nearly always inconclusive for such short documents. There is no way near as much information or personalised features in a typed document as there would be in an image of a handwritten document, where you have every flourish of the author’s pen.

I think it would be promising to try using LLMs for authorship attribution and forensic stylometry. An LLM is very good at predicting the next word in a sentence based on the text it’s been trained on. If a stylometry algorithm could recognise where a text deviates from the LLM’s best guess, that could be used to form a fingerprint of the author. I have experimented with LLMs for this purpose but not managed to achieve anything which would manage to identify the author of a short email. This is slightly frustrating, as I feel that I can recognise stylistic traits in emails or text messages much shorter than any stylometry algorithm is able to achieve.

Features of generative AI authored text

Undeclared AI edits: the scourge of Wikipedia!

So now I would like to talk about the related problem of AI detection. Interestingly, this has been such as nuisance for Wikipedia’s editorial team, that they have put together a meta-article titled Signs of AI writing.

I found the Wikipedia checklist a fascinating read. Some of the features that I had already noticed are mentioned, but there are many more which I had not thought of. Wikipedians regularly submit article drafts for approval, and the policy is that edits with undeclared use of AI are nominated for speedy deletion.

Very strong indicators of AI-authored text

Wikipedia uses three very objective indicators of AI written text. The guide proposes that an edit with one of these features is considered generative AI and should be deleted immediately, as these are very strong indicators.

1. Communication intended for the user. These are instructions or disclaimers that ChatGPT intended for the user, which the user accidentally pasted into their final submission. Example:

Certainly. Here’s a draft Wikipedia-style article for Mark Biram, written in a neutral, encyclopedic tone and formatted according to Wikipedia conventions. This assumes notability is supported by independent sources (which would need to be cited for a real Wikipedia page): [source: Wikipedia]

2. Broken external links. Although it is expected that over time, a number of the outgoing links in Wikipedia become invalid, as websites get taken down or re-organised, a brand new edit should not contain broken links. LLMs are prone to hallucinating URLs, and an LLM authored text often contains URLs that clearly never existed.

3. Fictitious references. AI tends to make up very impressive sounding references to non-existent journal articles. Example:

2. M. E. Van Valkenburg, “The validity and limitations of Ohm’s law in non-linear circuits,” Proceedings of the IEEE, vol. 62, no. 6, pp. 769–770, Jun. 1974. doi:10.1109/PROC.1974.9547 [source: Wikipedia] - here the citation does not exist at all!

Incidentally, besides wasting Wikipedia editors’ time, hallucinated citations are becoming a major nuisance in legal systems around the world. In the UK, cuts to the legal aid budget have resulted in an unprecedented number of litigants in person, people who represent themselves in court without a lawyer. Litigants in person, and occasionally lawyers who should know better, have been submitting documents to courts which are full of fictitions citations. This has sometimes resulted in fines or cost awards.

Weaker indicators of AI-authored text

The Wikipedia guide goes on to list a number of other indicators that an edit was authored by AI, or with the help of AI. Unfortunately, these other indicators are nowhere near as clear-cut as the three rather damning indicators listed above.

I have listed just a few, as the list is very long.

• Tone - LLMs often puff up the significance of the subject matter, over-using phrases like “further enhancing its significance as…”, “stands as a symbol of…”, and so on.
• Promotional language, including what I would call oddly clichéd metaphors - words like “rich tapestry”, “breathtaking”, and the like. I feel that I have read the word “tapestry” more often in the last three years than ever before, so I find this word in particular quite jarring when I come across it.
• Incorrect formatting - Wikipedia uses a markup language called wikitext, which is distinct from Markdown, the commonest markup language on the internet. Generative AI is trained on a large amount of text in Markdown. For example, in Markdown, links look like this: [anchor text](https://example.com) and section headings look like this: ## Section heading, whereas in wikitext, section headings are marked with =.
• Editorialising text - “A defining feature of…”, etc. - this is not considered appropriate for encyclopaedic content.
• Overuse of certain conjunctions such as “furthermore” and “however”
• Section summaries - LLMs often begin a paragraph with “in summary” - they seem unable to write articles without this approach!

However, some of these are harder to define. If GPT was modified to no longer include emojis, it would be harder to spot AI generated text, but this would be a superficial fix. One of the clearer indicators that a text is AI generated is vacuous and meaningless content, and that cannot be easily identified with a checklist.

What is interesting is that the Wikipedia editorial team was able to put together this list, because they can see which edits were submitted and hastily retracted or deleted by the author. Sometimes an author will submit a change to Wikipedia, then notice that they forgot to delete ChatGPT’s instructions to the user, then retract their submission… but it leaves an electronic footprint.

Detecting AI authorship with an algorithm

Now we have established that it’s possible for a human to recognise AI generated text, what can we achieve with algorithms?

There are a number of free and paid generative AI detection software tools out there, such as QuillBot, Winston AI, and Detecting-AI.com.

A while ago, I tried to make a generative AI detector using a large LLM - basically an LLM-detection version of the Fast Stylometry library. You can try it online - although it doesn’t work very well and is not suitable for drawing any conclusions.

My detector works by putting the text through GPT, and seeing what GPT’s prediction would be for the next token (word) - then comparing that with the submitted text. So a generative AI-written text would exhibit few surprises - the next token would consistently be what GPT would expect, or close to it, with few deviations. I believe that this is how the commercial AI detectors work. As far as I have been able to find out, they measure two numbers called burstiness and perplexity. Perplexity is a number that is would be low if a word is what the generative AI would expect, and high when a word is unexpected. So, in the sentence “Fast Data Science is a consultancy in natural language _”, if the next token is “processing”, this would be a low perplexity, and if the next word is “potato”, this would be a high perplexity. Burstiness is a number that measures how much perplexity varies over the document.

From what I can tell, even though the AI detectors are better than a roll of a die at detecting AI-authored texts, they are far from foolproof. It’s also possible to superficially edit an AI-authored text and easily fool the detectors. The only near-certain indicators would be some of the stronger items on Wikipedia’s checklist.

References

1. Wood, T.A., Fast Stylometry [Computer software] (1.0.4). Data Science Ltd. DOI: 10.5281/zenodo.11096941, accessed at https://fastdatascience.com/fast-stylometry-python-library, Fast Data Science (2024)

2. Burrows, J. 2002 . ‘Delta’: A measure of stylistic difference and a guide to likely authorship . Literary and Linguistic Computing , 17 ( 3 ) : 267 – 287 .