OpenAI can’t detect AI-generated content after all; shut down its AI Classifier tool due to low accuracy rate

You’ve probably read that AI can detect AI-generated content. As it turns out, OpenAI, the AI company that took the Internet by storm with its popular chatbot tool, can’t even tell if something was written by AI or a human.

On July 20, OpenAI shut down its AI Classifier due to its consistently low accuracy rate. The AI classifier was launched in January with the aim of assisting educators, journalists, and researchers in detecting AI-generated content.

“As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy. We are working to incorporate feedback and are currently researching more effective provenance techniques for text, and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated,” read a post on the blog that earlier introduced the AI Classifier.

We wrote about AI Classifier back in late January after OpenAI released the free tool that the company said would help t to “distinguish between text written by a human and text written by AIs.” At the time, OpenAI also warned that the tool was “not fully reliable.”

In a blog post, OpenAI said: “While it is impossible to reliably detect all AI-written text, we believe good classifiers can inform mitigations for false claims that AI-generated text was written by a human: for example, running automated misinformation campaigns, using AI tools for academic dishonesty, and positioning an AI chatbot as a human.”

“Our classifier is not fully reliable. In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases. Compared to our previously released classifier, this new classifier is significantly more reliable on text from more recent AI systems,” OpenAI added.

There are many ways that AI-generated text can be exploited and it’s becoming increasingly difficult to distinguish between AI and human-created content. OpenAI said that its AI classifier tool was not very effective, often yielding false positives. As a result, the company said that it’s now focusing on developing ways to assist users in identifying AI-generated audio or visual content. The goal is to provide better tools to combat misuse and enhance content verification.