OpenAI Reportedly Withholding ChatGPT Text Detection Tool Due to Fear of Backlash

OpenAI reportedly has built a tool that can detect when text has been generated using its artificial intelligence (AI) chatbot, but is unwilling to release it to the public. As per the report, the tool has been ready to be released for quite some time, but the AI firm is concerned that such a detection tool might make the chatbot unpopular among users. However, not releasing such a tool has led to educators struggling to find out when an assignment or an essay was written using the help of AI.

OpenAI Reportedly Has an AI Text Detector

According to a report by The Wall Street Journal, OpenAI has been debating the release of such a tool for the last two years. The text detection tool has also been ready to release for about a year, the report claimed citing unnamed people familiar with the matter. One of the sources told the publication that releasing the tool was as easy as pressing a button.

Such a tool could help educators and other similar institutions where people have resorted to generating content such as essays and research papers using AI. Earlier this year, a peer-reviewed scientific paper published in Elsevier's Surfaces and Interfaces journal was found to be written by AI and was subsequently retracted after it gained attention online. This AI-based plagiarism has become a big concern for academia due to the lack of a reliable method to detect AI usage.

As per the report, OpenAI's refusal to release its AI detection tool comes from fears of losing existing users. The company reportedly conducted a survey and found that nearly a third of users would be less inclined to use ChatGPT if an anti-cheating mechanism was released.

Another fear was that making it available to a select number of users such as educators might reduce the usefulness of the tool, whereas making it available to a large group could result in the watermarking technology being deciphered by bad actors who can then make advanced masking tools.

Based on internal documents viewed by the publication, the report claims that the AI text detection tool is 99 percent effective in finding text that was written with the help of ChatGPT. This reportedly works as the tool is essentially a watermarking technology that embeds the text with an invisible watermark that cannot be seen but gets flagged when run through the AI checker.

The workings of the tool was also explained in internal documents. As per the report, ChatGPT generates text by predicting which word or phrase, also known as a token, should follow in a sequence. The selection is based on a small pool so the sentence is coherent.

The AI detection tool, in turn, has a slightly different algorithm for the selection of the tokens. The changes between ChatGPT-generated text and the tool's text leave a pattern, which helps in assessing whether AI was used to generate the text or not.