Tools that promise to detect AI writing are becoming widespread in schools and media organizations, yet their reliability remains highly questionable. Despite warnings from developers and experts, many institutions are using these detectors to make decisions that affect people's reputations and careers. This rush to embrace AI detection is creating a new era of distrust, where accusations based on flawed algorithms can overshadow factual evidence.

How It Started

Long before ChatGPT became a household name, educators and editors frequently used anti-plagiarism tools to verify the originality of written work. These systems compare a manuscript against vast databases of web pages, academic papers, and published books to identify matching sentences and phrases. Tools like Turnitin assign a similarity percentage, indicating how much of a student's writing overlaps with existing sources. But even these traditional plagiarism detectors have faced criticism over false positives and ambiguity about whether duplicated passages were intentional or accidental. Some educators have already abandoned them for these reasons.

Now, the hunt for copied content has evolved into a war on AI-generated text. As quickly as students adopted ChatGPT, Google Gemini, and Microsoft Copilot, teachers adopted so-called AI detectors with equal speed. A survey from the Center for Democracy and Technology found that 43 percent of sixth to 12th grade teachers in the US regularly used AI detectors between 2024 and 2025. Some universities that already used Turnitin discovered that the service automatically enabled its AI detection feature when the tool launched in 2023, without explicit consent from instructors or students.

Unlike traditional plagiarism checkers, AI detectors like GPTZero, Pangram, and Turnitin's proprietary system do not match text against a database. Instead, they rely on their own AI models to estimate the likelihood that a piece of writing was generated by a machine. This process is arguably even murkier than matching text on the web. GPTZero explains that its algorithm analyzes a text's wording, rhythm, and structure, and looks for patterns in length and tone that appear more frequently in AI-written content. This relatively subjective evaluation is not as solid as a direct text match, and it can easily be confused by writers who use English as a second language. Despite these limitations, Turnitin claims that its AI detector falsely flags less than 1 percent of human-written content, while Pangram asserts a false positive rate of just 1 in 10,000. GPTZero similarly claims a very low rate of mistaking human content for AI.

How It's Going

People online are already accusing each other of "sounding like AI," but the ready availability of AI detection tools is only adding fuel to the LLM witch hunt. In several high-profile cases, AI writing accusations have directly impacted people's livelihoods and reputations. Last month, the publisher Minotaur dropped a $2 million book deal over concerns that its author, Jerry Falade, used AI — something he vehemently denies. The accusation alone was enough to derail his career, even though no definitive proof was presented.

There is also the case of Thierry Rignol, a French national who sued Yale University after a professor accused him of using AI to write portions of his final exam. The accusation resulted in a failing grade and a one-year suspension. The professor had used GPTZero to scan Rignol's writing for signs of AI. The lawsuit argues that AI surveillance and detection tools are known to unfairly target non-native English speakers like Rignol. In February, a student at Adelphi University won a lawsuit against the school after a professor similarly claimed he used AI to write an essay. The lawsuit did not specify which AI tool was used, but Adelphi University has a licensing agreement with Turnitin.

A 2023 Stanford study found that AI detectors falsely flagged essays written by non-native English speakers as AI more often than essays by native speakers. Many services still claim their tools are accurate when dealing with text written by non-native speakers, but the study contradicts this. These tools may also be biased against neurodivergent writers, whose unique writing patterns can be misinterpreted as machine-generated.

As the University of California, Los Angeles points out, AI detection tools are trained to pick up on patterns that could indicate AI use, such as repetitive terms and phrases, text that sounds too formal or informal, and nonsensical phrasing. Some tools, like QuillBot, also measure the "unpredictability" of text. According to UCLA, AI tends to make the most "obvious" or most common language choices compared with human-produced writing. Detectors may also flag consistent sentence structure as a sign of AI. But these measurements are not indicative of AI on their own, as some people may simply have a writing style with these qualities.

Even Turnitin, which touts low false positive rates, maintains that its tool "may not always be accurate" and should not be used to take action against a student. Grammarly warns that users "should never rely on the results of an AI detector alone," while GPTZero says "no AI detector can ever truly be 100% perfect." OpenAI even shut down its own AI writing detector in 2023 due to low accuracy. Despite these disclaimers, AI writing accusations are still being flung across the web.

Last week, in a video broadcast to the more than 3.5 million followers across his social channels, Jack Osbourne accused journalist Kat Tenbarge of using AI to write an article for Rolling Stone. He flaunted the results from an AI detector called Getsolved as "proof" of his claim. Tenbarge has refuted the claim in a video and a post on her website, but Osbourne has not retracted his accusation or deleted the video, leaving Tenbarge to deal with online trolls. This is just one of many examples of false accusations, from writers to students, with accusers often ignoring the disclaimers that come with these tools.

What Happens Next

The uncertainty surrounding AI detectors is enough for some educational institutions to stop using them altogether. Yale University, Johns Hopkins University, Vanderbilt University, Georgetown University, and others have disabled or restricted the use of AI detection tools. The Massachusetts Institute of Technology also warns that "AI detectors don't work."

Instead of relying on tools to weed out AI, many schools are encouraging educators to rethink their lessons. The University of Chicago suggests telling students to slow down their reading, breaking up longer writing assignments, and requiring students to reflect on their work. Stanford University says professors can consider holding assessments in classrooms, while MIT advises professors to leave room for students to disclose whether they used AI for help on an assignment, without penalty.

With AI becoming more prevalent in and outside the classroom, efforts to suss out what is written by a human or a machine are ramping up as well. Some online platforms are exacerbating suspicions surrounding AI use. Substack has built Pangram into its app, allowing users to scan blogs for suspected AI-generated content, while LinkedIn added a "seems like AI slop" button on posts. The result is a new era of distrust, where readers constantly question whether what they are reading is AI and real human writers try their best not to sound like it.

By the Way

• The Authors Guild is helping writers get ahead of accusations by giving them "Human Authored" certifications. There are also Not by AI and Written by Human badges users can add to their work online.
• Wikipedia created a guide to help editors spot AI writing, which includes looking for writing that "puffs up" the importance of a topic or provides "superficial analysis of information." The site has also banned AI-generated articles.
• The New York Times has a quiz that asks readers to look at five pairs of passages and choose which blurb they like better — the catch is that one of them is written with AI.
• Inside Higher Ed spoke to some educators about how they are approaching AI in school, with one lecturer raising concerns about the costs and resources that could go into AI-proofing assignments.
• The fanfiction community is having its own internal struggle over how to determine which stories are generated by AI, as detailed by a report from a tech news outlet.

The growing reliance on AI detectors is not just a technical issue; it is a social one. As these tools become more embedded in education, publishing, and social media, the risk of false accusations rises. The disclaimers from developers are often buried in fine print, while the confidence of accusers is loud and public. Schools are beginning to adapt by emphasizing process-oriented assignments and classroom discussions, but the broader culture is still caught in a cycle of suspicion. Until the technology improves or trust is restored through better practices, the burden of proof will continue to fall on individuals who must prove their humanity.

Source:The Verge News