Source: The Conversation (Au and NZ)
Since artificial intelligence (AI) software ChatGPT was released in 2022, teachers have been looking for ways to tell when AI has been used to generate text in assessments. AI-detection software is still sold by some tech companies, such as Pangram, as a potential solution.
This month, the New South Wales Education Standards Authority updated its rules to stop schools from relying on AI-detection software when deciding if a student has cheated. The reasoning is that these tools are not always reliable.
While most Australian unis have have already shifted away from AI-detection software in some form, my research suggests the education sector should go one step further, and not use AI-detection tools at all.
Where are unis at?
Unis each have their own rules around how they use AI-detection software, or whether they use it at all.
Some unis, such as the University of Melbourne, the University of Sydney and the University of NSW, use AI-detection tools in some form to assist in flagging potential cases of misconduct.
Other unis, such as Curtin University and the University of Queensland, have opted to not use AI-detection tools at all.
Not always reliable
The reason is because this software is not always accurate.
When AI-detection tools incorrectly flag a student’s work as including AI-generated text, this is called a false positive. AI companies want to develop detectors that have low false-positive rates.
For instance, Turnitin, a United States-based company that sells AI-detection software, promotes a false-positive rate of less than 1%. However, Turnitin says this software is only reliable in detecting AI use when documents contain more than 20% AI-generated content.
Impacts on students
A small false-positive rate can still lead to big problems.
In 2023, academic integrity expert Phillip Dawson
estimated that even a 0.7% false-positive rate would result in over 100,000 assessments being wrongly flagged as AI generated each year in the Australian higher education sector alone.
This happened on a smaller scale at the Australian Catholic University, which referred nearly 6,000 students for alleged academic misconduct in 2024, about 90% of them over AI use. The ABC reported last year the uni later dismissed any cases where the evidence for misconduct was solely based on Turnitin’s AI-detection technology.
Turnitin states its software should not be used in isolation to decide whether a student has used AI improperly:
Rather, we provide data for educators to make an informed decision based on their academic and institutional policies.
But this still has an impact on students’ wellbeing. Research shows false accusations of academic misconduct harm students and cause significant stress and anxiety.
Why aren’t detection tools more accurate?
Companies measure the accuracy of AI-detection software under controlled conditions. To calculate the false positive rate, companies test text they know was AI generated and text they know was entirely human written. Using these texts, they then measure how often the detector gets it right or wrong.
However, we don’t really know all the details of how companies test their AI-detection software.
Testing doesn’t translate well
My research with colleagues, published in the Journal of Higher Education Policy and Management, suggests these controlled tests do not translate well when AI detectors are used in real world contexts, such as in unis.
First, detection tools attempt to distinguish between AI-generated text and human-written text. However, this is not always so straightforward, as anything produced by AI could plausibly have been written by a human depending on their writing style. Students may also submit work that has been written with the assistance of AI, rather than sections of text simply generated by it.
Second, even if a teacher uses an AI detector with a very low false-positive rate, it still doesn’t tell them the chance an individual student’s assessment is AI generated.
This is because a 1% false positive rate means one wrong flag for every hundred papers that humans wrote. So if you know how many human-written papers there are, you know how many to expect to be falsely flagged. But in the real world, where students complete assessments outside the classroom, we can never know in advance how many papers are human written — that’s why we’re using an AI detector in the first place — so we can’t know what the chances are that the flag is correct.
Multiple AI tools are not a solution
It seems like using more than one AI detector would increase the chances of an accurate result. But AI detectors are all built on the same assumption that humans and AI write differently. Agreement between multiple AI detectors doesn’t mean they are right. It only means they made the same mistake.
False negatives are also a problem. This is when a detector fails to identify text where AI has been used. These mistakes get far less attention than false positives, and are a big problem for educators when determining whether a student has cheated.
What’s the alternative?
The question assumes there must be a technical solution that allows us to continue using unsupervised written assessments as the sole basis for judging student learning. But not every unreliable technology needs a replacement. The same applies to AI watermarking, which has its own limitations.
Instead of searching for signs of cheating, teachers should look at whether their students have learned what an assessment was designed to measure, and if the work they submit provides valid evidence of that learning.
Rather than attempting to make all assessments AI-proof, they can identify a number of points across a course where it matters most that students demonstrate key outcomes under supervision, through exams, practicals, presentations or interviews.
It’s an approach supported by Australia’s higher education regulator that leaves room for students to use AI where it suits the task, and to learn how to use it well. None of this depends on software guessing how a piece of text was written.
Read more:
Does ‘AI-watermarking’ mean the party is over for cheating students?
Mark A. Bassett does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.
Original source: https://analysis1.mil-osi.com/2026/09/23/unis-and-schools-are-moving-away-from-ai-detection-software-they-should-stop-using-it-altogether/