Why AI detectors disagree, and how to read their results
You upload the same picture to two AI detectors. One says it is almost certainly AI. The other says it is almost certainly real. This happens often, and it does not mean one tool is broken. This guide explains why detectors disagree and how to make sense of their results.
1. What a detector score means
Most AI detectors give a score or a percentage. Sightengine, for example, describes its score as the model's confidence, from 0 to 1, that an image was produced or edited by a generative AI model.
So a score of 0.9 means the tool is fairly sure the file looks like the AI examples it has learned from. It does not mean that 90% of the picture was made with AI. It is also not proof. Each tool sets its own scale and its own idea of what counts as "likely", so a 70% from one tool and a 70% from another are not the same thing.
2. Each tool learned from different examples
Most detectors are trained on large sets of real and AI-made files. They learn to spot patterns that the AI examples share. A tool can only recognise what looks like the material it was trained on.
- New generators. When a new image or video model comes out, a detector may not recognise its output until it is updated. Sightengine says its model covers current generators and that more are added as they appear.
- Lab tests and real life. Researchers who built the Deepfake-Eval-2024 benchmark collected deepfakes that circulated online in 2024. They found that the older academic test sets were out of date. Open-source detectors did far worse on real-world fakes than on those test sets. Commercial tools did better, but still did not match human forensic analysts.
Two tools trained on different examples can look at the same file and reach opposite conclusions. Both are reporting honestly what their own model sees.
3. The file you check matters
Detectors look at fine detail in the pixels or the sound. Anything that throws detail away makes their job harder.
- Screenshots, crops and resized copies lose information.
- Social media and messaging apps compress files when they are uploaded.
- A video downloaded from a repost may have been compressed several times.
In a 2023 test, Bellingcat found that the detector AI or Not correctly flagged all 100 original Midjourney images it was given. When ten of them were compressed to a smaller file size, it called seven photorealistic AI images the work of a human. Bellingcat concluded that the larger the file, the more accurate the detector. Sightengine also says its confidence may drop on heavily degraded images and on screenshots.
If two people check different copies of the same picture, they may get different answers for this reason alone. Always use the best copy you can find. Our guide on reverse image search shows how to find larger, earlier copies.
4. Watermark checkers answer a narrower question
Some tools do not guess from patterns at all. They look for a hidden watermark that an AI company placed in its own output. Google's SynthID Detector checks for watermarks from Google and its partners, including OpenAI, NVIDIA and Kakao. OpenAI's own checker covers content from its tools.
These tools answer a different question from a general detector. They can tell you a watermark from a particular company was found. They cannot tell you about AI tools that do not use that watermark. OpenAI says that a "not detected" result is an absence of evidence, and not proof that a person made the content.
So a watermark checker can find nothing while a general detector gives a high AI score, and both can be right. The picture may have come from a tool that adds no watermark. See what Content Credentials are for a related kind of record.
5. Detectors are wrong in both directions
Detectors miss AI content, and they also flag real content as AI.
- In Bellingcat's test, AI or Not wrongly flagged six of 20 real photography contest entries as AI-generated.
- In October 2026 Full Fact found that Google's public SynthID Detector labelled five images as made or edited with Google AI. The people who posted them said they were genuine, and Full Fact saw no signs of editing. After Google updated the tool, it rated those five as "Unsure". Google said that using the detector without an existing reason to suspect AI raises the chance of false positives.
- With text, a 2023 Stanford study found that AI text detectors consistently misclassified writing by non-native English speakers as AI-generated. Our guide on AI-written text covers this.
A high AI score is not proof that something is fake. A low one is not proof that it is real. Treat every score as one clue among several.
6. Why RELTY never averages scores
Suppose one tool gives 95% AI and another gives 5%. The average is 50%, which looks like a fair middle answer. It hides the most useful thing you have learned, which is that the tools strongly disagree.
Averaging also treats every tool as equally good at every kind of file. They are not. One may be strong on faces and weak on landscapes. Another may handle compressed files better. Only the separate results let you see that.
That is why RELTY's reports will show each tool's result on its own, with what it checks and what it cannot see. When tools disagree, the report will say so. The illustrative sample report on the home page shows the idea.
7. How to read several results
- Write each result down separately. Note the tool, the score and the exact file you used.
- Check what each tool claims to detect. Is it a general detector, a face-swap detector or a watermark checker? Read its own description of its limits.
- Note the file quality. A low score on a blurry screenshot means much less than a low score on the original file.
- Look for agreement with other evidence. A high AI score counts for more when the source is unknown, there are no earlier copies and the picture has visible flaws.
- Accept "unclear". When the tools disagree and nothing else settles it, the honest answer is that you do not know.
The checks that do not depend on detectors often decide the case. Finding who posted it first, searching for earlier copies and reading existing fact-checks are covered in our guides to AI images and viral videos.
8. Decide what you can say
- Verified. A reliable source, Content Credentials or independent reports support the content. A low AI score alone is not enough.
- AI-generated or altered. A watermark, the creator, a platform label or several strong pieces of evidence point the same way. A high AI score alone is not enough.
- Misleading context. The content is genuine, so detectors find nothing, but it is being used to tell a false story.
- Unverified. The tools disagree, or the scores are the only evidence you have.
If you cannot verify something, say so, and do not share it as fact.
In the news
Recent reports from RELTY's AI news that show why this check matters.
- 3 November 2026 · The Hindu
New light-based technique spots deepfake videos
The Hindu reports a system that uses light to detect manipulated video. In tests it identified fakes with 99.86% accuracy and real videos with 95.72% accuracy. A tool can be better at one kind of answer than the other, so look at both figures.
Sources
The pages this guide relies on, checked on 8 November 2026.
- AI-generated image detection. Sightengine documentation.
- Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024. Chandra and others, 2025 (arXiv).
- Testing AI or Not: How well does an AI image detector do its job?. Bellingcat, September 2023.
- SynthID. Google DeepMind.
- Content provenance. OpenAI developer documentation.
- Google updates AI detector after Full Fact investigation finds it flagging apparently real images. Full Fact, October 2026.
- GPT detectors are biased against non-native English writers. Liang, Yuksekgonul, Mao, Wu and Zou, Stanford University, 2023 (arXiv).
How to tell if text was written by AI
How to spot fake reviews and listings made with AI
What are Content Credentials and how do you check them?
How to tell if an image is AI-generated
Browse the RELTY directory of tools
Links to other organisations are for reference only. RELTY has no partnership with, or endorsement from, any of them.
