AI content detection sounds like a neat little Python project.
You paste text into a model, run one function, and get:
AI-generated: 92%
Cute, right?
Well… kind of.
The problem is that AI text detection is not as clean as it looks. Human writing can look formulaic. AI writing can be edited by humans. Non-native English writing can get unfairly flagged. Short text is hard to judge. Different detectors disagree. And a confident-looking score can still be wrong.
So the safest way to build AI content detection in Python is not:
This was written by AI.
It is more like:
This text has AI-like signals and should be reviewed.
In this guide, we’ll build a practical AI content detection workflow with Python. We’ll use simple linguistic signals, a transformer-based classifier, optional API-based detection, paragraph-level review, batch processing, and a safer report format that shows uncertainty instead of pretending the detector is magic.
What does “AI content detection” actually mean?
AI content detection means estimating whether a piece of text has patterns commonly associated with AI-generated writing.
That wording matters.
A detector usually cannot know the true writing process. Maybe a human wrote the text from scratch. Maybe AI drafted it. Maybe AI only fixed grammar. Maybe a human translated it. Maybe a student wrote it, then used AI to polish the flow. Maybe it is a company template reused 50 times.
So the goal should be:
Estimate AI-like risk.
Not:
Prove authorship.
A safer detector output looks like this:
{
“ai_risk_score”: 0.71,
“label”: “needs_review”,
“confidence”: “medium”,
“signals”: [
“Low sentence length variation”,
“Repeated generic phrases”,
“Transformer classifier returned high AI-like probability”
],
“warning”: “This result is probabilistic and should not be treated as proof.”
}
That is much better than a dramatic “AI detected” badge.
Why we can write this guide
We’ve spent around 6 years working with AI APIs, NLP systems, text classification, content workflows, and developer tutorials. We also researched current AI detection papers, Hugging Face model tooling, provenance updates, and real-world concerns around false positives.
The main lesson is simple: AI detection can be useful, but it needs humility.
A 2026 paper called Why AI-Generated Text Detection Fails found that detectors can perform well on benchmark-style data but fail under cross-domain and cross-generator shifts. The authors argue that many detectors rely on dataset-specific stylistic cues instead of stable signals of machine authorship. That is exactly why your Python detector should include warnings, segment-level review, and human confirmation instead of acting like a judge.
When should you use AI content detection?
AI detection is useful when it supports a workflow.
Good use cases:
| Use case | Safer role for detection |
| CMS content review | Flag generic or suspicious drafts |
| SEO quality checks | Find templated low-effort content |
| Community moderation | Queue likely spam for review |
| Academic integrity | Start a review, not prove misconduct |
| Hiring platforms | Flag suspicious responses carefully |
| Publishing workflows | Ask for sources or revision history |
| Customer review platforms | Detect mass-generated review spam |
| Internal content QA | Find repetitive AI-like drafts |
The key phrase is “support a workflow.”
The detector should help a human decide what to check next. It should not become the only decision-maker, especially in school, hiring, legal, compliance, finance, or workplace contexts.
Recent reporting also shows institutions are still struggling with this. For example, the Financial Times reported in July 2026 that some universities are dropping or reducing AI detector use because of reliability and false-positive concerns, while Turnitin itself frames its detector as a starting point or data point rather than definitive proof. That is the right product mindset: detection can inform review, but it should not replace it.
What are we going to build?
We’ll build a Python detector that combines three layers:
- Text signals
Basic explainable features like sentence length variation, repeated phrases, lexical diversity, and generic wording. - Model-based detection
A Hugging Face text classification model trained or fine-tuned for AI-generated text detection. - Review report
A structured result with risk score, reasons, paragraph-level flags, and warnings.
The final output will look like this:
{
“label”: “needs_review”,
“ai_risk_score”: 0.76,
“heuristic_score”: 0.42,
“model_score”: 0.98,
“confidence”: “medium”,
“recommendation”: “Send this text to human review.”,
“warning”: “AI detection is probabilistic and can produce false positives and false negatives.”
}
That is the kind of output you can use in an app, dashboard, CMS, or internal review tool.
Step 1: Install the Python packages
Start with a few simple libraries.
pip install transformers torch scikit-learn pandas numpy
We’ll use:
| Package | Why |
| transformers | Load text classification models |
| torch | Run model inference |
| scikit-learn | Optional metrics and text features |
| pandas | Batch processing |
| numpy | Scoring and calculations |
Hugging Face’s Transformers pipeline docs show that pipeline(“text-classification”) can load text classification models with a simple interface. That makes it a good fit for a practical detector tutorial.
Step 2: Create basic text statistics
Let’s start with explainable signals.
These will not prove AI authorship, but they help describe the writing style.
Create signals.py:
import re
import math
from collections import Counter
def split_sentences(text):
sentences = re.split(r”(?<=[.!?])\s+”, text.strip())
return [sentence.strip() for sentence in sentences if sentence.strip()]
def tokenize(text):
words = re.findall(r”\b\w+\b”, text.lower())
return words
def average(numbers):
if not numbers:
return 0
return sum(numbers) / len(numbers)
def standard_deviation(numbers):
if not numbers:
return 0
avg = average(numbers)
variance = average([(number – avg) ** 2 for number in numbers])
return math.sqrt(variance)
def analyze_text_signals(text):
sentences = split_sentences(text)
words = tokenize(text)
sentence_lengths = [len(tokenize(sentence)) for sentence in sentences]
unique_words = set(words)
lexical_diversity = len(unique_words) / len(words) if words else 0
return {
“word_count”: len(words),
“sentence_count”: len(sentences),
“average_sentence_length”: round(average(sentence_lengths), 2),
“sentence_length_variation”: round(standard_deviation(sentence_lengths), 2),
“lexical_diversity”: round(lexical_diversity, 3)
}
Test it:
sample = “””
Artificial intelligence is changing how teams create content.
It can help summarize documents, draft emails, and classify messages.
However, teams should still review important outputs before publishing.
“””
print(analyze_text_signals(sample))
Example output:
{
“word_count”: 31,
“sentence_count”: 3,
“average_sentence_length”: 10.33,
“sentence_length_variation”: 1.25,
“lexical_diversity”: 0.903
}
These features are useful because they make the report explainable.
A user can see why the text was flagged instead of only seeing a mysterious score.
Step 3: Detect repeated phrases
AI-generated drafts often repeat phrases, especially when the prompt is broad or the output is low-effort.
Let’s add n-gram repetition detection.
def get_ngrams(words, size):
return [
” “.join(words[index:index + size])
for index in range(len(words) – size + 1)
]
def find_repeated_phrases(text, size=3, min_count=2):
words = tokenize(text)
ngrams = get_ngrams(words, size)
counts = Counter(ngrams)
repeated = [
{
“phrase”: phrase,
“count”: count
}
for phrase, count in counts.items()
if count >= min_count
]
return sorted(repeated, key=lambda item: item[“count”], reverse=True)
Use it:
text = “””
AI can improve workflows. AI can improve productivity.
AI can improve customer support. AI can improve content creation.
“””
print(find_repeated_phrases(text, size=2, min_count=2))
Output:
[
{
“phrase”: “ai can”,
“count”: 4
},
{
“phrase”: “can improve”,
“count”: 4
}
]
This is not automatically “AI.” Repetition can be normal in legal, technical, or instructional text.
But it is useful evidence for review.
Step 4: Detect generic AI-style phrases
Some phrases appear a lot in weak AI-generated content.
Let’s catch them.
GENERIC_AI_PHRASES = [
“in today’s digital landscape”,
“it is important to note”,
“in conclusion”,
“furthermore”,
“moreover”,
“additionally”,
“delve into”,
“unlock the power”,
“seamless experience”,
“revolutionize”,
“when it comes to”,
“in the ever-evolving world”,
“game-changer”,
“transform the way”
]
def find_generic_phrases(text):
normalized = text.lower()
return [
phrase
for phrase in GENERIC_AI_PHRASES
if phrase in normalized
]
Test:
text = “In today’s digital landscape, businesses can unlock the power of AI.”
print(find_generic_phrases(text))
Output:
[
“in today’s digital landscape”,
“unlock the power”
]
Again, this is a quality signal. A human can write these phrases too. The point is to flag sections that may need editing.
Step 5: Build a simple heuristic score
Now let’s combine signals into a small risk score.
This is not a full detector. It is an explainable layer.
def heuristic_ai_risk(text):
signals = analyze_text_signals(text)
repeated_phrases = find_repeated_phrases(text, size=3, min_count=2)
generic_phrases = find_generic_phrases(text)
score = 0
reasons = []
if signals[“word_count”] < 80:
reasons.append(“Text is short, so detection is less reliable.”)
if signals[“sentence_count”] >= 5 and signals[“sentence_length_variation”] < 3:
score += 0.2
reasons.append(“Sentence lengths are very uniform.”)
if signals[“word_count”] > 100 and signals[“lexical_diversity”] < 0.45:
score += 0.2
reasons.append(“Lexical diversity is low.”)
if len(repeated_phrases) > 3:
score += 0.2
reasons.append(“Several repeated phrases were found.”)
if len(generic_phrases) > 2:
score += 0.2
reasons.append(“Several generic AI-style phrases were found.”)
score = min(score, 1)
if score >= 0.7:
label = “needs_review”
elif score >= 0.4:
label = “some_ai_like_signals”
else:
label = “low_or_unclear”
return {
“heuristic_score”: round(score, 2),
“label”: label,
“signals”: signals,
“repeated_phrases”: repeated_phrases[:10],
“generic_phrases”: generic_phrases,
“reasons”: reasons,
“warning”: “This heuristic score is not proof that AI wrote the text.”
}
Use it:
sample = “””
In today’s digital landscape, businesses must unlock the power of AI.
Furthermore, AI can improve productivity.
Moreover, AI can improve workflows.
Additionally, AI can improve customer experiences.
In conclusion, it is important to note that AI is a game-changer.
“””
print(heuristic_ai_risk(sample))
This gives you a simple explainable report.
It will not be enough for serious detection, but it is helpful in a layered system.
Step 6: Use a transformer classifier
Now let’s add a model-based detector.
You can use a Hugging Face model fine-tuned for AI-generated text detection. Model availability changes, so check the model card carefully before using one in production.
Look for:
- Training data.
- Supported languages.
- Intended use.
- Limitations.
- Evaluation results.
- False-positive notes.
- License.
- Whether it handles newer models.
- Whether it handles edited AI text.
Example code:
from transformers import pipeline
detector = pipeline(
“text-classification”,
model=”your-ai-text-detection-model”,
truncation=True
)
def model_ai_risk(text):
result = detector(text)[0]
label = result[“label”]
score = float(result[“score”])
return {
“model_label”: label,
“model_score”: round(score, 4)
}
Use it:
text = “Paste your text here.”
print(model_ai_risk(text))
The exact labels depend on the model. Some return AI and HUMAN. Others return LABEL_0 and LABEL_1. Always read the model card.
A 2026 paper called AI Generated Text Detection compared traditional ML and transformer-based approaches. The authors found that TF-IDF logistic regression gave a reasonable baseline, while BiLSTM and DistilBERT performed better, with DistilBERT reaching the strongest ROC-AUC in their setup. That supports using transformer classifiers, but the paper also notes limitations around dataset diversity and generalization.
Step 7: Combine heuristic and model scores
A model score alone is hard to explain. A heuristic score alone is weak.
So combine them.
def normalize_model_result(model_result):
label = model_result[“model_label”].lower()
score = model_result[“model_score”]
ai_like_labels = [“ai”, “generated”, “machine”, “label_1”]
if any(ai_label in label for ai_label in ai_like_labels):
return score
return 1 – score
def combined_ai_detection_report(text, model_result=None):
heuristic = heuristic_ai_risk(text)
heuristic_score = heuristic[“heuristic_score”]
if model_result is not None:
model_score = normalize_model_result(model_result)
combined_score = (0.35 * heuristic_score) + (0.65 * model_score)
else:
model_score = None
combined_score = heuristic_score
combined_score = round(combined_score, 2)
if combined_score >= 0.75:
label = “needs_review”
confidence = “medium”
recommendation = “Send this text to human review.”
elif combined_score >= 0.45:
label = “unclear”
confidence = “low”
recommendation = “Review if the context is sensitive.”
else:
label = “low_risk”
confidence = “low”
recommendation = “No strong AI-like signal found, but this is not proof of human authorship.”
return {
“ai_risk_score”: combined_score,
“label”: label,
“confidence”: confidence,
“heuristic_score”: heuristic_score,
“model_score”: model_score,
“heuristic_details”: heuristic,
“recommendation”: recommendation,
“warning”: “AI detection is probabilistic and can produce false positives and false negatives.”
}
Use it:
text = “””
Paste the text you want to check here.
“””
model_result = model_ai_risk(text)
report = combined_ai_detection_report(text, model_result)
print(report)
This gives you a safer result than a one-score detector.
Step 8: Analyze text by paragraph
Instead of flagging the whole document, show the risky sections.
This is much more useful for editors.
def split_paragraphs(text):
paragraphs = re.split(r”\n\s*\n”, text.strip())
return [
paragraph.strip()
for paragraph in paragraphs
if paragraph.strip()
]
def analyze_paragraphs(text, use_model=False):
paragraphs = split_paragraphs(text)
results = []
for index, paragraph in enumerate(paragraphs, start=1):
model_result = model_ai_risk(paragraph) if use_model else None
report = combined_ai_detection_report(paragraph, model_result)
results.append({
“paragraph_number”: index,
“text_preview”: paragraph[:160],
“ai_risk_score”: report[“ai_risk_score”],
“label”: report[“label”],
“reasons”: report[“heuristic_details”][“reasons”]
})
return results
Use it:
paragraph_results = analyze_paragraphs(long_text, use_model=False)
for result in paragraph_results:
print(result)
Example output:
[
{
“paragraph_number”: 1,
“text_preview”: “Artificial intelligence is changing how teams…”,
“ai_risk_score”: 0.2,
“label”: “low_risk”,
“reasons”: []
},
{
“paragraph_number”: 2,
“text_preview”: “In today’s digital landscape…”,
“ai_risk_score”: 0.62,
“label”: “unclear”,
“reasons”: [
“Several generic AI-style phrases were found.”
]
}
]
This is a better user experience. The reviewer can inspect the flagged paragraphs instead of staring at one scary document score.
Step 9: Batch-check many files
If you are building this for a CMS, school platform, content QA tool, or moderation queue, you may need batch processing.
Create a CSV like this:
id,text
1,”Paste first text here”
2,”Paste second text here”
3,”Paste third text here”
Then process it:
import pandas as pd
def batch_detect_csv(input_path, output_path, use_model=False):
df = pd.read_csv(input_path)
reports = []
for _, row in df.iterrows():
text = row[“text”]
model_result = model_ai_risk(text) if use_model else None
report = combined_ai_detection_report(text, model_result)
reports.append({
“id”: row[“id”],
“ai_risk_score”: report[“ai_risk_score”],
“label”: report[“label”],
“confidence”: report[“confidence”],
“recommendation”: report[“recommendation”],
“word_count”: report[“heuristic_details”][“signals”][“word_count”]
})
output_df = pd.DataFrame(reports)
output_df.to_csv(output_path, index=False)
batch_detect_csv(
input_path=”texts.csv”,
output_path=”ai_detection_results.csv”,
use_model=False
)
Now you have a review queue.
For high-volume apps, do not run the largest model on every text. Use a cheaper heuristic first, then run a classifier only on unclear or high-risk cases.
Step 10: Build a simple Flask API
Let’s turn the detector into an API.
Install Flask:
pip install flask
Create app.py:
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route(“/detect-ai”, methods=[“POST”])
def detect_ai():
data = request.get_json(force=True)
text = data.get(“text”, “”)
if not text:
return jsonify({
“error”: “Text is required.”
}), 400
if len(text.split()) < 80:
report = combined_ai_detection_report(text)
report[“label”] = “too_short”
report[“recommendation”] = “Text is too short for reliable detection.”
return jsonify(report)
report = combined_ai_detection_report(text)
return jsonify(report)
if __name__ == “__main__”:
app.run(debug=True)
Run:
python app.py
Test:
curl -X POST http://127.0.0.1:5000/detect-ai \
-H “Content-Type: application/json” \
-d ‘{“text”:”Paste your text here.”}’
This gives you a small AI-detection API you can connect to an app, CMS, or internal dashboard.
What about DetectGPT?
DetectGPT is a research method for detecting machine-generated text using probability curvature.
The DetectGPT paper proposed that text generated by a model tends to occupy specific curvature regions of that model’s log probability function. The authors reported that DetectGPT improved detection of fake news articles generated by GPT-NeoX compared with zero-shot baselines.
That is interesting research, but DetectGPT-style methods are heavier than a basic Python classifier.
They usually need:
- Access to model log probabilities.
- Perturbations of the original text.
- Repeated model scoring.
- More compute.
- Careful implementation.
For most product teams, the practical version is:
classifier + explainable signals + review workflow
DetectGPT is worth studying if you are building a serious detector, but it may be more complexity than a normal app needs.
Why false positives matter so much
A false positive means human text gets flagged as AI.
That can hurt people.
Examples:
| Context | Why false positives are serious |
| School | A student may be wrongly accused |
| Hiring | A real applicant may be unfairly filtered |
| Publishing | A writer may be asked to “prove” their own work |
| Workplace | An employee may be accused of cheating |
| Moderation | Legit content may be removed |
So your UI should not say:
This was written by AI.
Use:
This text has AI-like signals and should be reviewed.
The difference is huge.
The 2026 Why AI-Generated Text Detection Fails paper is useful here again because it shows how detectors can rely on unstable cues that break across domains, formatting styles, and text lengths. That means a detector can look strong in a benchmark and still behave badly in the wild.
What about provenance and watermarking?
Text detection tries to infer authorship from the text.
Provenance tries to verify where content came from.
Those are different things.
OpenAI’s 2026 post on advancing content provenance describes work around Content Credentials and SynthID watermark signals for media generated by ChatGPT, the OpenAI API, and Codex. The post focuses heavily on media provenance, but the general lesson applies here too: direct provenance signals are stronger than guessing from style.
For normal plain text, you usually do not have reliable provenance metadata. That is why text detection is probabilistic.
A stronger workflow may combine:
| Signal | What it adds |
| AI detector score | Risk signal |
| Revision history | How the text was created |
| User disclosure | Whether AI assistance was used |
| Source citations | Whether claims can be checked |
| Provenance metadata | Stronger signal when available |
| Human review | Final decision |
That is much safer than detector-only judgment.
Where LLMAPI fits
LLMAPI can fit around AI detection when detection is part of a larger content review workflow.
For example:
- Python detector returns a risk score.
- If risk is low, the content passes.
- If risk is unclear, LLMAPI routes the text to a model for review assistance.
- The model explains which sections feel generic, unsupported, repetitive, or citation-light.
- The reviewer gets a summary and decides what to do.
Useful LLMAPI follow-up tasks:
| Task | Example |
| Content quality review | “Which paragraphs feel generic?” |
| Citation check | “Which factual claims need sources?” |
| Rewrite suggestions | “Make this section less templated.” |
| Reviewer summary | “Explain why this was flagged.” |
| Policy review | “Does this violate content guidelines?” |
| Model routing | Use cheaper models for simple review and stronger models for hard cases |
| Fallback | Retry with another model if one provider fails |
This is useful because a detector score alone does not help much. Review guidance does.
What should the UI say?
If you build this into an app, the wording matters.
Use labels like:
Low risk
Unclear
Needs review
Too short to classify
Avoid labels like:
Human
AI
Cheating
Fake
A better UI can show:
| UI field | Example |
| Risk label | Needs review |
| Risk score | 76% |
| Confidence | Medium |
| Evidence | Repetition, generic wording, model score |
| Segment view | Paragraphs 2 and 5 flagged |
| Warning | This result is probabilistic |
| Action | Review, approve, request sources, ask for revision |
The goal is to help reviewers, not scare users.
Common mistakes
AI detection projects go wrong when the output feels too certain.
Watch out for these:
| Mistake | Better approach |
| Treating score as proof | Treat it as a review signal |
| Checking very short text | Mark short text as unreliable |
| Ignoring false positives | Add warnings and human review |
| Using one model blindly | Test on your own data |
| No segment view | Show paragraph-level results |
| No explanation | Include text signals and reasons |
| No privacy policy | Explain how submitted text is handled |
| No language testing | Test every language you support |
| No appeal/review path | Let users provide sources or drafts |
| No benchmark set | Create your own test examples |
The biggest mistake is making the detector sound more certain than it is.
A safer production workflow
Here is the workflow we would actually ship:
- User submits text.
- App checks text length and language.
- Python calculates explainable text signals.
- Model classifier returns AI-like probability.
- App combines scores into a risk label.
- App highlights risky paragraphs.
- App shows uncertainty warning.
- Low-risk text passes.
- Unclear text gets optional review.
- High-risk text goes to human review.
- Reviewer sees evidence, not only a score.
- Final decision is made by a human.
This workflow is much more defensible than “AI score says no.”
The practical takeaway
You can detect AI-like content with Python, but you should build the detector carefully.
Use simple text features for explainability. Add a transformer classifier for stronger detection. Analyze paragraphs instead of only whole documents. Mark short text as unreliable. Return a risk score, not a verdict. Add warnings. Keep humans in the loop for anything sensitive.
The best Python detector is not the one that shouts “AI!” the loudest.
It is the one that says:
Here are the signals.
Here is the uncertainty.
Here is what should be reviewed next.
That is how AI content detection becomes useful without becoming unfair.