Finding similar images sounds simple until you ask one tiny question:

Similar how?

Because there are different kinds of “similar.”

Two images can be similar because they are almost the same file. Or because they show the same product. Or because they have the same vibe. Or because both contain a dog on a beach. Or because both are screenshots of a phishing page that looks like your bank login page. Or because both are product photos with a white background and weirdly perfect lighting.

So before we write Python, we need to choose the right similarity method.

In this guide, we’ll build practical image similarity workflows with Python:

1. Perceptual hashing for near-duplicate images.
2. CLIP/OpenCLIP embeddings for semantic similarity.
3. FAISS for fast local vector search.
4. Qdrant for scalable vector search with metadata.
5. Image-to-image and text-to-image search.
6. Evaluation tips so your search results do not quietly become nonsense.

What kind of similar images are you trying to find?

Start here, because this decides the whole setup.

Similarity type	Example	Best method
Exact duplicate	Same image uploaded twice	File hash
Near duplicate	Same image resized or compressed	Perceptual hash
Same object/product	Same sneaker from different angles	Image embeddings
Same visual style	Similar mood, colors, layout	Image embeddings
Same concept	“A red car in snow”	CLIP/OpenCLIP embeddings
Same brand/screenshot layout	Fake login page looks like real one	Perceptual hash + embeddings
Large-scale image search	Millions of images	Vector database
Search by text	“blue ceramic mug”	CLIP text/image embeddings

If you only need duplicate cleanup, do not start with a giant AI model.

If you want Pinterest-style “find things that look similar,” use embeddings.

If you want text search over images, use CLIP-style embeddings because they place images and text in a shared vector space.

Why we can write this guide

We’ve spent around 6 years working with AI APIs, image embeddings, vector search, computer vision workflows, content automation, and developer tutorials. We also checked current docs for CLIP, OpenCLIP, FAISS, Qdrant, perceptual hashing, and recent image similarity research.

The practical lesson is simple: image search gets much better when you separate near-duplicate detection from semantic similarity.

Perceptual hashing is great for “this image is basically the same.” CLIP-style embeddings are better for “these images mean similar things.” Vector indexes like FAISS or Qdrant help you search those embeddings quickly.

The simplest option: Find near-duplicates with perceptual hashing

Perceptual hashing creates a small signature of an image’s visual content.

Unlike a normal file hash, it can still match images after small changes like resizing, compression, or slight blur. The pHash documentation explains that perceptual hashing aims to create signatures robust to distortions, so visually similar media should have relatively close hash distances.

That makes pHash-style methods useful for:

1. Duplicate image cleanup.
2. Copyright monitoring.
3. Reuploaded image detection.
4. Screenshot similarity.
5. Basic moderation queues.
6. Product catalog deduplication.

Install:

pip install pillow imagehash

Basic example:

from PIL import Image

import imagehash

image_1 = Image.open(“image_1.jpg”)

image_2 = Image.open(“image_2.jpg”)

hash_1 = imagehash.phash(image_1)

hash_2 = imagehash.phash(image_2)

distance = hash_1 – hash_2

print(“Hash 1:”, hash_1)

print(“Hash 2:”, hash_2)

print(“Distance:”, distance)

A smaller distance means the images are more visually similar.

Example decision logic:

def are_near_duplicates(path_1, path_2, threshold=8):

    image_1 = Image.open(path_1)

    image_2 = Image.open(path_2)

    hash_1 = imagehash.phash(image_1)

    hash_2 = imagehash.phash(image_2)

    distance = hash_1 – hash_2

    return {

        “similar”: distance <= threshold,

        “distance”: distance,

        “threshold”: threshold

    }

The threshold depends on your images. Start with something like 5-10, then tune it with real examples.

When perceptual hashing is enough

Use perceptual hashing when the images are basically the same.

Good fit	Weak fit
Same image, resized	Different images with same meaning
Same screenshot, compressed	Same product from different angle
Duplicate media uploads	Similar style/aesthetic
Watermark/copy detection	Text-to-image search
Fast local comparison	Large semantic image search

A 2025 paper called PhishSnap used perceptual hashing for privacy-preserving, on-device phishing-page detection and reported 0.79 accuracy, 0.76 precision, and 0.78 recall on its test set. That fits this section because screenshot similarity and near-duplicate visual matching are exactly where perceptual hashing can be useful.

The better option for search: Use image embeddings

Image embeddings are vectors that represent the visual meaning of an image.

Instead of comparing pixels directly, you compare vectors.

That means your system can find images that are conceptually similar, even when they are not near-duplicates.

Example:

Query image: red running shoe on white background

Similar result: same shoe from another angle

Similar result: red sneaker on model foot

Similar result: product listing photo of red athletic shoe

That is hard for perceptual hashing. It is exactly what embeddings are for.

The classic model here is CLIP. OpenAI’s CLIP paper, Learning Transferable Visual Models From Natural Language Supervision, explains that CLIP was trained to match images with natural-language descriptions using 400 million image-text pairs. That is why it can compare both image-to-image and text-to-image similarity.

Install CLIP for Python

OpenAI’s CLIP GitHub repo includes installation and inference examples for generating image and text features.

Install:

pip install torch torchvision pillow

pip install git+https://github.com/openai/CLIP.git

Now create image embeddings:

import torch

import clip

from PIL import Image

import numpy as np

device = “cuda” if torch.cuda.is_available() else “cpu”

model, preprocess = clip.load(“ViT-B/32”, device=device)

def get_image_embedding(image_path):

    image = preprocess(Image.open(image_path).convert(“RGB”)).unsqueeze(0).to(device)

    with torch.no_grad():

        embedding = model.encode_image(image)

    embedding = embedding / embedding.norm(dim=-1, keepdim=True)

    return embedding.cpu().numpy()[0]

The normalization step matters because similarity search usually works best with normalized vectors and cosine similarity.

Compare two images with cosine similarity

Once you have embeddings, comparing two images is simple.

def cosine_similarity(vector_1, vector_2):

    return float(np.dot(vector_1, vector_2))

embedding_1 = get_image_embedding(“image_1.jpg”)

embedding_2 = get_image_embedding(“image_2.jpg”)

score = cosine_similarity(embedding_1, embedding_2)

print(“Similarity:”, score)

If embeddings are normalized, the dot product acts like cosine similarity.

Typical interpretation:

Score	Meaning
0.90+	Very similar
0.75-0.89	Likely related
0.55-0.74	Maybe similar
Below 0.55	Probably different

Please treat these numbers as starting points, not universal truth. Different datasets need different thresholds.

Build a small local image search engine

Now let’s search through a folder of images.

from pathlib import Path

image_folder = Path(“images”)

image_paths = [

    path for path in image_folder.iterdir()

    if path.suffix.lower() in [“.jpg”, “.jpeg”, “.png”, “.webp”]

]

embeddings = []

metadata = []

for path in image_paths:

    embedding = get_image_embedding(str(path))

    embeddings.append(embedding)

    metadata.append({

        “path”: str(path),

        “filename”: path.name

    })

embeddings = np.array(embeddings).astype(“float32”)

print(“Indexed images:”, len(metadata))

print(“Embedding shape:”, embeddings.shape)

Now search by image:

def search_by_image(query_path, embeddings, metadata, top_k=5):

    query_embedding = get_image_embedding(query_path).astype(“float32”)

    scores = embeddings @ query_embedding

    top_indices = np.argsort(scores)[::-1][:top_k]

    return [

        {

            “path”: metadata[index][“path”],

            “filename”: metadata[index][“filename”],

            “score”: float(scores[index])

        }

        for index in top_indices

    ]

results = search_by_image(“query.jpg”, embeddings, metadata)

for result in results:

    print(result)

This works fine for a small folder.

But if you have thousands or millions of images, use a vector index.

Use FAISS for faster local similarity search

FAISS is a library for efficient similarity search and clustering of dense vectors. Its docs say FAISS is written in C++ and has Python wrappers, which makes it a strong fit for local image search experiments and production-ish pipelines.

Install:

pip install faiss-cpu

Use FAISS with normalized CLIP embeddings:

import faiss

dimension = embeddings.shape[1]

index = faiss.IndexFlatIP(dimension)

index.add(embeddings)

print(“Images in index:”, index.ntotal)

IndexFlatIP uses inner product. Since we normalized the embeddings, inner product works like cosine similarity.

Search:

def search_faiss_by_image(query_path, index, metadata, top_k=5):

    query_embedding = get_image_embedding(query_path).astype(“float32”)

    query_embedding = np.expand_dims(query_embedding, axis=0)

    scores, indices = index.search(query_embedding, top_k)

    results = []

    for score, index_id in zip(scores[0], indices[0]):

        results.append({

            “path”: metadata[index_id][“path”],

            “filename”: metadata[index_id][“filename”],

            “score”: float(score)

        })

    return results

results = search_faiss_by_image(“query.jpg”, index, metadata, top_k=5)

for result in results:

    print(result)

A 2024 paper on the FAISS library describes FAISS as a toolkit for vector similarity search, clustering, compression, and vector transformations. That fits image search because once every image becomes a vector, the whole problem becomes “find the nearest vectors quickly.”

Add text-to-image search with CLIP

This is where CLIP gets fun.

Because CLIP embeds images and text into the same space, you can search your image collection with a text query.

Example:

red chair in a modern living room

And your system can return matching images.

Add a text embedding function:

def get_text_embedding(text):

    tokens = clip.tokenize([text]).to(device)

    with torch.no_grad():

        embedding = model.encode_text(tokens)

    embedding = embedding / embedding.norm(dim=-1, keepdim=True)

    return embedding.cpu().numpy()[0]

Search images with text:

def search_by_text(query_text, index, metadata, top_k=5):

    query_embedding = get_text_embedding(query_text).astype(“float32”)

    query_embedding = np.expand_dims(query_embedding, axis=0)

    scores, indices = index.search(query_embedding, top_k)

    return [

        {

            “path”: metadata[index_id][“path”],

            “filename”: metadata[index_id][“filename”],

            “score”: float(score)

        }

        for score, index_id in zip(scores[0], indices[0])

    ]

results = search_by_text(“a black dog running on grass”, index, metadata)

for result in results:

    print(result)

This is how you build a simple visual search engine.

Use cases:

1. Search product photos by description.
2. Find stock images by concept.
3. Search screenshots by UI type.
4. Search moodboards by vibe.
5. Search a media library without manual tags.
6. Find similar memes, thumbnails, or ad creatives.
7. Build Pinterest-style discovery.

Use OpenCLIP for more model choices

OpenCLIP is an open-source implementation of CLIP with many pretrained model options.

The OpenCLIP GitHub repo describes it as an open-source implementation of CLIP and includes many pretrained models and usage examples. This is useful if you want to test different CLIP-like models instead of only OpenAI’s original CLIP.

Install:

pip install open_clip_torch torch pillow

Use OpenCLIP:

import open_clip

import torch

from PIL import Image

device = “cuda” if torch.cuda.is_available() else “cpu”

model, _, preprocess = open_clip.create_model_and_transforms(

    “ViT-B-32”,

    pretrained=”laion2b_s34b_b79k”,

    device=device

)

tokenizer = open_clip.get_tokenizer(“ViT-B-32”)

def get_openclip_image_embedding(image_path):

    image = preprocess(Image.open(image_path).convert(“RGB”)).unsqueeze(0).to(device)

    with torch.no_grad():

        embedding = model.encode_image(image)

    embedding = embedding / embedding.norm(dim=-1, keepdim=True)

    return embedding.cpu().numpy()[0]

def get_openclip_text_embedding(text):

    tokens = tokenizer([text]).to(device)

    with torch.no_grad():

        embedding = model.encode_text(tokens)

    embedding = embedding / embedding.norm(dim=-1, keepdim=True)

    return embedding.cpu().numpy()[0]

OpenCLIP is especially useful if you want to experiment with model size, dataset, speed, and quality tradeoffs.

Save embeddings so you do not recompute everything

Do not recompute embeddings every time your app starts.

Save them.

Simple NumPy save:

np.save(“image_embeddings.npy”, embeddings)

Save metadata:

import json

with open(“image_metadata.json”, “w”, encoding=”utf-8″) as file:

    json.dump(metadata, file, indent=2)

Load later:

embeddings = np.load(“image_embeddings.npy”)

with open(“image_metadata.json”, “r”, encoding=”utf-8″) as file:

    metadata = json.load(file)

Then rebuild FAISS:

dimension = embeddings.shape[1]

index = faiss.IndexFlatIP(dimension)

index.add(embeddings.astype(“float32”))

For a small app, this is enough.

For a larger app, use a vector database.

Use Qdrant when you need metadata filters

FAISS is great for local vector search. But if your app needs filtering, persistence, distributed search, APIs, payload metadata, or production search endpoints, a vector database is easier.

Qdrant’s documentation describes Qdrant as an AI-native vector search engine. Its search docs cover similarity search and filtering, and the quickstart shows loading data and returning results in decreasing similarity order.

This matters because real image search often needs filters.

Example:

Find similar images, but only in category = “shoes” and brand = “Nike”.

Install:

pip install qdrant-client

Run local Qdrant with Docker:

docker run -p 6333:6333 qdrant/qdrant

Create a collection:

from qdrant_client import QdrantClient

from qdrant_client.models import Distance, VectorParams, PointStruct

client = QdrantClient(url=”http://localhost:6333″)

collection_name = “image_search”

client.recreate_collection(

    collection_name=collection_name,

    vectors_config=VectorParams(

        size=embeddings.shape[1],

        distance=Distance.COSINE

    )

)

Upload images and metadata:

points = []

for index_id, embedding in enumerate(embeddings):

    points.append(

        PointStruct(

            id=index_id,

            vector=embedding.tolist(),

            payload=metadata[index_id]

        )

    )

client.upsert(

    collection_name=collection_name,

    points=points

)

Search:

query_embedding = get_image_embedding(“query.jpg”)

results = client.search(

    collection_name=collection_name,

    query_vector=query_embedding.tolist(),

    limit=5

)

for result in results:

    print(result.score, result.payload)

Now you have a real vector-search backend.

Add metadata that makes search useful

Image similarity alone is nice. Image similarity plus metadata is much better.

Store metadata like:

Metadata	Why it helps
File path or URL	Display the result
Category	Filter by product/content type
Brand	Product or asset search
Source	Track upload origin
Created date	Filter recent assets
Width/height	Useful for design workflows
Tags	Hybrid search and UI filters
Caption	Helps with text search
Owner/user ID	Privacy and access control
License	Avoid using restricted images

Example metadata:

{

  “path”: “images/red-sneaker-01.jpg”,

  “category”: “shoes”,

  “brand”: “ExampleBrand”,

  “color”: “red”,

  “source”: “product_catalog”,

  “license”: “internal”

}

Then you can support queries like:

1. Find similar images from the same brand.
2. Find product photos with similar style.
3. Find duplicate uploads from one user.
4. Search only licensed assets.
5. Search only images created this month.

That is where image search becomes product-ready.

Combine pHash and embeddings

For many apps, the best setup uses both.

Use perceptual hashing first for near-duplicates. Use embeddings next for semantic similarity.

Example workflow:

1. User uploads image.
2. Compute pHash.
3. Check if a near-duplicate already exists.
4. If yes, flag as duplicate.
5. If no, compute CLIP embedding.
6. Add embedding to vector index.
7. Use embedding for semantic search.

Why use both?

Method	Best at
pHash	Near-duplicate detection
CLIP/OpenCLIP embeddings	Semantic visual similarity
FAISS/Qdrant	Fast nearest-neighbor search

This saves compute and gives better results.

Example:

def get_phash(image_path):

    image = Image.open(image_path)

    return str(imagehash.phash(image))

def add_image(image_path):

    perceptual_hash = get_phash(image_path)

    # First, compare against existing hashes for near-duplicates.

    # Then compute embedding only if the image is new enough.

    embedding = get_image_embedding(image_path)

    return {

        “path”: image_path,

        “phash”: perceptual_hash,

        “embedding”: embedding

    }

This is a very normal production pattern.

Build a small Flask image search API

Let’s turn this into an API.

Install:

pip install flask

Create app.py:

from flask import Flask, request, jsonify

from pathlib import Path

import tempfile

import numpy as np

import faiss

app = Flask(__name__)

embeddings = np.load(“image_embeddings.npy”).astype(“float32”)

metadata = json.load(open(“image_metadata.json”, “r”, encoding=”utf-8″))

dimension = embeddings.shape[1]

index = faiss.IndexFlatIP(dimension)

index.add(embeddings)

@app.route(“/search-by-image”, methods=[“POST”])

def search_by_uploaded_image():

    if “image” not in request.files:

        return jsonify({“error”: “Image file is required”}), 400

    image_file = request.files[“image”]

    with tempfile.NamedTemporaryFile(suffix=”.jpg”) as temp:

        image_file.save(temp.name)

        query_embedding = get_image_embedding(temp.name).astype(“float32”)

        query_embedding = np.expand_dims(query_embedding, axis=0)

        scores, indices = index.search(query_embedding, 5)

    results = []

    for score, index_id in zip(scores[0], indices[0]):

        results.append({

            “score”: float(score),

            “metadata”: metadata[index_id]

        })

    return jsonify({

        “results”: results

    })

if __name__ == “__main__”:

    app.run(debug=True)

Tiny note: make sure your real file imports include json, plus your get_image_embedding function from earlier.

Test:

curl -X POST http://localhost:5000/search-by-image \

  -F “[email protected]”

Now you have a working image similarity API.

How do you evaluate similar image search?

Please test with the images your app will actually use.

For an e-commerce app, test product images.

For a stock image library, test concepts and styles.

For duplicate cleanup, test resized, compressed, cropped, and edited versions.

Track:

Metric	What it tells you
Top-1 accuracy	Is the first result usually correct?
Top-5 accuracy	Is a good result in the first five?
Precision@K	Are the top results mostly relevant?
Recall@K	Are you finding enough relevant images?
Duplicate recall	Do near-duplicates get caught?
False positives	Are unrelated images ranked too high?
Latency	Is search fast enough?
Index size	Can your system scale?
Review effort	How much human cleanup remains?

For user-facing search, Top-5 quality usually matters more than one perfect score. Users can pick from a small group. But if your system auto-flags duplicates or copyright matches, false positives matter a lot more.

What can go wrong?

Image similarity search has a few sneaky failure modes.

Problem	What happens
Background dominates	Similar backgrounds rank higher than actual object
Color dominates	Same color, wrong object
Tiny object ignored	Model focuses on scene instead
Cropped object	Embedding changes more than expected
Text in image	CLIP may or may not capture exact wording
Near-duplicates missed	pHash threshold too strict
False duplicates	pHash threshold too loose
Domain mismatch	Model was not trained for your exact image type
No metadata filters	Search results feel random
Bad evaluation set	You think it works until users try it

A useful fix is to combine signals:

1. pHash for duplicate-like matching.
2. Embeddings for semantic matching.
3. Metadata filters for product/app logic.
4. Human review for high-stakes matches.

Where LLMAPI fits after image search

LLMAPI can fit after image similarity search when your app needs reasoning, captions, reports, or workflow routing around the results.

For example:

1. User uploads an image.
2. Python generates an embedding.
3. FAISS or Qdrant returns similar images.
4. Your app sends the result metadata to LLMAPI.
5. LLMAPI generates a summary, tags, product recommendation, moderation note, or search explanation.

Useful follow-up tasks:

Task	Example
Search explanation	“These images match because they show red sneakers on white backgrounds.”
Auto-tagging	Add tags based on similar images
Product recommendations	Suggest visually similar products
Duplicate review note	Explain why two assets may be duplicates
Content moderation	Summarize why image needs review
Catalog enrichment	Generate captions from similar-item metadata
Report writing	Create weekly visual search insights
Workflow routing	Send similar assets to design, legal, or catalog teams

Image embeddings find the neighbors. LLMAPI helps explain, route, or transform what happens next.

Common mistakes when building image similarity search

Here are the classics.

Mistake	Better approach
Using file hashes for visual similarity	Use pHash for near-duplicates
Using pHash for semantic search	Use CLIP/OpenCLIP embeddings
Recomputing embeddings every run	Save embeddings
No normalization	Normalize vectors before cosine search
No metadata	Store path, category, tags, source, owner
No evaluation set	Test with real use cases
One universal threshold	Tune thresholds by dataset
No filters	Add category/brand/date/user filters
Using brute force forever	Add FAISS or vector database
Trusting results blindly	Add review for high-risk matches

The biggest mistake is mixing up duplicate search and semantic search. They look similar from the outside, but they need different methods.

The first version we’d build

Start simple.

For a small image library:

1. Store images in a folder or object storage.
2. Generate CLIP/OpenCLIP embeddings.
3. Normalize embeddings.
4. Store embeddings in .npy.
5. Store metadata in JSON.
6. Use FAISS for search.
7. Add text-to-image search.
8. Add pHash duplicate detection.
9. Show top 5 results.
10. Tune thresholds with real examples.

For a production app:

1. Store images in object storage.
2. Store metadata in a database.
3. Store vectors in Qdrant, Pinecone, Weaviate, Milvus, or another vector database.
4. Add user/team access filters.
5. Add pHash for near-duplicates.
6. Add batch embedding jobs.
7. Add monitoring and evaluation.
8. Add review workflows.
9. Add LLMAPI for summaries, tags, and routing.

That gives you a path from cute local demo to actual product.

The practical takeaway

To find similar images with Python, choose the method based on what “similar” means.

Use perceptual hashing when you want to find duplicates or near-duplicates. Use CLIP or OpenCLIP embeddings when you want semantic similarity, visual search, or text-to-image search. Use FAISS for fast local vector search. Use Qdrant or another vector database when you need persistence, filters, metadata, and production APIs.

A good image similarity system usually looks like this:

Image → pHash check → embedding → vector search → metadata filters → ranked results → review when needed

That setup works for product catalogs, stock libraries, design assets, moderation queues, visual recommendations, screenshot search, and duplicate cleanup.

Start with a small local version. Test it on real images. Tune thresholds. Then move to a vector database when the image library grows.