Make Python Search Smarter with Semantic Search

That is the classic keyword search problem. Keyword search is fast, familiar, and often good enough when users know the exact words. But users are not little SQL robots. They type vague, messy, emotional, half-formed queries like:

“the thing where payment failed”
“can I get my money back”
“docs about user limits”
“why upload broken”
“client asked about contract ending”
“that policy for cancelling subscription”
“how do I connect Slack again”

The user knows what they mean. The database does not.

Semantic search helps bridge that gap. Instead of matching only exact words, we represent text as embeddings — numerical vectors that capture meaning — then search for content that is close in meaning to the query. Pinecone’s docs describe dense vectors as vectors that represent the meaning and relationships of data, and semantic search retrieves records with dense vectors most similar to the query.

In this guide, we’ll build a Python semantic search workflow that understands what users actually mean, even when their queries are vague, typo-heavy, or painfully human.

The search problem hiding inside most apps

Most product search starts simple.

The user enters a query.
The app searches text fields.
Results come back.

That works when the query and document use the same words.

User query	Document wording	Keyword search result
refund policy	refund policy	Good
password reset	reset password	Good
invoice export	invoice export	Good
cancel subscription	subscription cancellation	Usually fine
money back	refund	Maybe bad
billing reversal	refund	Maybe bad
account locked out	login blocked	Maybe bad
cannot upload file	file import failed	Maybe bad
customer is angry about being charged twice	duplicate billing complaint	Often bad

Semantic search helps when meaning matters more than exact phrasing.

It can connect:

User says	Document says
“money back”	refund
“charged twice”	duplicate billing
“can’t log in”	authentication failure
“cancel my plan”	subscription termination
“upload broken”	file import error
“team permissions”	workspace roles
“old invoices”	billing history

This is the search users expected all along.

They just did not know the word “embeddings.”

How semantic search works in one pass

The basic workflow:

documents
→ split into chunks
→ create embeddings
→ store vectors + metadata
→ embed user query
→ compare query vector to document vectors
→ return nearest matches

Each text chunk becomes a vector.

The query also becomes a vector.

Then we compare vectors using similarity search.

Sentence Transformers documentation describes semantic search as embedding the query and corpus into the same vector space, then finding the closest embeddings based on semantic similarity.

A tiny example:

Text	Meaning
“Users can cancel subscriptions from billing settings.”	Cancellation policy
“Refunds are reviewed within 7 business days.”	Refund policy
“CSV uploads fail when files exceed 25 MB.”	Upload troubleshooting
“Admins can invite teammates from workspace settings.”	Team management

Query:

how do I stop paying for my plan?

A semantic search system can return the cancellation policy even if the exact words “stop paying” never appear.

That is the whole trick.

Why we can write this guide

We’ve spent around 6 years working with AI APIs, embeddings, semantic search, RAG workflows, Python automation, vector databases, structured outputs, and developer tutorials. We also checked current documentation from LLMAPI, Pinecone, Sentence Transformers, FAISS, and OpenAI’s Q&A guidance while preparing this guide.

The current tooling landscape is very practical. Sentence Transformers supports semantic search by computing embeddings for corpus documents and queries, then calculating similarity scores; its docs say small corpora up to about 1 million entries can use a manual implementation before moving to heavier vector search infrastructure. Pinecone supports dense-vector semantic search, sparse-vector lexical search, full-text search, and hybrid search patterns for combining semantic and keyword-style retrieval. FAISS is a library focused on efficient vector similarity search, which makes it useful when local or self-managed similarity search is enough.

The practical lesson: semantic search can start small in Python, then grow into a real retrieval system when your corpus, traffic, or relevance requirements get bigger.

What we’ll build

We’ll build this in layers.

Layer	What it does
Dataset	A small set of searchable documents
Chunking	Splits longer documents into useful pieces
Embeddings	Converts chunks and queries into vectors
Local search	Finds similar chunks with Python
Vector database option	Scales search with Pinecone-style storage
Hybrid search	Combines keyword and semantic signals
Reranking	Improves top results
LLMAPI answer layer	Turns retrieved results into helpful answers
Evaluation	Checks whether search actually improved

This structure matters because semantic search is rarely one magic function.

Good search is a pipeline.

Step 1: Create a tiny document set

Let’s start with product support content.

Create documents.py:

DOCUMENTS = [
    {
        "id": "billing_refunds",
        "title": "Refund policy",
        "text": """
Customers can request a refund within 14 days of the original payment.
Refunds are reviewed by the billing team and usually processed within 7 business days.
Duplicate charges should be reported with the invoice number and payment date.
"""
    },
    {
        "id": "billing_cancel",
        "title": "Cancel subscription",
        "text": """
Users can cancel a subscription from Billing Settings.
After cancellation, the plan remains active until the end of the current billing period.
Admins can download invoices before closing the workspace.
"""
    },
    {
        "id": "upload_limits",
        "title": "File upload limits",
        "text": """
CSV uploads support files up to 25 MB.
If an upload fails, check the file size, encoding, and required column names.
Large imports should be split into smaller files.
"""
    },
    {
        "id": "workspace_roles",
        "title": "Workspace roles",
        "text": """
Workspace owners can invite teammates and assign roles.
Admins can manage billing, integrations, and user permissions.
Members can access shared projects but cannot change billing settings.
"""
    }
]

This is tiny on purpose.

We want the logic to be easy to see before adding infrastructure.

Step 2: Install packages

Install Sentence Transformers and basic helpers:

pip install sentence-transformers numpy pandas python-dotenv openai pydantic

Sentence Transformers is a good local starting point because it gives us ready-to-use embedding models for semantic similarity and search. Its docs list models trained for semantic search and explain that query and passage embeddings can be compared with cosine similarity, dot product, or other similarity functions depending on the model.

Step 3: Build local semantic search

Create semantic_search_local.py:

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import semantic_search
from documents import DOCUMENTS


model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")


def build_corpus():
    corpus = []

    for doc in DOCUMENTS:
        corpus.append({
            "id": doc["id"],
            "title": doc["title"],
            "text": doc["text"].strip()
        })

    return corpus


def create_corpus_embeddings(corpus):
    texts = [item["text"] for item in corpus]
    return model.encode(texts, convert_to_tensor=True)


def search(query: str, top_k: int = 3):
    corpus = build_corpus()
    corpus_embeddings = create_corpus_embeddings(corpus)

    query_embedding = model.encode(query, convert_to_tensor=True)

    hits = semantic_search(
        query_embedding,
        corpus_embeddings,
        top_k=top_k
    )[0]

    results = []

    for hit in hits:
        item = corpus[hit["corpus_id"]]

        results.append({
            "id": item["id"],
            "title": item["title"],
            "text": item["text"],
            "score": float(hit["score"])
        })

    return results


if __name__ == "__main__":
    query = "how do I get my money back if I was charged twice?"

    for result in search(query):
        print(result["score"], result["title"])

Try these queries:

how do I get my money back?

I was charged twice

upload keeps breaking

who can change payment settings?

The search should find related documents even when the query does not use the exact document wording.

That is already smarter than basic keyword matching.

Step 4: Add chunking for longer documents

For real apps, documents are bigger.

A whole policy page may contain ten different topics. If we embed the entire page as one vector, the result can be too broad.

Chunking fixes that.

Create chunking.py:

def chunk_text(text: str, max_chars: int = 800, overlap: int = 100) -> list[str]:
    text = text.strip()

    if len(text) <= max_chars:
        return [text]

    chunks = []
    start = 0

    while start < len(text):
        end = min(start + max_chars, len(text))
        chunk = text[start:end].strip()

        if chunk:
            chunks.append(chunk)

        if end == len(text):
            break

        start = end - overlap

    return chunks

Now create chunk records:

from documents import DOCUMENTS
from chunking import chunk_text


def build_chunks():
    chunks = []

    for doc in DOCUMENTS:
        doc_chunks = chunk_text(doc["text"], max_chars=500, overlap=80)

        for index, chunk in enumerate(doc_chunks):
            chunks.append({
                "chunk_id": f"{doc['id']}:{index}",
                "document_id": doc["id"],
                "title": doc["title"],
                "text": chunk
            })

    return chunks

Chunking helps because semantic search retrieves the exact section that answers the query, not only the document that vaguely contains the answer.

Step 5: Search chunks instead of full documents

Update the local search:

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import semantic_search

from build_chunks import build_chunks


model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")


class LocalSemanticIndex:
    def __init__(self):
        self.chunks = build_chunks()
        self.embeddings = model.encode(
            [chunk["text"] for chunk in self.chunks],
            convert_to_tensor=True
        )

    def search(self, query: str, top_k: int = 5):
        query_embedding = model.encode(query, convert_to_tensor=True)

        hits = semantic_search(
            query_embedding,
            self.embeddings,
            top_k=top_k
        )[0]

        results = []

        for hit in hits:
            chunk = self.chunks[hit["corpus_id"]]

            results.append({
                **chunk,
                "score": float(hit["score"])
            })

        return results


if __name__ == "__main__":
    index = LocalSemanticIndex()

    results = index.search(
        "I need to stop my subscription but keep access until the end of the month"
    )

    for result in results:
        print(result["score"], result["title"], result["chunk_id"])

For small datasets, this is enough.

For larger datasets, use a vector database or FAISS.

Step 6: Add FAISS for local vector search

FAISS is useful when the corpus grows and manual similarity search gets slow. The FAISS library is dedicated to vector similarity search and is widely used for nearest-neighbor retrieval.

Install:

pip install faiss-cpu

Create semantic_search_faiss.py:

import faiss
import numpy as np
from sentence_transformers import SentenceTransformer

from build_chunks import build_chunks


model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")


class FaissSemanticIndex:
    def __init__(self):
        self.chunks = build_chunks()

        embeddings = model.encode(
            [chunk["text"] for chunk in self.chunks],
            convert_to_numpy=True,
            normalize_embeddings=True
        )

        self.embeddings = embeddings.astype("float32")

        dimension = self.embeddings.shape[1]
        self.index = faiss.IndexFlatIP(dimension)
        self.index.add(self.embeddings)

    def search(self, query: str, top_k: int = 5):
        query_embedding = model.encode(
            [query],
            convert_to_numpy=True,
            normalize_embeddings=True
        ).astype("float32")

        scores, indexes = self.index.search(query_embedding, top_k)

        results = []

        for score, item_index in zip(scores[0], indexes[0]):
            chunk = self.chunks[item_index]

            results.append({
                **chunk,
                "score": float(score)
            })

        return results

Why IndexFlatIP?

Because we normalized the embeddings, inner product behaves like cosine similarity. Sentence Transformers notes that some models produce normalized vectors where dot product, cosine similarity, and Euclidean distance can be used depending on setup.

This is a good local option for prototypes, internal tools, and smaller production systems.

Step 7: Use Pinecone when search needs to scale

A local FAISS index is great until you need managed storage, metadata filtering, multi-user data isolation, updates, scaling, and production operations.

Pinecone is one common vector database option. Its docs describe semantic search with dense vectors and show indexes that can use integrated embedding models or external embeddings. Pinecone also supports metadata, dense vector search, sparse vectors, full-text search, and hybrid search patterns.

Install:

pip install pinecone

Example shape:

from pinecone import Pinecone
from sentence_transformers import SentenceTransformer

from build_chunks import build_chunks


pc = Pinecone(api_key="YOUR_PINECONE_API_KEY")
index = pc.Index("support-search")

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")


def upsert_chunks():
    chunks = build_chunks()
    texts = [chunk["text"] for chunk in chunks]

    embeddings = model.encode(
        texts,
        convert_to_numpy=True,
        normalize_embeddings=True
    )

    records = []

    for chunk, embedding in zip(chunks, embeddings):
        records.append({
            "id": chunk["chunk_id"],
            "values": embedding.tolist(),
            "metadata": {
                "document_id": chunk["document_id"],
                "title": chunk["title"],
                "text": chunk["text"]
            }
        })

    index.upsert(vectors=records)


def search_pinecone(query: str, top_k: int = 5):
    query_embedding = model.encode(
        query,
        convert_to_numpy=True,
        normalize_embeddings=True
    )

    response = index.query(
        vector=query_embedding.tolist(),
        top_k=top_k,
        include_metadata=True
    )

    return response["matches"]

Use a vector database when you need:

Large-scale search
Fast retrieval
Metadata filters
Multi-tenant data
Frequent updates
Production monitoring
Managed indexes
Hybrid retrieval
High availability

Start local. Move managed when the product needs it.

Step 8: Add metadata filters

Semantic similarity alone is not always enough.

Sometimes users want:

Only docs from one workspace
Only active policies
Only support articles
Only product docs
Only English content
Only documents updated this year
Only invoices from one customer

Metadata filters solve this.

Example chunk metadata:

{
  "chunk_id": "billing_refunds:0",
  "document_id": "billing_refunds",
  "title": "Refund policy",
  "workspace_id": "workspace_123",
  "doc_type": "support_article",
  "language": "en",
  "updated_at": "2026-08-01"
}

Search with filters:

response = index.query(
    vector=query_embedding.tolist(),
    top_k=5,
    include_metadata=True,
    filter={
        "workspace_id": {"$eq": "workspace_123"},
        "doc_type": {"$eq": "support_article"}
    }
)

Metadata filtering makes semantic search safer and more useful.

A query should only search the content the user is allowed to see.

Step 9: Add hybrid search

Semantic search understands meaning.

Keyword search is still useful.

Why?

Because exact terms matter.

Examples:

Error codes
Invoice numbers
Product IDs
Function names
Legal clause numbers
SKU codes
Acronyms
Names
Dates
Version numbers

A user searching ERR_AUTH_402 probably does not want “general login issues.” They want that exact error.

Hybrid search combines semantic and lexical signals.

Pinecone’s search overview describes hybrid search as combining dense and sparse vectors, and its docs also discuss mixing dense vectors, sparse vectors, and full-text search fields in an index.

A simple hybrid strategy in Python:

Run semantic search.
Run keyword search.
Merge results.
Rerank combined candidates.

Very simple keyword search:

def keyword_search(query: str, chunks: list[dict], top_k: int = 5):
    query_terms = set(query.lower().split())
    results = []

    for chunk in chunks:
        text = chunk["text"].lower()
        score = sum(1 for term in query_terms if term in text)

        if score > 0:
            results.append({
                **chunk,
                "keyword_score": score
            })

    return sorted(
        results,
        key=lambda item: item["keyword_score"],
        reverse=True
    )[:top_k]

Merge candidates:

def merge_results(semantic_results, keyword_results):
    merged = {}

    for item in semantic_results:
        merged[item["chunk_id"]] = {
            **item,
            "semantic_score": item.get("score", 0),
            "keyword_score": 0
        }

    for item in keyword_results:
        existing = merged.get(item["chunk_id"], item)
        existing["keyword_score"] = item.get("keyword_score", 0)
        merged[item["chunk_id"]] = existing

    return list(merged.values())

This is basic, but it shows the idea.

Semantic search helps with meaning. Keyword search protects exact matches.

Together, they usually behave better than either one alone.

Step 10: Add reranking

Semantic search retrieves candidates.

Reranking reorders the best candidates more carefully.

A common flow:

query
→ retrieve top 20 chunks
→ rerank top 20
→ return top 5

Why rerank?

Because vector similarity is fast, but the top result is not always the best answer. Reranking can compare the query and each candidate more directly.

Use reranking when:

Search quality matters a lot
Results are close together
Documents are similar
The corpus has repeated content
Users complain about “almost right” results
You are building RAG
You need higher precision in top 3 results

A simple LLMAPI reranker can score candidates:

import json
from llmapi_client import client


def rerank_with_llmapi(query: str, candidates: list[dict], top_k: int = 5):
    compact_candidates = [
        {
            "chunk_id": item["chunk_id"],
            "title": item["title"],
            "text": item["text"]
        }
        for item in candidates
    ]

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
You rerank search results for relevance.

Return only valid JSON:
{
  "results": [
    {
      "chunk_id": "string",
      "relevance_score": 0,
      "reason": "short reason"
    }
  ]
}

Rules:
- Score from 0 to 100.
- Prefer results that directly answer the query.
- Penalize results that are only loosely related.
- Do not invent content outside the candidate text.
"""
            },
            {
                "role": "user",
                "content": json.dumps({
                    "query": query,
                    "candidates": compact_candidates
                })
            }
        ],
        temperature=0
    )

    ranking = json.loads(response.choices[0].message.content)
    scores = {
        item["chunk_id"]: item
        for item in ranking["results"]
    }

    reranked = sorted(
        candidates,
        key=lambda item: scores.get(
            item["chunk_id"],
            {}
        ).get("relevance_score", 0),
        reverse=True
    )

    return reranked[:top_k]

For high-volume search, use a dedicated reranker model instead of calling an LLM for every query.

For an MVP or internal tool, this can be enough to test whether reranking helps.

Step 11: Generate answers with LLMAPI

Semantic search returns documents.

Sometimes users want an answer.

This becomes a basic RAG workflow:

query
→ semantic search
→ top chunks
→ LLMAPI answer grounded in chunks
→ response with sources

Create llmapi_client.py:

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

client = OpenAI(
    api_key=os.environ["LLMAPI_API_KEY"],
    base_url=os.environ.get("LLMAPI_BASE_URL", "https://api.llmapi.ai/v1")
)

LLMAPI’s quick-start docs show an OpenAI-compatible chat completions flow through /v1/chat/completions, which makes it straightforward to add an answer-generation layer after retrieval.

Create answer_with_sources.py:

import json
from llmapi_client import client


def answer_from_search_results(query: str, search_results: list[dict]) -> dict:
    sources = [
        {
            "chunk_id": result["chunk_id"],
            "title": result["title"],
            "text": result["text"]
        }
        for result in search_results
    ]

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Answer the user's question using only the provided sources.

Return only valid JSON:
{
  "answer": "string",
  "source_ids": ["string"],
  "missing_information": ["string"]
}

Rules:
- Use only the source text.
- If the answer is not in the sources, say that the available documents do not contain enough information.
- Do not invent policies, numbers, dates, or steps.
- Keep the answer clear and practical.
"""
            },
            {
                "role": "user",
                "content": json.dumps({
                    "query": query,
                    "sources": sources
                })
            }
        ],
        temperature=0
    )

    return json.loads(response.choices[0].message.content)

Example:

index = LocalSemanticIndex()

query = "Can I cancel and still use the app until the month ends?"
results = index.search(query, top_k=3)

answer = answer_from_search_results(query, results)

print(answer)

Possible output:

{
  "answer": "Yes. After cancellation, the plan remains active until the end of the current billing period.",
  "source_ids": ["billing_cancel:0"],
  "missing_information": []
}

Now search becomes more than a results page.

It becomes an assistant grounded in your own content.

Step 12: Build a FastAPI search endpoint

Create app.py:

from fastapi import FastAPI, Query
from semantic_search_local import LocalSemanticIndex
from answer_with_sources import answer_from_search_results

app = FastAPI(
    title="Semantic Search API",
    description="Search documents by meaning with Python and LLMAPI.",
    version="1.0.0"
)

index = LocalSemanticIndex()


@app.get("/health")
def health_check():
    return {
        "status": "ok"
    }


@app.get("/search")
def search(
    q: str = Query(..., min_length=2),
    top_k: int = Query(5, ge=1, le=20)
):
    results = index.search(q, top_k=top_k)

    return {
        "query": q,
        "results": results
    }


@app.get("/answer")
def answer(
    q: str = Query(..., min_length=2),
    top_k: int = Query(5, ge=1, le=10)
):
    results = index.search(q, top_k=top_k)
    answer_payload = answer_from_search_results(q, results)

    return {
        "query": q,
        "answer": answer_payload,
        "search_results": results
    }

Run it:

uvicorn app:app --reload

Try:

curl "http://127.0.0.1:8000/search?q=I%20was%20charged%20twice"

And:

curl "http://127.0.0.1:8000/answer?q=How%20do%20I%20stop%20paying%20but%20keep%20access"

Now you have both semantic search and a grounded answer endpoint.

Step 13: Add query cleanup

Users type weird things.

That is normal.

Before embedding, lightly clean queries.

Create query_cleaning.py:

import re


def clean_query(query: str) -> str:
    query = query.strip()
    query = re.sub(r"\s+", " ", query)
    return query

Do not over-clean.

You usually want to preserve the user’s wording because embeddings can handle natural language. Removing too much can make queries worse.

Good cleanup:

Trim whitespace
Normalize repeated spaces
Remove empty input
Detect extremely long queries
Remove control characters

Risky cleanup:

Removing “not”
Removing all punctuation
Aggressive stemming
Dropping short words blindly
Rewriting user intent without review

Semantic search works partly because users can be natural.

Do not turn the query back into 2006 search syntax.

Step 14: Add query expansion with LLMAPI

Sometimes users type very short queries.

Example:

billing issue

That could mean refunds, duplicate charges, invoices, payment failures, cancellation, plan upgrades, or receipts.

LLMAPI can expand vague queries into search-friendly alternatives.

Create query_expansion.py:

import json
from llmapi_client import client


def expand_query(query: str) -> list[str]:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Create search query variants.

Return only valid JSON:
{
  "queries": ["string"]
}

Rules:
- Keep the original user intent.
- Add up to 4 useful variants.
- Do not add unrelated topics.
- Prefer natural language queries.
"""
            },
            {
                "role": "user",
                "content": query
            }
        ],
        temperature=0.2
    )

    payload = json.loads(response.choices[0].message.content)
    return [query] + payload["queries"]

Then search each variant and merge results.

This helps when user queries are too vague for a single embedding.

Use carefully. Query expansion can also widen the search too much.

Step 15: Add result explanations

Search results are better when users can see why something matched.

Use LLMAPI to explain a top result.

import json
from llmapi_client import client


def explain_search_match(query: str, result: dict) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": """
Explain why this search result is relevant to the user's query.

Rules:
- Use only the result text.
- Keep it to one sentence.
- Do not overstate relevance.
"""
            },
            {
                "role": "user",
                "content": json.dumps({
                    "query": query,
                    "result_title": result["title"],
                    "result_text": result["text"]
                })
            }
        ],
        temperature=0
    )

    return response.choices[0].message.content.strip()

Example:

This result is relevant because it explains that duplicate charges should be reported with invoice details.

This makes search feel less random.

It is especially useful for internal knowledge bases, legal/admin search, support docs, and research tools.

Step 16: Evaluate search quality

Do not judge search quality by clicking around for three minutes and saying “seems fine.”

Create evaluation queries.

Example:

[
  {
    "query": "I was charged twice",
    "expected_document_id": "billing_refunds"
  },
  {
    "query": "how do I stop my plan",
    "expected_document_id": "billing_cancel"
  },
  {
    "query": "file import keeps failing",
    "expected_document_id": "upload_limits"
  },
  {
    "query": "who can invite users",
    "expected_document_id": "workspace_roles"
  }
]

Simple evaluator:

def evaluate_search(index, eval_cases: list[dict], top_k: int = 3):
    hits_at_1 = 0
    hits_at_k = 0

    for case in eval_cases:
        results = index.search(case["query"], top_k=top_k)
        document_ids = [result["document_id"] for result in results]

        if document_ids and document_ids[0] == case["expected_document_id"]:
            hits_at_1 += 1

        if case["expected_document_id"] in document_ids:
            hits_at_k += 1

    total = len(eval_cases)

    return {
        "total": total,
        "hit_rate_at_1": hits_at_1 / total if total else 0,
        f"hit_rate_at_{top_k}": hits_at_k / total if total else 0
    }

Track:

Metric	Meaning
Hit rate@1	Correct result is first
Hit rate@3	Correct result is in top 3
MRR	Correct result appears higher
Recall@k	Relevant results are retrieved
Precision@k	Top results are actually useful
No-answer accuracy	System refuses when content is missing
User click-through	Real user behavior
Search-to-resolution	Did search solve the task?

Semantic search should be measured like a product feature, not admired like a magic trick.

Step 17: Handle missing answers

A search system should know when it does not have enough information.

Example user query:

Do you support wire transfers in Brazil?

If your docs never mention Brazil or wire transfers, the system should not make something up.

For answer generation, add a strict rule:

If the answer is not clearly supported by the retrieved sources, say the available documents do not contain enough information.

Also use score thresholds.

Example:

def should_answer(results: list[dict], min_score: float = 0.35) -> bool:
    if not results:
        return False

    return results[0]["score"] >= min_score

Then:

results = index.search(query, top_k=5)

if not should_answer(results):
    return {
        "answer": "I could not find enough relevant information in the available documents.",
        "source_ids": [],
        "missing_information": [query]
    }

Thresholds need testing.

Do not copy a number from a tutorial and treat it like gravity.

Step 18: Update the index when documents change

Semantic search has two data layers:

Original document text.
Embeddings generated from that text.

When documents change, embeddings must be updated too.

Track:

Event	Action
New document	Chunk and embed
Edited document	Re-chunk and re-embed
Deleted document	Remove chunks and vectors
Permission change	Update metadata/filtering
Language change	Re-embed if needed
Chunking strategy change	Rebuild index
Embedding model change	Rebuild index

Store model metadata:

{
  "chunk_id": "billing_refunds:0",
  "embedding_model": "sentence-transformers/all-MiniLM-L6-v2",
  "chunking_version": "v1",
  "document_version": "2026-08-24"
}

This helps when search results change and someone asks why.

Someone will always ask why.

Step 19: Add permissions and tenant isolation

Semantic search can accidentally leak information if metadata filters are missing.

For multi-user apps, always filter by access.

Examples:

App	Required filter
Team workspace	workspace_id
Enterprise docs	organization_id
Private user notes	user_id
Role-based docs	role or permission group
Region-specific policies	region
Draft/public docs	status
Customer records	customer_id

Search should happen inside the user’s allowed content scope.

A good query flow:

user request
→ auth check
→ build allowed metadata filter
→ semantic search inside allowed scope
→ answer with allowed sources only

Do not rely on the LLM to “not mention private data.”

Filter retrieval before the model sees anything.

Step 20: Common mistakes

Mistake	Better approach
Embedding huge documents as one vector	Chunk documents
Using semantic search for exact IDs only	Add keyword or hybrid search
No metadata filters	Filter by workspace/user/permissions
No eval queries	Measure search quality
No missing-answer behavior	Add thresholds and refusal rules
Sending too many chunks to LLMAPI	Retrieve and rerank first
No document versioning	Track embedding and chunk versions
Ignoring deleted docs	Remove stale vectors
No source IDs	Return citations/source chunks
Over-cleaning queries	Keep natural language
No reranking	Rerank when top results are weak
Treating similarity score as truth	Test thresholds with real data

The most annoying semantic search bug is a result that feels close but answers the wrong question.

Reranking, filters, and evaluation are how we fight that.

Where LLMAPI helps

LLMAPI is useful around the semantic search pipeline.

Use it for:

Need	LLMAPI role
Answer generation	Turn retrieved chunks into grounded answers
Query expansion	Create better variants of vague queries
Result explanations	Explain why a result matched
Summaries	Summarize retrieved documents
Reranking	Score candidate results for relevance
Classification	Route query by topic before search
Missing-info handling	Explain what the docs do not answer
Search UX copy	Make results easier to understand
RAG workflows	Combine search + answer generation

A good search product flow:

user query
→ clean query
→ embed query
→ semantic/hybrid search
→ rerank
→ LLMAPI grounded answer
→ source-backed response

LLMAPI’s role is to make retrieved content useful. The search layer still needs to retrieve the right content first.

Closing notes for builders

Semantic search makes Python search feel less brittle because it listens for meaning, not only exact wording.

That matters when users type like humans instead of documentation authors. They say “money back” when the docs say “refund.” They say “upload broken” when the support article says “CSV import failed.” They ask vague, emotional, half-complete questions because they are trying to solve a problem, not impress the search bar.

Start with a small local embedding index. Add chunking. Measure whether the right documents appear in the top results. Move to FAISS or a vector database when scale demands it. Add metadata filters before you touch private or multi-tenant content. Use hybrid search when exact terms matter. Use reranking when the top results are close. Use LLMAPI when you want grounded answers, summaries, query expansion, or result explanations.

That gives you a search system that feels much closer to how users actually think — messy queries included.

You might also want to read

LLM Guides Aug 24, 2026

Multimodal Chat with LLMAPI: Talk, Upload, Analyze

LLM Guides Aug 24, 2026

MiniMax on LLMAPI: Language, Speech, and Video AI in One Place

LLM Guides Aug 24, 2026

How to Automate Financial Documents with Mindee OCR

LLM Guides Aug 21, 2026

The AI Assistant That Hunts Down Busywork

Deploy in minutes

Get My API Key