paraphrase-MiniLM-L6-v2: Self-Hosting & Deployment Guide
paraphrase-MiniLM-L6-v2 is a compact sentence-transformers embedding model from Sentence Transformers that maps text into 384-dimensional vectors.
What is paraphrase-MiniLM-L6-v2?
paraphrase-MiniLM-L6-v2 is a sentence-transformers model that encodes sentences and short paragraphs into 384-dimensional dense vector embeddings. It is mainly used for semantic search and information retrieval, where it helps find relevant texts based on meaning rather than keywords. It is also widely applied to clustering and paraphrase or similarity detection across large text collections. The model belongs to the MiniLM-based sentence-transformers family, related to models such as all-MiniLM-L6-v2 and paraphrase-multilingual-MiniLM-L12-v2.
How to run paraphrase-MiniLM-L6-v2 yourself
Open weights (apache-2.0), 23M parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.
paraphrase-MiniLM-L6-v2 VRAM requirements
| Precision / quantization | Weights | Figure | Source |
|---|---|---|---|
| BF16 (full precision) | 0.05 GB | Estimate | 23M params × 2 bytes |
File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.
GPU requirements: deploy paraphrase-MiniLM-L6-v2 on your hardware
| Hardware | BF16 (full precision) |
|---|---|
| NVIDIA H200141 GB | 1× |
| NVIDIA H10080 GB | 1× |
| NVIDIA A100 80GB80 GB | 1× |
| NVIDIA L40S48 GB | 1× |
GPUs needed per variant.
| Hardware | BF16 (full precision) |
|---|---|
| AWS p5.48xlarge8× H100 · 640 GB | Fits |
| AWS p4de.24xlarge8× A100 80GB · 640 GB | Fits |
| AWS g6e.12xlarge4× L40S · 192 GB | Fits |
| Google Cloud a3-highgpu-8g8× H100 · 640 GB | Fits |
| Azure ND H100 v58× H100 · 640 GB | Fits |
Whole-instance GPU memory; public instance specs.
Not on LLM.API yet
We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.
Providers
Different companies host the same model. OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto (highest tool-calling accuracy).
| Provider | Input | Output | Cache read /M | Latency | Throughput | Uptime |
|---|---|---|---|---|---|---|
|
|
~$0.10 per 1M tokens | ~$0.80 per 1M tokens | — | ~250ms | ~2,000 tps | ~99.9% |
|
|
~$1.20 per 1M tokens | ~$1.20 per 1M tokens | — | ~220ms | ~1.5k tps | 99.9% |
AWS SageMaker (MiniLM-based endpoint)
|
~$1.50 per 1M tokens | ~$1.50 per 1M tokens | — | ~250ms | ~1k tps | 99.9% |
|
|
~$1.60 per 1M tokens | ~$1.60 per 1M tokens | — | ~260ms | ~900 tps | 99.9% |
No hosted provider publishes a price for this exact checkpoint on OpenRouter; self-hosting cost depends on your GPUs. Checked weekly. Last updated 11 Oct 2026.
Try this model
Test paraphrase-MiniLM-L6-v2 right here — free to start.
Suggestions for your first prompt
Not available on LLM API yet
We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.
Browse available modelsparaphrase-MiniLM-L6-v2: Self-Hosting & Deployment Guide in production
What the model is good at.
-
Sentence Embeddings
Maps sentences and short paragraphs into 384-dimensional dense vector embeddings that capture semantic meaning for downstream applications.
-
Semantic Similarity
Computes similarity between sentence embeddings, enabling comparison of meaning for paraphrase detection and related text identification tasks.
-
Semantic Search
Supports semantic search by embedding queries and documents into the same vector space for relevance-based retrieval using similarity scores.
-
Text Clustering
Enables clustering of texts by encoding them as vectors, allowing grouping of semantically related sentences or documents together.
-
Efficient Deployment
Compact transformer model with about 22.7M parameters, suitable for resource-constrained environments and real-time text embedding workloads.
What you get when you call paraphrase-MiniLM-L6-v2: Self-Hosting & Deployment Guide through LLM.API.
-
Unified AI Routing
Automatically route each request to the best-fit model across providers based on cost, latency, and quality—without changing your integration.
One endpoint, every model -
Cost-Aware Orchestration
Optimize spend by dynamically mixing premium and budget models, enforcing price caps, and simulating cost impact before changes hit production.
More output, less spend -
Resilient Fallback Engine
Stay online during model or provider outages with automatic retries, failover routes, and graceful degradation tuned to your app’s SLAs.
Never go dark -
End-to-End Observability
Trace every request across providers with logs, metrics, and evaluations so you can debug prompts, track regressions, and confidently ship changes.
See every token -
Task-Level Abstractions
Call high-level tasks like chat, tools, embeddings, and rerankers through a consistent API so you can swap models without rewriting logic.
Tasks, not providers -
High-Throughput Batch API
Process large workloads efficiently with batched inference, parallel execution, and provider-aware rate limits for faster, cheaper bulk operations.
Scale jobs, not code
When to Use — When NOT to Use
Use it if...
- You need lightweight sentence embeddings for semantic similarity with tight memory constraints.
- You need fast paraphrase detection or duplicate question identification at scale.
- Your use case involves clustering short texts or sentences into semantic groups.
- Your use case involves building a simple semantic search over short documents.
- You need a compact model for on-device or edge semantic text applications.
- Your use case involves generating embeddings as features for downstream ML classifiers.
Avoid if...
- You need state-of-the-art accuracy on complex semantic similarity or entailment benchmarks.
- Your workload requires understanding long documents far beyond a few sentences.
- You need multilingual support beyond the primarily English capabilities of this model.
- Your workload requires domain-specific embeddings tuned for legal, medical, or scientific texts.
- You need generative capabilities like summarization, translation, or question answering directly.
- Your workload requires robust performance on noisy, code-mixed, or highly informal text.
COMMUNITY
What developers say about paraphrase-MiniLM-L6-v2
Summarised from public developer discussions and write-ups. Opinions are the sources’, not LLM.API’s.
- Developers on GitHub report performance issues when using the model for very short text retrieval (1-4 words), noting that semantic search results may occasionally favor irrelevant matches over expected ones.
- A developer on Hugging Face raised concerns about large Docker deployment sizes due to dependencies; maintainers suggested installing a CPU-only version of PyTorch to significantly reduce the environment's footprint.
- Developers have actively sought to extend the model for multilingual support, though maintainers recommend using specialized models like MS MARCO or specific distillation techniques for those use cases.
SOURCES
Frequently Asked Questions
-
What is paraphrase-MiniLM-L6-v2?
paraphrase-MiniLM-L6-v2 is a Sentence Transformers model that encodes sentences into dense vectors optimized for semantic similarity and paraphrase detection.
-
What is paraphrase-MiniLM-L6-v2 best suited for?
It is best for semantic search, clustering, duplicate detection, and measuring sentence-level similarity in low-latency, resource-constrained applications.
-
How much does it cost to use paraphrase-MiniLM-L6-v2 via LLM.API?
LLM.API pricing is usage-based; check the paraphrase-MiniLM-L6-v2 entry in the LLM.API pricing page for the latest per-request and per-token rates.
-
What is the context window of paraphrase-MiniLM-L6-v2?
paraphrase-MiniLM-L6-v2 is typically used with short texts or sentences, and does not support long-document context windows like large generative LLMs.
-
How fast is paraphrase-MiniLM-L6-v2 on LLM.API?
As a small MiniLM-based encoder, it provides low-latency embeddings, making it suitable for real-time or interactive use cases on LLM.API.
-
What modalities does paraphrase-MiniLM-L6-v2 support?
paraphrase-MiniLM-L6-v2 is a text-only model that produces fixed-size vector embeddings from input text.
-
How do I access paraphrase-MiniLM-L6-v2 through LLM.API?
Call the LLM.API embeddings endpoint with the model name paraphrase-MiniLM-L6-v2 and your text inputs, using your LLM.API authentication key.
-
How does paraphrase-MiniLM-L6-v2 compare to larger Sentence Transformers models?
It trades some embedding quality for significantly lower latency and memory usage, making it ideal when performance and cost are priorities.
-
Does paraphrase-MiniLM-L6-v2 support multilingual text?
paraphrase-MiniLM-L6-v2 is primarily optimized for English; performance on other languages may be inconsistent and should be empirically validated.
-
What are the main limitations of paraphrase-MiniLM-L6-v2?
It may underperform on complex reasoning, domain-specific jargon, or long documents compared to larger, more specialized embedding or generative models.
COMPARE
Competitive Models
-
Text Embedding 3 Small
Text Embedding 3 Small is an OpenAI embedding model optimized for low-latency, low-cost vector representations of text. It offers strong semantic performance while being suitable for large-scale or resource-constrained applications.
-
Text Embedding Ada 002
text-embedding-ada-002 is an OpenAI embedding model that converts text into numerical vectors for measuring semantic similarity. It is an improved, more performant successor to earlier Ada-based embedding models and became a widely used default for production applications.
Get one key to every model
Swap your API key. Keep your code.