E5-Base-v2: Self-Hosting & Deployment Guide
E5-Base-v2 is an English sentence and paragraph embedding model from Intfloat that encodes text into a 768-dimensional dense vector space, optimized for high-quality semantic similarity tasks.
What is E5-Base-v2?
E5-Base-v2 is a transformer-based text embedding model that maps English sentences and paragraphs to 768-dimensional vectors for semantic representation. It is mainly used for semantic search, document and passage retrieval, similarity scoring, and clustering in information retrieval systems. It also serves as a backbone encoder for downstream tasks such as reranking, retrieval-augmented generation, and domain-specific search applications. E5-Base-v2 belongs to Intfloat’s E5 family of embedding models, which includes earlier E5 variants and larger v2 models like E5-Large-v2.
How to run E5-Base-v2 yourself
Open weights (mit), 109M parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.
E5-Base-v2 VRAM requirements
| Precision / quantization | Weights | Figure | Source |
|---|---|---|---|
| BF16 (full precision) | 0.22 GB | Estimate | 109M params × 2 bytes |
| GGUF Q4_K_M | 0.07 GB | File size | mradermacher/e5-base-v2-i1-GGUF |
| GGUF Q2_K | 0.05 GB | File size | mradermacher/e5-base-v2-i1-GGUF |
File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.
GPU requirements: deploy E5-Base-v2 on your hardware
| Hardware | BF16 (full precision) | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|---|
| NVIDIA H200141 GB | 1× | 1× | 1× |
| NVIDIA H10080 GB | 1× | 1× | 1× |
| NVIDIA A100 80GB80 GB | 1× | 1× | 1× |
| NVIDIA L40S48 GB | 1× | 1× | 1× |
GPUs needed per variant.
| Hardware | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|
| RTX 509032 GB | 1× | 1× |
| RTX 409024 GB | 1× | 1× |
| RTX 309024 GB | 1× | 1× |
GGUF via llama.cpp; more cards or CPU offload for larger files.
| Hardware | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|
| Mac, 512 GBM3 Ultra | Fits | Fits |
| Mac, 192 GBM2 Ultra | Fits | Fits |
| Mac, 128 GBM4 Max / M3 Max | Fits | Fits |
| Mac, 64 GBM4 Pro / Max | Fits | Fits |
Assumes ~75% of unified memory is usable by the GPU.
| Hardware | BF16 (full precision) | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|---|
| AWS p5.48xlarge8× H100 · 640 GB | Fits | Fits | Fits |
| AWS p4de.24xlarge8× A100 80GB · 640 GB | Fits | Fits | Fits |
| AWS g6e.12xlarge4× L40S · 192 GB | Fits | Fits | Fits |
| Google Cloud a3-highgpu-8g8× H100 · 640 GB | Fits | Fits | Fits |
| Azure ND H100 v58× H100 · 640 GB | Fits | Fits | Fits |
Whole-instance GPU memory; public instance specs.
Not on LLM.API yet
We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.
Providers
Different companies host the same model. OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto (highest tool-calling accuracy).
| Provider | Input | Output | Cache read /M | Latency | Throughput | Uptime |
|---|---|---|---|---|---|---|
|
|
~$0.10 | ~$0.00 | — | ~220ms | ~2500 tps | ~99.9% |
|
|
$0.02 | $0.00 | — | ~250ms | ~3000 tps | 99.9% |
|
|
~$0.025 | $0.00 | — | ~260ms | ~2800 tps | 99.9% |
AWS Bedrock (Cohere Embed)
|
~$0.03 | $0.00 | — | ~300ms | ~2200 tps | 99.9% |
No hosted provider publishes a price for this exact checkpoint on OpenRouter; self-hosting cost depends on your GPUs. Checked weekly. Last updated 4 Oct 2026.
Try this model
Test E5-Base-v2 right here — free to start.
Suggestions for your first prompt
Not available on LLM API yet
We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.
Browse available modelsHow to Run E5-Base-v2: GPU Requirements, VRAM & vLLM in production
What the model is good at.
-
Text Embedding
Encodes English sentences and paragraphs into 768-dimensional dense vectors for downstream machine learning and NLP applications.
-
Semantic Search
Generates embeddings optimized for semantic search, enabling retrieval of relevant documents based on meaning rather than keywords.
-
Sentence Similarity
Produces high-quality embeddings suitable for computing semantic similarity scores between sentences, queries, and documents.
-
Text Clustering
Supports grouping related texts by embedding them in a shared vector space, facilitating unsupervised clustering and topic exploration.
-
Vector Retrieval
Integrates into retrieval pipelines as a dense retriever model, powering vector databases and hybrid search systems.
What you get when you call How to Run E5-Base-v2: GPU Requirements, VRAM & vLLM through LLM.API.
-
Unified AI Routing
Dynamically route each request to the best model across providers based on latency, cost, or quality. One integration, continuously optimized decisions.
One endpoint, all models. -
Cost-Aware Orchestration
Control and minimize spend with per-model pricing visibility, routing policies, and automatic fallbacks to cheaper equivalents without code changes.
Reduce AI cost at scale. -
Resilient Fallback Logic
Keep your app online with built-in retries and cross-provider failover when a model or region degrades, no custom reliability code required.
Never ship single-vendor risk. -
End-to-End Observability
Trace every call with latency, errors, tokens, and provider breakdowns in one place. Debug, optimize, and compare models with production-grade telemetry.
See every token, everywhere. -
Task-Level Abstractions
Describe what you need—chat, generation, extraction, tools—and let LLM.API select and tune the right models and prompts for each task.
Think tasks, not providers. -
High-Throughput Batch
Submit thousands of requests in a single batch API call with smart chunking, parallelism, and retries to saturate provider capacity safely.
Batch at production scale.
When to Use — When NOT to Use
Use it if...
- You need a strong general-purpose text embedding model for semantic search applications.
- You need multilingual sentence embeddings covering many languages with a single compact model.
- Your use case involves building retrieval-augmented generation systems needing high-quality dense retrieval.
- You need cost-efficient embeddings for large-scale document indexing and similarity search.
- Your use case involves clustering or topic modeling on short to medium-length text passages.
- You need to power recommendation or matching systems based on semantic text similarity.
- Your use case involves reranking candidate documents using cosine similarity between embeddings.
Avoid if...
- You need a generative language model for text completion, dialogue, or content creation.
- Your workload requires reasoning over very long documents beyond typical embedding input limits.
- You need domain-specialized embeddings, like code or biomedical text, with state-of-the-art performance.
- Your workload requires real-time token-by-token streaming or interactive conversational behavior.
- You need structured outputs such as JSON, SQL queries, or function-call arguments directly.
- Your workload requires fine-grained token-level tasks like sequence labeling or tagging.
- You need cross-modal embeddings aligning text with images, audio, or video representations.
COMMUNITY
What developers say about How to Run E5-Base-v2
Summarised from public developer discussions and write-ups. Opinions are the sources’, not LLM.API’s.
- Developers emphasize that E5 models are strictly prefix-dependent, requiring "query: " and "passage: " tags; omitting these prefixes is a common mistake that significantly degrades performance in production.
- Developers at Superlinked report that e5-base-v2 is a primary choice for teams outgrowing smaller models like all-MiniLM-L6-v2, offering a balance of 109M parameters and 58ms latency.
- Engineers at The Neural Base recommend caching the model globally during deployment, noting that loading it is a blocking operation that can take up to 23 seconds.
SOURCES
Frequently Asked Questions
-
What is E5-Base-v2?
E5-Base-v2 is an Intfloat text-embedding model designed for high-quality semantic search, retrieval, and clustering tasks.
-
What tasks is E5-Base-v2 best suited for?
E5-Base-v2 is best for dense retrieval, semantic similarity, reranking, and building vector search over documents, queries, and short passages.
-
How is E5-Base-v2 priced when used through LLM.API?
E5-Base-v2 usage on LLM.API is billed per input token or character, following LLM.API’s unified metered pricing for embedding models.
-
What is the context window of E5-Base-v2?
E5-Base-v2 typically supports input texts up to a few thousand tokens, after which inputs should be chunked before embedding.
-
How fast is E5-Base-v2 in terms of latency?
E5-Base-v2 generally provides low-latency embedding generation suitable for real-time or near-real-time search applications, depending on request size and concurrency.
-
What modalities does E5-Base-v2 support?
E5-Base-v2 is a text-only model that converts text inputs into dense vector embeddings.
-
How do I call E5-Base-v2 via the LLM.API gateway?
You can select the E5-Base-v2 model name in LLM.API’s embeddings endpoint, passing your text inputs and receiving embedding vectors in the response.
-
How does E5-Base-v2 compare to larger embedding models?
E5-Base-v2 generally offers a strong quality–performance tradeoff, with smaller size and lower cost than many larger embedding models while maintaining competitive retrieval quality.
-
What are the main limitations of E5-Base-v2?
E5-Base-v2 may struggle with very long documents, highly specialized domains, and tasks requiring generative output or multimodal understanding.
-
Can I use E5-Base-v2 for multilingual embeddings?
E5-Base-v2 is primarily optimized for English, so performance on other languages may be less reliable compared with dedicated multilingual embedding models.
COMPARE
Competitive Models
-
Text Embedding 3 Small
Text Embedding 3 Small is an OpenAI embedding model optimized for low-latency, low-cost vector representations of text. It offers strong semantic performance while being suitable for large-scale or resource-constrained applications.
-
Text Embedding Ada 002
text-embedding-ada-002 is an OpenAI embedding model that converts text into numerical vectors for measuring semantic similarity. It is an improved, more performant successor to earlier Ada-based embedding models and became a widely used default for production applications.
Get one key to every model
Swap your API key. Keep your code.