Bonus: Top up now and we'll double your first deposit — get x2 credits instantly.
Get Started
Sign Up
Sign Up

bge-m3: Self-Hosting & Deployment Guide

bge-m3 is a multilingual text embedding model from BAAI that produces dense, sparse, and ColBERT-style multi-vector embeddings in a single pass, enabling powerful hybrid retrieval.

Browse available models

What is bge-m3?

bge-m3 is a multi-functionality, multilingual, multi-granularity text embedding model developed by BAAI that outputs dense, sparse, and ColBERT-style embeddings simultaneously. It is primarily used for information retrieval, semantic search, and retrieval-augmented generation, where a single model can power dense, sparse (lexical), and hybrid search pipelines. It also supports over 100 languages and long documents (up to around 8k tokens) for use cases like cross-lingual search, question answering over large corpora, and document similarity. It belongs to BAAI’s BGE (Beijing General Embedding) family, extending earlier BGE embedding models with unified training for dense, sparse, and multi-vector retrieval.

How to run bge-m3 yourself

Open weights (mit). Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.

bge-m3 VRAM requirements

Precision / quantizationWeightsFigureSource
Full precision weights2.3 GBFile sizeBAAI/bge-m3
FP81.4 GBFile sizeForturne/bge-m3-FP8
MLX 4-bit0.32 GBFile sizemlx-community/bge-m3-mlx-4bit
GGUF Q8_00.63 GBFile sizegpustack/bge-m3-GGUF
GGUF Q4_K_M0.44 GBFile sizegpustack/bge-m3-GGUF
GGUF Q2_K0.37 GBFile sizegpustack/bge-m3-GGUF

File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.

GPU requirements: deploy bge-m3 on your hardware

HardwareFull precision weightsFP8MLX 4-bitGGUF Q8_0GGUF Q4_K_MGGUF Q2_K
NVIDIA H200141 GB1×1×1×1×1×1×
NVIDIA H10080 GB1×1×1×1×1×1×
NVIDIA A100 80GB80 GB1×1×1×1×1×1×
NVIDIA L40S48 GB1×1×1×1×1×1×

GPUs needed per variant.

HardwareGGUF Q8_0GGUF Q4_K_MGGUF Q2_K
RTX 509032 GB1×1×1×
RTX 409024 GB1×1×1×
RTX 309024 GB1×1×1×

GGUF via llama.cpp; more cards or CPU offload for larger files.

HardwareGGUF Q8_0GGUF Q4_K_MGGUF Q2_KMLX 4-bit
Mac, 512 GBM3 UltraFitsFitsFitsFits
Mac, 192 GBM2 UltraFitsFitsFitsFits
Mac, 128 GBM4 Max / M3 MaxFitsFitsFitsFits
Mac, 64 GBM4 Pro / MaxFitsFitsFitsFits

Assumes ~75% of unified memory is usable by the GPU.

HardwareFull precision weightsFP8MLX 4-bitGGUF Q8_0GGUF Q4_K_MGGUF Q2_K
AWS p5.48xlarge8× H100 · 640 GBFitsFitsFitsFitsFitsFits
AWS p4de.24xlarge8× A100 80GB · 640 GBFitsFitsFitsFitsFitsFits
AWS g6e.12xlarge4× L40S · 192 GBFitsFitsFitsFitsFitsFits
Google Cloud a3-highgpu-8g8× H100 · 640 GBFitsFitsFitsFitsFitsFits
Azure ND H100 v58× H100 · 640 GBFitsFitsFitsFitsFitsFits

Whole-instance GPU memory; public instance specs.

Not on LLM.API yet

We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.

Browse available models

Providers

Different companies host the same model. OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto (highest tool-calling accuracy).

Provider Input Output Cache read /M Latency Throughput Uptime
BAAI (Official API) ~$0.10 per 1M tokens $0.00 per 1M tokens — ~180ms ~40 kTkn/s ~99.5%
Fireworks AI ~$0.06 per 1M tokens $0.00 per 1M tokens — ~150ms ~60 kTkn/s ~99.9%
Together AI ~$0.07 per 1M tokens $0.00 per 1M tokens — ~160ms ~55 kTkn/s ~99.9%
Replicate ~$0.12 per 1M tokens $0.00 per 1M tokens — ~220ms ~30 kTkn/s ~99.0%

No hosted provider publishes a price for this exact checkpoint on OpenRouter; self-hosting cost depends on your GPUs. Checked weekly. Last updated 4 Oct 2026.

Try this model

Test bge-m3 right here — free to start.

bge-m3
Hi! Want to test the model?

Suggestions for your first prompt

Not available on LLM API yet

We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.

Browse available models

bge-m3: Self-Hosting & Deployment Guide in production

What the model is good at.

  • Dense Retrieval

    Generates high-quality dense text embeddings for semantic similarity search, ranking, and retrieval across many tasks and domains.

  • Sparse Retrieval

    Produces sparse lexical-token representations enabling BM25‑like keyword matching, hybrid search, and improved recall in information retrieval.

  • Multi-Vector Embeddings

    Outputs ColBERT-style multi-vector embeddings for fine-grained late interaction retrieval, improving accuracy on complex search queries.

  • Multilingual Support

    Supports over one hundred languages in a shared embedding space, enabling cross-lingual search, retrieval, and comparison of text.

  • Long-Context Encoding

    Encodes long texts, from short sentences to multi-thousand-token documents, into unified embeddings suitable for RAG pipelines.

What you get when you call bge-m3: Self-Hosting & Deployment Guide through LLM.API.

  • Intelligent AI Routing

    Automatically route each request to the optimal model or provider based on latency, cost, and quality—without changing your integration or client code.

    One endpoint, any model

  • Cost-Aware Controls

    Enforce per-key, per-project, and per-model budgets while auto-selecting cheaper equivalents so you never blow past spend limits in production.

    Predictable spend at scale

  • Resilient Fallback Logic

    Define multi-provider failover policies so traffic transparently reroutes on timeouts, errors, or quota issues—no manual retries or brittle custom logic.

    Stay online, even upstream

  • End-to-End Observability

    Get centralized logs, traces, and metrics across every provider, model, and project to debug prompts, track latency, and optimize performance in one place.

    See every token flow

  • Task-Level Abstractions

    Call high-level tasks like chat, tools, RAG, and scoring instead of provider-specific APIs, so you can swap models without rewriting business logic.

    Code to tasks, not vendors

  • High-Throughput Batch

    Run large-scale batch inference jobs across providers with automatic chunking, retries, and concurrency control to maximize throughput and minimize unit cost.

    Bulk inference made easy

When to Use — When NOT to Use

Use it if...

  • You need a strong general-purpose embedding model that supports multiple languages efficiently.
  • You need text, code, and retrieval tasks covered by a single unified embedding model.
  • Your use case involves semantic search or dense retrieval over large multilingual document corpora.
  • Your use case involves building rerankers or hybrid search pipelines using high-quality embeddings.
  • You need compact embeddings that balance retrieval quality and storage or memory constraints.
  • Your use case involves retrieval-augmented generation requiring robust cross-lingual semantic similarity.
  • You need an open-source embedding model that can run on-premise or offline.

Avoid if...

  • You need generative capabilities like text completion or chat, not just embeddings.
  • Your workload requires extremely long-context understanding beyond what typical embedding models handle.
  • You need embeddings optimized specifically for images, audio, or multimodal content.
  • Your workload requires fine-grained token-level reasoning instead of sentence or document embeddings.
  • You need a fully managed, production-ready API service with enterprise SLAs from the provider.
  • Your workload requires models trained on very domain-specific data like medical records.
  • You need embeddings explicitly aligned for safety-sensitive applications with strong guardrails.

COMMUNITY

What developers say about bge-m3

LMArena—Speed—Sources: LMArena, Artificial Analysis

Summarised from public developer discussions and write-ups. Opinions are the sources’, not LLM.API’s.

  • Developers on GitHub and Hugging Face value BGE-M3 for its ability to generate dense, sparse, and multi-vector (ColBERT) embeddings in a single pass, enabling efficient hybrid search pipelines.
  • Engineers in optimization discussions note that inference speed can be accelerated by 2-3x using tools like Infinity on GPU, or via ONNX for high-performance CPU inference.
  • Some developers report challenges with specialized features like sparse retrieval in standard inference frameworks, as these require accessing unpooled model outputs rather than just final dense embeddings.

SOURCES

Frequently Asked Questions

  • What is bge-m3?

    bge-m3 is a BAAI embedding model that supports multi-lingual, multi-function, and multi-granularity text and retrieval tasks.

  • What is bge-m3 best suited for?

    bge-m3 is best suited for semantic search, dense retrieval, reranking, and building multilingual retrieval-augmented generation systems.

  • What context window does bge-m3 support?

    bge-m3 typically processes sequences up to 512 tokens per input text when generating embeddings.

  • How fast is bge-m3 in terms of latency?

    bge-m3 is relatively lightweight and can generate embeddings with low latency on modern GPUs for typical retrieval workloads.

  • What modalities does bge-m3 support?

    bge-m3 is a text-only embedding model and does not support image, audio, or video inputs.

  • How is bge-m3 priced when accessed through LLM.API?

    LLM.API usage-based pricing for bge-m3 is per-token for input text embeddings and is configured by the LLM.API platform, not BAAI.

  • How do I access bge-m3 via LLM.API?

    You call the LLM.API embeddings endpoint specifying the provider as BAAI and the model name as bge-m3 in your request parameters.

  • How does bge-m3 compare to other embedding models?

    bge-m3 offers strong multilingual retrieval quality and flexible embedding functions compared to many English-only or single-task embedding models.

  • What are the main limitations of bge-m3?

    bge-m3 cannot generate or chat, only embeds text, and its performance may degrade on extremely long documents or unsupported languages.

  • Can bge-m3 handle both query and document embeddings?

    Yes, bge-m3 supports using different instruction prompts to produce query, document, and other specialized embeddings for retrieval pipelines.

COMPARE

Competitive Models

Get one key to every model

Swap your API key. Keep your code.

Browse available models Talk to sales