Bonus: Top up now and we'll double your first deposit — get x2 credits instantly.
Get Started
Sign Up
Sign Up

bge-reranker-large: Self-Hosting & Deployment Guide

bge-reranker-large is an open-weight embedding model that turns text into vectors for search and retrieval.

Browse available models

What is bge-reranker-large?

bge-reranker-large is an open-weight embedding model that turns text into vectors for search and retrieval, published by BAAI with open weights at BAAI/bge-reranker-large. LLM API does not route this model today, so there is no endpoint or code sample for it here. What this page does give you is the self-hosting picture: published weight sizes, the VRAM each quantisation needs and the commands to serve it yourself. Every figure is read from the public repository and refreshed weekly. At import it had 2,802,724 downloads on Hugging Face in the previous 30 days, under the mit licence.

How to run bge-reranker-large yourself

Open weights (mit), 560M parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.

bge-reranker-large VRAM requirements

Precision / quantizationWeightsFigureSource
BF16 (full precision)1.1 GBEstimate560M params × 2 bytes
GGUF Q8_00.6 GBFile sizeqmaru/bge-reranker-large-gguf
GGUF Q4_K_M0.41 GBFile sizeqmaru/bge-reranker-large-gguf

File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.

GPU requirements: deploy bge-reranker-large on your hardware

HardwareBF16 (full precision)GGUF Q8_0GGUF Q4_K_M
NVIDIA H200141 GB1×1×1×
NVIDIA H10080 GB1×1×1×
NVIDIA A100 80GB80 GB1×1×1×
NVIDIA L40S48 GB1×1×1×

GPUs needed per variant.

HardwareGGUF Q8_0GGUF Q4_K_M
RTX 509032 GB1×1×
RTX 409024 GB1×1×
RTX 309024 GB1×1×

GGUF via llama.cpp; more cards or CPU offload for larger files.

HardwareGGUF Q8_0GGUF Q4_K_M
Mac, 512 GBM3 UltraFitsFits
Mac, 192 GBM2 UltraFitsFits
Mac, 128 GBM4 Max / M3 MaxFitsFits
Mac, 64 GBM4 Pro / MaxFitsFits

Assumes ~75% of unified memory is usable by the GPU.

HardwareBF16 (full precision)GGUF Q8_0GGUF Q4_K_M
AWS p5.48xlarge8× H100 · 640 GBFitsFitsFits
AWS p4de.24xlarge8× A100 80GB · 640 GBFitsFitsFits
AWS g6e.12xlarge4× L40S · 192 GBFitsFitsFits
Google Cloud a3-highgpu-8g8× H100 · 640 GBFitsFitsFits
Azure ND H100 v58× H100 · 640 GBFitsFitsFits

Whole-instance GPU memory; public instance specs.

Not on LLM.API yet

We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.

Browse available models

Not available on LLM API yet

We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.

Browse available models

When to Use — When NOT to Use

Use it if...

  • You need vectors for semantic search, clustering or RAG retrieval (bge-reranker-large)
  • You want to keep documents inside your own network while indexing them
  • The weights are published openly, so you can pin one checkpoint and keep vectors reproducible
  • You are comparing self-hosting cost against a managed embedding endpoint

Avoid if...

  • LLM API does not route this model today, so there is no endpoint here to call
  • You want zero operations — serving, scaling and upgrades are yours to run
  • You need text generation — this model returns vectors or scores, not answers
  • You already have an index built with a different model: vectors are not interchangeable

Get one key to every model

Swap your API key. Keep your code.

Browse available models Talk to sales