DeepSeek-OCR: Self-Hosting & Deployment Guide

DeepSeek-OCR is an open-weight vision-language model that reads images alongside text.

What is DeepSeek-OCR?

DeepSeek-OCR is an open-weight vision-language model that reads images alongside text, published by DeepSeek with open weights at deepseek-ai/DeepSeek-OCR. LLM API does not route this model today, so there is no endpoint or code sample for it here. What this page does give you is the self-hosting picture: published weight sizes, the VRAM each quantisation needs and the commands to serve it yourself. Every figure is read from the public repository and refreshed weekly. At import it had 2,247,449 downloads on Hugging Face in the previous 30 days, under the mit licence.

How to run DeepSeek-OCR yourself

Open weights (mit), 3.3B parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.

DeepSeek-OCR VRAM requirements

Precision / quantization	Weights	Figure	Source
BF16 (full precision)	6.7 GB	Estimate	3.3B params × 2 bytes
FP8	3.7 GB	File size	richarddavison/DeepSeek-OCR-2-FP8
MLX 4-bit	2.5 GB	File size	mlx-community/DeepSeek-OCR-4bit
GGUF Q8_0	3.6 GB	File size	sabafallah/DeepSeek-OCR-2-GGUF
GGUF Q4_K_M	2 GB	File size	sabafallah/DeepSeek-OCR-2-GGUF

File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.

GPU requirements: deploy DeepSeek-OCR on your hardware

Datacenter GPUsConsumer GPUsApple SiliconCloud instances

Hardware	BF16 (full precision)	FP8	MLX 4-bit	GGUF Q8_0	GGUF Q4_K_M
NVIDIA H200141 GB	1×	1×	1×	1×	1×
NVIDIA H10080 GB	1×	1×	1×	1×	1×
NVIDIA A100 80GB80 GB	1×	1×	1×	1×	1×
NVIDIA L40S48 GB	1×	1×	1×	1×	1×

GPUs needed per variant.

Hardware	GGUF Q8_0	GGUF Q4_K_M
RTX 509032 GB	1×	1×
RTX 409024 GB	1×	1×
RTX 309024 GB	1×	1×

GGUF via llama.cpp; more cards or CPU offload for larger files.

Hardware	GGUF Q8_0	GGUF Q4_K_M	MLX 4-bit
Mac, 512 GBM3 Ultra	Fits	Fits	Fits
Mac, 192 GBM2 Ultra	Fits	Fits	Fits
Mac, 128 GBM4 Max / M3 Max	Fits	Fits	Fits
Mac, 64 GBM4 Pro / Max	Fits	Fits	Fits

Assumes ~75% of unified memory is usable by the GPU.

Hardware	BF16 (full precision)	FP8	MLX 4-bit	GGUF Q8_0	GGUF Q4_K_M
AWS p5.48xlarge8× H100 · 640 GB	Fits	Fits	Fits	Fits	Fits
AWS p4de.24xlarge8× A100 80GB · 640 GB	Fits	Fits	Fits	Fits	Fits
AWS g6e.12xlarge4× L40S · 192 GB	Fits	Fits	Fits	Fits	Fits
Google Cloud a3-highgpu-8g8× H100 · 640 GB	Fits	Fits	Fits	Fits	Fits
Azure ND H100 v58× H100 · 640 GB	Fits	Fits	Fits	Fits	Fits

Whole-instance GPU memory; public instance specs.

Serve it yourself

vLLMSGLangllama.cpp (GGUF)MLX (Mac)

pip install -U vllm
vllm serve deepseek-ai/DeepSeek-OCR

pip install 'sglang[all]'
python -m sglang.launch_server --model-path deepseek-ai/DeepSeek-OCR

llama-server -hf sabafallah/DeepSeek-OCR-2-GGUF:Q4_K_M \
  --ctx-size 32768 --n-gpu-layers 99

pip install -U mlx-lm
mlx_lm.generate --model mlx-community/DeepSeek-OCR-4bit --prompt "Hello"

Commands use the official repositories above; vLLM and SGLang expose an OpenAI-compatible endpoint.

Not on LLM.API yet

We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.

Browse available models

Not available on LLM API yet

We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.

Browse available models

When to Use — When NOT to Use

Use it if...

You want to run DeepSeek-OCR on your own hardware and keep every prompt inside your network
The weights are published openly, so you can pin one checkpoint and keep it reproducible
You want to fine-tune or quantise the model rather than accept a hosted configuration
You process documents, screenshots or charts where the image must stay in-house

Avoid if...

LLM API does not route this model today, so there is no endpoint here to call
You have no GPU capacity: the VRAM figures below are the floor, not a suggestion
You want zero operations — serving, scaling and upgrades are yours to run
You need audio or video input; this checkpoint documents text and images only

Get one key to every model

Swap your API key. Keep your code.

Browse available models Talk to sales