DeepSeek V3.1: Self-Hosting & Deployment Guide
DeepSeek-V3.1 is a large hybrid reasoning model (671B parameters, 37B active) that supports both thinking and non-thinking modes via prompt templates.
What is DeepSeek V3.1?
DeepSeek-V3.1 is a large hybrid reasoning model (671B parameters, 37B active) that supports both thinking and non-thinking modes via prompt templates.
Developer: DeepSeek. Released 21 Aug 2025. Context window 163,840 tokens, up to 32,768 output tokens.
How to run DeepSeek V3.1 yourself
Open weights (mit), 684.5B parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.
DeepSeek V3.1 VRAM requirements
| Precision / quantization | Weights | Figure | Source |
|---|---|---|---|
| BF16 (full precision) | 1369.1 GB | Estimate | 684.5B params × 2 bytes |
| AWQ 4-bit | 398.1 GB | File size | QuantTrio/DeepSeek-V3.1-AWQ |
| MLX 4-bit | 377.6 GB | File size | mlx-community/DeepSeek-V3.1-4bit |
| GGUF Q8_0 | 713.3 GB | File size | unsloth/DeepSeek-V3.1-GGUF |
| GGUF Q4_K_M | 405.4 GB | File size | unsloth/DeepSeek-V3.1-GGUF |
| GGUF Q2_K | 245.7 GB | File size | unsloth/DeepSeek-V3.1-GGUF |
File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.
GPU requirements: deploy DeepSeek V3.1 on your hardware
| Hardware | BF16 (full precision) | AWQ 4-bit | MLX 4-bit | GGUF Q8_0 | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|---|---|---|---|
| NVIDIA H200141 GB | — | 4× | 4× | 6× | 4× | 3× |
| NVIDIA H10080 GB | — | 6× | 6× | — | 6× | 4× |
| NVIDIA A100 80GB80 GB | — | 6× | 6× | — | 6× | 4× |
| NVIDIA L40S48 GB | — | — | — | — | — | 6× |
GPUs needed per variant.
| Hardware | GGUF Q8_0 | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|---|
| RTX 509032 GB | — | — | — |
| RTX 409024 GB | — | — | — |
| RTX 309024 GB | — | — | — |
GGUF via llama.cpp; more cards or CPU offload for larger files.
| Hardware | GGUF Q8_0 | GGUF Q4_K_M | GGUF Q2_K | MLX 4-bit |
|---|---|---|---|---|
| Mac, 512 GBM3 Ultra | — | — | Fits | — |
| Mac, 192 GBM2 Ultra | — | — | — | — |
| Mac, 128 GBM4 Max / M3 Max | — | — | — | — |
| Mac, 64 GBM4 Pro / Max | — | — | — | — |
Assumes ~75% of unified memory is usable by the GPU.
| Hardware | BF16 (full precision) | AWQ 4-bit | MLX 4-bit | GGUF Q8_0 | GGUF Q4_K_M | GGUF Q2_K |
|---|---|---|---|---|---|---|
| AWS p5.48xlarge8× H100 · 640 GB | — | Fits | Fits | — | Fits | Fits |
| AWS p4de.24xlarge8× A100 80GB · 640 GB | — | Fits | Fits | — | Fits | Fits |
| AWS g6e.12xlarge4× L40S · 192 GB | — | — | — | — | — | — |
| Google Cloud a3-highgpu-8g8× H100 · 640 GB | — | Fits | Fits | — | Fits | Fits |
| Azure ND H100 v58× H100 · 640 GB | — | Fits | Fits | — | Fits | Fits |
Whole-instance GPU memory; public instance specs.
Serve it yourself
pip install -U vllm
vllm serve deepseek-ai/DeepSeek-V3.1pip install 'sglang[all]'
python -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.1llama-server -hf unsloth/DeepSeek-V3.1-GGUF:Q4_K_M \
--ctx-size 32768 --n-gpu-layers 99pip install -U mlx-lm
mlx_lm.generate --model mlx-community/DeepSeek-V3.1-4bit --prompt "Hello"Commands use the official repositories above; vLLM and SGLang expose an OpenAI-compatible endpoint.
Not on LLM.API yet
We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.
Providers
Companies that host this model today, with their public list prices. LLM API does not route this model yet.
List price by provider
InputOutputProvider list prices; the LLM.API discount applies on top.
| Provider | Input /M | Output /M | Cache read /M | Context | Precision | Uptime (24h) |
|---|---|---|---|---|---|---|
| DeepInfra | in $0.0002; out $0.0009 per 1M tokens | 164K tokens | tools, streaming, reasoning, JSON, structured | |||
| SiliconFlow | in $0.0003; out $0.001 per 1M tokens | 164K tokens | tools, streaming, reasoning, JSON, structured | |||
| CoreWeave | in $0.0006; out $0.0016 per 1M tokens | 161K tokens | tools, streaming, reasoning, JSON, structured | |||
| Mara | in $0.0006; out $0.0017 per 1M tokens | 131K tokens | streaming, reasoning, JSON, structured | |||
| SambaNova | in $0.0007; out $0.0015 per 1M tokens | 131K tokens | tools, streaming, reasoning |
List prices from third-party hosts in the OpenRouter public catalogue (deepseek/deepseek-chat-v3.1); LLM API does not serve this model and applies no discount. Updated weekly. Last updated 11 Oct 2026.
Source: OpenRouter public catalogue. Last updated 23 Sep 2026.
Not available on LLM API yet
We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.
Browse available modelsDeepSeek V3.1: Self-Hosting & Deployment Guide in production
What the model is good at.
Hybrid thinking and non-thinking modes
One set of weights serves both a direct chat mode and a deliberate reasoning mode, selected by the prompt template rather than by switching model.
Agentic tool use
Post-training focused on multi-step tool calling, which lifted its reported SWE-bench Verified agent score to 66.0 from 45.4 on V3-0324.
128K context handling
Context was extended to 128,000 tokens through an added long-context pre-training phase, enough for large codebases and long documents.
Code generation and repair
Reported 71.6% on the Aider polyglot coding benchmark, the result that moved the V3 family into production coding work.
Open weights under MIT
671B total parameters with 37B active per token, published openly so any provider can host it and you can self-host the same checkpoint.
What you get when you call DeepSeek V3.1: Self-Hosting & Deployment Guide through LLM.API.
-
Intelligent Model Routing
Dynamically route each request to the optimal model across providers based on latency, price, and quality—without changing your integration or redeploying code.
One API, many models -
Cost-Aware Orchestration
Automatically balance premium and budget models with per-call controls, caps, and policies so you can ship fast while keeping AI spend predictable and optimized.
Control spend by design -
Resilient Fallback Flows
Define cascading provider and model fallbacks so timeouts, quota limits, or regional outages transparently fail over—maintaining uptime without custom retry logic.
Fail soft, not hard -
End-to-End Observability
Trace every request across providers with logs, metrics, and structured payloads to debug prompts, compare models, and tune performance from a single dashboard.
See every token -
Task-Level Abstractions
Call high-level tasks like chat, generate, extract, or rank instead of vendor-specific APIs, so you can swap models without rewriting business logic.
Code to tasks, not vendors -
High-Throughput Batch Jobs
Run large-scale inference workloads as managed batches with concurrency, retries, and progress tracking built in—perfect for backfills, evaluations, and data processing.
Crush your backlogs
When to Use — When NOT to Use
Use it if...
- You want near-frontier coding quality at open-weight pricing
- Your workload benefits from switching between fast replies and deeper reasoning
- You run multi-step agents that depend on reliable tool calling
- You need the option to self-host the same checkpoint later
Avoid if...
- You need image, audio or video input — this model is text in, text out
- Your prompts exceed 128,000 tokens
- You want the newest DeepSeek generation: V3.2 and V4 supersede it on maths, context and coding
- You need a hard latency guarantee without benchmarking a specific host first
BENCHMARKS
DeepSeek V3.1 benchmark scores
Intelligence index
Scale: 0-100 index points
Reference price per 1M tokens
Bars compare published reference list prices, not LLM.API pricing.
Higher is better. The median covers every model tracked by Artificial Analysis.
Provider list prices, before the LLM.API discount.
Scores as published by Artificial Analysis and the official model card (source). Reference prices are provider list prices, not LLM.API pricing. Figures with no published value are omitted.
COMMUNITY
What developers say about DeepSeek V3.1
Summarised from publicly published developer write-ups and the model's own documentation. Opinions are the sources’, not LLM.API’s.
- Reviewers treat V3.1 as the release where DeepSeek became a credible agent backbone, thanks to the jump in SWE-bench Verified.
- The hybrid think / non-think toggle is repeatedly called the practical headline feature, because it removes the need to route between two models.
- Cost is the recurring theme: testers report frontier-adjacent coding results at a small fraction of proprietary pricing.
- By 2026 it is described as the stable, well-understood fallback rather than the leading choice, superseded by later DeepSeek releases.
SOURCES
Frequently Asked Questions
What is DeepSeek V3.1?
DeepSeek-V3.1 is a large hybrid reasoning model (671B parameters, 37B active) that supports both thinking and non-thinking modes via prompt templates.
Who makes DeepSeek V3.1?
DeepSeek V3.1 is developed by DeepSeek. It was released on 21 Aug 2025.
What is the context window of DeepSeek V3.1?
163,840 tokens, with up to 32,768 output tokens per response.
How much does DeepSeek V3.1 cost?
The reference list price is $0.25 per 1M input tokens and $0.95 per 1M output tokens. The cheapest host right now is DeepInfra at $0.25 / $0.95 per 1M tokens.
Which providers host DeepSeek V3.1?
DeepInfra, SiliconFlow, Novita, AtlasCloud, CoreWeave, Mara, Google, SambaNova.
What modalities does DeepSeek V3.1 support?
Text input and text output. It supports tool calling, structured outputs, JSON mode, reasoning.
Can I use DeepSeek V3.1 through LLM API?
Not yet. LLM API does not route this model at the moment. Browse the models page for close alternatives you can call today with one API key.
COMPARE
Competitive Models
-
DeepSeek V4 Flash
Up to 30%
DeepSeek V4 Flash is an efficiency-optimized Mixture-of-Experts large language model from DeepSeek, featuring a 1M-token context window and fast inference for high-throughput applications.
-
DeepSeek V4 Flash (free)
Up to 30%
DeepSeek V4 Flash (free) is an open-source, efficiency-optimized Mixture-of-Experts language model from DeepSeek, offering a 1M-token context window with only 13B parameters activated per token out of 284B total. It is designed to deliver fast, cost-effective long-context reasoning, coding, and agentic workflows.
-
DeepSeek V3.2
DeepSeek V3.2 is a large open-source Mixture-of-Experts language model from DeepSeek that emphasizes high reasoning performance and efficient long‑context inference. It is notable for its DeepSeek Sparse Attention and multi-latent attention mechanisms, which significantly cut compute and memory costs for long sequences.
Get one key to every model
Swap your API key. Keep your code.