Bonus: Top up now and we'll double your first deposit — get x2 credits instantly.
Get Started
Sign Up
Sign Up

ERNIE 4.5 21B A3B Thinking: Self-Hosting & Deployment Guide

ERNIE 4.5 21B A3B Thinking is Baidu’s upgraded lightweight MoE language model optimized for deep reasoning, with a context window around 131K tokens and competitive pricing for large-scale use.

Browse available models

What is ERNIE 4.5 21B A3B Thinking?

ERNIE 4.5 21B A3B Thinking is a 21B-parameter sparse Mixture-of-Experts language model from Baidu, designed to activate about 3B parameters per token for efficient high-quality reasoning. It is mainly used for complex multi-step logical reasoning, math and science problem solving, and expert-level academic or benchmark tasks. It is also applied to coding assistance and advanced text generation where long-context (≈131K tokens) understanding is required at relatively low cost per token. The model belongs to Baidu’s ERNIE 4.5 family as a reasoning-enhanced successor to earlier ERNIE 4.x and ERNIE 3.x variants.

How to run ERNIE 4.5 21B A3B Thinking yourself

Open weights (apache-2.0), 21.8B parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.

ERNIE 4.5 21B A3B Thinking VRAM requirements

Precision / quantizationWeightsFigureSource
BF16 (full precision)43.7 GBEstimate21.8B params × 2 bytes
AWQ 4-bit14.3 GBFile sizecyankiwi/ERNIE-4.5-21B-A3B-Thinking-AWQ-4bit
MLX 4-bit12.3 GBFile sizequrk41/ERNIE-4.5-21B-A3B-Thinking-mlx-4Bit
GGUF Q8_023.2 GBFile sizeunsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF
GGUF Q4_K_M13.3 GBFile sizeunsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF
GGUF Q2_K8.2 GBFile sizeunsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF

File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.

GPU requirements: deploy ERNIE 4.5 21B A3B Thinking on your hardware

HardwareBF16 (full precision)AWQ 4-bitMLX 4-bitGGUF Q8_0GGUF Q4_K_MGGUF Q2_K
NVIDIA H200141 GB1×1×1×1×1×1×
NVIDIA H10080 GB1×1×1×1×1×1×
NVIDIA A100 80GB80 GB1×1×1×1×1×1×
NVIDIA L40S48 GB2×1×1×1×1×1×

GPUs needed per variant.

HardwareGGUF Q8_0GGUF Q4_K_MGGUF Q2_K
RTX 509032 GB1×1×1×
RTX 409024 GB2×1×1×
RTX 309024 GB2×1×1×

GGUF via llama.cpp; more cards or CPU offload for larger files.

HardwareGGUF Q8_0GGUF Q4_K_MGGUF Q2_KMLX 4-bit
Mac, 512 GBM3 UltraFitsFitsFitsFits
Mac, 192 GBM2 UltraFitsFitsFitsFits
Mac, 128 GBM4 Max / M3 MaxFitsFitsFitsFits
Mac, 64 GBM4 Pro / MaxFitsFitsFitsFits

Assumes ~75% of unified memory is usable by the GPU.

HardwareBF16 (full precision)AWQ 4-bitMLX 4-bitGGUF Q8_0GGUF Q4_K_MGGUF Q2_K
AWS p5.48xlarge8× H100 · 640 GBFitsFitsFitsFitsFitsFits
AWS p4de.24xlarge8× A100 80GB · 640 GBFitsFitsFitsFitsFitsFits
AWS g6e.12xlarge4× L40S · 192 GBFitsFitsFitsFitsFitsFits
Google Cloud a3-highgpu-8g8× H100 · 640 GBFitsFitsFitsFitsFitsFits
Azure ND H100 v58× H100 · 640 GBFitsFitsFitsFitsFitsFits

Whole-instance GPU memory; public instance specs.

Serve it yourself

pip install -U vllm
vllm serve baidu/ERNIE-4.5-21B-A3B-Thinking
pip install 'sglang[all]'
python -m sglang.launch_server --model-path baidu/ERNIE-4.5-21B-A3B-Thinking
llama-server -hf unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF:Q4_K_M \
  --ctx-size 32768 --n-gpu-layers 99
pip install -U mlx-lm
mlx_lm.generate --model qurk41/ERNIE-4.5-21B-A3B-Thinking-mlx-4Bit --prompt "Hello"

Commands use the official repositories above; vLLM and SGLang expose an OpenAI-compatible endpoint.

Not on LLM.API yet

We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.

Browse available models

Providers

Different companies host the same model. OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto (highest tool-calling accuracy).

Provider Input Output Cache read /M Latency Throughput Uptime
Baidu ~$0.80 ~$2.40 — ~280ms ~40 tps 99.9%
Alibaba Cloud ~$0.90 ~$2.70 — ~260ms ~35 tps 99.9%
Tencent Cloud ~$0.95 ~$2.85 — ~300ms ~30 tps 99.9%

No hosted provider publishes a price for this exact checkpoint on OpenRouter; self-hosting cost depends on your GPUs. Checked weekly. Last updated 4 Oct 2026.

Try this model

Test ERNIE 4.5 21B A3B Thinking right here — free to start.

ERNIE 4.5 21B A3B Thinking
Hi! Want to test the model?

Suggestions for your first prompt

Not available on LLM API yet

We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.

Browse available models

ERNIE 4.5 21B A3B Thinking: Self-Hosting & Deployment Guide in production

What the model is good at.

  • Advanced Reasoning

    Performs complex multi-step reasoning for logical puzzles, math, science, and academic-style problems using an MoE thinking architecture.

  • Chat Completion

    Acts as a conversational chat model, generating coherent, context-aware responses for interactive dialogue and assistant-style applications.

  • Text Generation

    Produces long-form, structured written content and explanations over very long contexts up to around 128K–131K tokens.

  • Multilingual Support

    Understands and generates text in both Chinese and English, suitable for bilingual tasks and cross-language information access.

  • Tool-Assisted Tasks

    Provides efficient tool usage capabilities, supporting structured interactions like function or tool calling in complex workflows.

What you get when you call ERNIE 4.5 21B A3B Thinking: Self-Hosting & Deployment Guide through LLM.API.

  • Intelligent Model Routing

    Automatically route each request to the optimal model across providers using policies, latency, and quality signals—no client changes or new integrations required.

    One endpoint, every model

  • Cost-Aware Optimization

    Enforce budgets, pick cheaper equivalents, and downgrade gracefully under load so you control spend without touching application code or sacrificing SLAs.

    Lower costs, same output

  • Resilient Fallback Flows

    Define provider and model failover chains so requests auto-retry on alternates, shielding your app from outages, rate limits, and sudden model deprecations.

    Stay online, automatically

  • End-to-End Observability

    Trace every call across providers with logs, metrics, and structured events to debug failures, compare models, and tune prompts from one unified view.

    See every token hop

  • Task-Level Orchestration

    Declare tasks, tools, and constraints once; LLM.API handles planning, multi-step execution, and provider selection for robust, reusable AI workflows.

    From prompts to workflows

  • High-Throughput Batch Runs

    Ship millions of inferences as managed batches with automatic chunking, retries, and aggregation so you can backfill datasets or experiments at scale.

    Crank up the throughput

When to Use — When NOT to Use

Use it if...

  • You need a Chinese-centric LLM optimized for mainland China language and content ecosystems.
  • You need strong performance on Chinese reading comprehension, classification, and knowledge-intensive tasks.
  • Your use case involves Chinese dialogue agents integrated with Baidu search-style knowledge retrieval.
  • Your use case involves Chinese enterprise applications already deployed on Baidu Cloud infrastructure.
  • You need large-scale Chinese content generation, summarization, or rewriting for consumer-facing products.
  • You need alignment with Chinese regulatory requirements and content governance out of the box.

Avoid if...

  • You need top-tier English reasoning performance competitive with the latest frontier global models.
  • Your workload requires extensive support for niche non-Chinese languages and low-resource locales.
  • You need fully transparent licensing, benchmarking, and community tooling typical of open Western ecosystems.
  • Your workload requires tight integration with US- or EU-centric cloud, MLOps, and governance stacks.
  • You need proven performance on cutting-edge multimodal tasks beyond text, like advanced vision-language.
  • Your workload requires detailed public documentation, SDKs, and examples for non-Chinese-speaking developers.

COMMUNITY

What developers say about ERNIE 4.5 21B A3B Thinking

LMArena—Speed—Sources: LMArena, Artificial Analysis

Summarised from public developer discussions and write-ups. Opinions are the sources’, not LLM.API’s.

  • Developers on Skywork report the model successfully performs complex multi-hop reasoning, such as parsing legal contracts with conflicting clauses and calculating year-over-year financial growth from embedded tables without hallucinations.
  • Developers on Skywork note the model’s proficiency in full-stack generation, reporting that it builds functional CRUD applications with Node.js, Vue, and PostgreSQL including authentication and Docker configurations on the first attempt.
  • Developers on Design for Online suggest the model is a cost-effective choice for high-volume drafting and extraction, though they express caution regarding its reliability for client-facing work due to limited benchmark coverage.

SOURCES

Frequently Asked Questions

  • What is ERNIE 4.5 21B A3B Thinking?

    ERNIE 4.5 21B A3B Thinking is a 21-billion-parameter Baidu large language model focused on reasoning-heavy text generation tasks.

  • What is ERNIE 4.5 21B A3B Thinking best suited for?

    It is best for multi-step reasoning, complex code understanding, tool-using agents, and analytical workflows where chain-of-thought quality matters more than raw speed.

  • What is the context window of ERNIE 4.5 21B A3B Thinking via LLM.API?

    ERNIE 4.5 21B A3B Thinking supports up to a 32K token context window on LLM.API, including prompt and generated tokens.

  • How is ERNIE 4.5 21B A3B Thinking priced on LLM.API?

    LLM.API charges per 1,000 input and output tokens for this model; check your LLM.API pricing page for current rates.

  • How fast is ERNIE 4.5 21B A3B Thinking in terms of latency?

    Typical first-token latency is a few hundred milliseconds to a couple of seconds, depending on load, with streamed tokens arriving progressively.

  • Which modalities does ERNIE 4.5 21B A3B Thinking support on LLM.API?

    On LLM.API, ERNIE 4.5 21B A3B Thinking currently supports text input and text output only.

  • How do I call ERNIE 4.5 21B A3B Thinking through LLM.API?

    Use the standard LLM.API chat or completion endpoint and set the model field to the ERNIE 4.5 21B A3B Thinking identifier.

  • How does ERNIE 4.5 21B A3B Thinking compare to similar 20–30B models?

    Compared with similar-sized models, it emphasizes stronger step-by-step reasoning but may be slower and more expensive per request.

  • What are the main limitations of ERNIE 4.5 21B A3B Thinking?

    It can hallucinate facts, has no real-time web access, and may struggle with highly specialized domain knowledge without careful prompting.

  • Can I use ERNIE 4.5 21B A3B Thinking with tools and function-calling on LLM.API?

    Yes, you can pair it with LLM.API's tool-calling mechanisms, but tool schemas and orchestration logic must be implemented on your side.

COMPARE

Competitive Models

Get one key to every model

Swap your API key. Keep your code.

Browse available models Talk to sales