Phi-4: Self-Hosting & Deployment Guide
Phi-4 is a 14B open-weight model trained for maths and reasoning at small scale.
What is Phi-4?
Phi-4 is a 14B open-weight model trained for maths and reasoning at small scale, published by Microsoft with open weights at microsoft/phi-4. LLM API does not route this model today, so there is no endpoint or code sample for it here. What this page does give you is the self-hosting picture: published weight sizes, the VRAM each quantisation needs and the commands to serve it yourself. Every figure is read from the public repository and refreshed weekly.
How to run Phi-4 yourself
Open weights (mit), 14.7B parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.
Phi-4 VRAM requirements
| Precision / quantization | Weights | Figure | Source |
|---|---|---|---|
| BF16 (full precision) | 29.3 GB | Estimate | 14.7B params × 2 bytes |
| FP8 | 15.7 GB | File size | RedHatAI/phi-4-FP8-dynamic |
| AWQ 4-bit | 9.1 GB | File size | stelterlab/phi-4-AWQ |
File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.
GPU requirements: deploy Phi-4 on your hardware
| Hardware | BF16 (full precision) | FP8 | AWQ 4-bit |
|---|---|---|---|
| NVIDIA H200141 GB | 1× | 1× | 1× |
| NVIDIA H10080 GB | 1× | 1× | 1× |
| NVIDIA A100 80GB80 GB | 1× | 1× | 1× |
| NVIDIA L40S48 GB | 1× | 1× | 1× |
GPUs needed per variant.
| Hardware | BF16 (full precision) | FP8 | AWQ 4-bit |
|---|---|---|---|
| AWS p5.48xlarge8× H100 · 640 GB | Fits | Fits | Fits |
| AWS p4de.24xlarge8× A100 80GB · 640 GB | Fits | Fits | Fits |
| AWS g6e.12xlarge4× L40S · 192 GB | Fits | Fits | Fits |
| Google Cloud a3-highgpu-8g8× H100 · 640 GB | Fits | Fits | Fits |
| Azure ND H100 v58× H100 · 640 GB | Fits | Fits | Fits |
Whole-instance GPU memory; public instance specs.
Serve it yourself
pip install -U vllm
vllm serve microsoft/phi-4pip install 'sglang[all]'
python -m sglang.launch_server --model-path microsoft/phi-4Commands use the official repositories above; vLLM and SGLang expose an OpenAI-compatible endpoint.
Not on LLM.API yet
We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.
Not available on LLM API yet
We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.
Browse available modelsWhy run Phi-4 on LLM.API?
Unified AI Routing
Reach Phi-4 and sibling models through one OpenAI-compatible endpoint.
Cost Control
Production: Compare provider price points and keep spend visible as you scale Phi-4.
Reliability Layer
Retry and route across configured providers when a single upstream blips.
Observability
Trace prompts, tokens, and errors for Phi-4 alongside the rest of your stack.
Drop-in SDKs
Keep using familiar OpenAI client patterns with base URL https://api.llmapi.ai/v1.
Model Breadth
Swap Phi-4 for chat, media, or embedding alternatives without rewriting auth.
When to Use — When NOT to Use
Use it if...
- You want to run Phi-4 on your own hardware and keep every prompt inside your network
- The weights are published openly, so you can pin one checkpoint and keep it reproducible
- You want to fine-tune or quantise the model rather than accept a hosted configuration
- Your workload rewards deliberate step-by-step reasoning over fast replies
Avoid if...
- LLM API does not route this model today, so there is no endpoint here to call
- You have no GPU capacity: the VRAM figures below are the floor, not a suggestion
- You want zero operations — serving, scaling and upgrades are yours to run
- You need image, audio or video input; this checkpoint is text in, text out
COMMUNITY
What developers say about Phi-4
Summarised from public developer discussions and write-ups. Opinions are the sources’, not LLM.API’s.
- Developers on Hacker News noted initial implementation bugs in Phi-4, such as incorrect end-of-sentence tokens and chat template errors, leading community members to release "Llama-fied" versions and GGUF fixes for better local compatibility.
- Developers on Hugging Face describe Phi-4 as a powerful specialized tool for coding and math with improved alignment, though they report it suffers from poor general world knowledge compared to Llama or Gemma.
- Developers on Hacker News observed that Phi-4's reasoning variants can be highly verbose and noted that its architectural shift away from sliding window attention caused hyperparameter conflicts in some early third-party model files.
SOURCES
Get one key to every model
Swap your API key. Keep your code.