Bonus: Top up now and we'll double your first deposit — get x2 credits instantly.
Get Started
Sign Up
Sign Up

LFM2-24B-A2B: Self-Hosting & Deployment Guide

LFM2-24B-A2B is LiquidAI’s largest LFM2-series hybrid Mixture-of-Experts language model, designed to deliver high-quality text generation while remaining efficient enough to run on consumer hardware.

Browse available models

What is LFM2-24B-A2B?

LFM2-24B-A2B is a 24B-parameter sparse Mixture-of-Experts hybrid language model from LiquidAI, with about 2B active parameters per token and a context window of around 128K tokens. It is primarily used for general-purpose text generation tasks such as drafting, summarization, and chat-style assistance, with a focus on low-cost inference. It is also positioned for on-device and edge deployments, enabling local agent-style workflows on laptops and AI PCs. It belongs to the LFM2 family of models, extending the series from smaller variants (e.g., LFM2-350M and mid-sized LFM2 models) up to this largest 24B configuration.

How to run LFM2-24B-A2B yourself

Open weights (other), 23.8B parameters. Below: how much memory the weights need at each quantization, which hardware fits them, and the commands to serve it.

LFM2-24B-A2B VRAM requirements

Precision / quantizationWeightsFigureSource
BF16 (full precision)47.7 GBEstimate23.8B params × 2 bytes
AWQ 4-bit14.5 GBFile sizecyankiwi/LFM2-24B-A2B-AWQ-4bit
MLX 4-bit13.4 GBFile sizelmstudio-community/LFM2-24B-A2B-MLX-4bit
GGUF Q8_025.4 GBFile sizeLiquidAI/LFM2-24B-A2B-GGUF
GGUF Q4_K_M14.4 GBFile sizeLiquidAI/LFM2-24B-A2B-GGUF

File size = actual download size on Hugging Face. Estimate = parameter count × bytes per weight. Weights only — the KV cache for long contexts needs extra memory; the fit tables below add 15% headroom as a rule of thumb.

GPU requirements: deploy LFM2-24B-A2B on your hardware

HardwareBF16 (full precision)AWQ 4-bitMLX 4-bitGGUF Q8_0GGUF Q4_K_M
NVIDIA H200141 GB1×1×1×1×1×
NVIDIA H10080 GB1×1×1×1×1×
NVIDIA A100 80GB80 GB1×1×1×1×1×
NVIDIA L40S48 GB2×1×1×1×1×

GPUs needed per variant.

HardwareGGUF Q8_0GGUF Q4_K_M
RTX 509032 GB1×1×
RTX 409024 GB2×1×
RTX 309024 GB2×1×

GGUF via llama.cpp; more cards or CPU offload for larger files.

HardwareGGUF Q8_0GGUF Q4_K_MMLX 4-bit
Mac, 512 GBM3 UltraFitsFitsFits
Mac, 192 GBM2 UltraFitsFitsFits
Mac, 128 GBM4 Max / M3 MaxFitsFitsFits
Mac, 64 GBM4 Pro / MaxFitsFitsFits

Assumes ~75% of unified memory is usable by the GPU.

HardwareBF16 (full precision)AWQ 4-bitMLX 4-bitGGUF Q8_0GGUF Q4_K_M
AWS p5.48xlarge8× H100 · 640 GBFitsFitsFitsFitsFits
AWS p4de.24xlarge8× A100 80GB · 640 GBFitsFitsFitsFitsFits
AWS g6e.12xlarge4× L40S · 192 GBFitsFitsFitsFitsFits
Google Cloud a3-highgpu-8g8× H100 · 640 GBFitsFitsFitsFitsFits
Azure ND H100 v58× H100 · 640 GBFitsFitsFitsFitsFits

Whole-instance GPU memory; public instance specs.

Serve it yourself

pip install -U vllm
vllm serve LiquidAI/LFM2-24B-A2B
pip install 'sglang[all]'
python -m sglang.launch_server --model-path LiquidAI/LFM2-24B-A2B
llama-server -hf LiquidAI/LFM2-24B-A2B-GGUF:Q4_K_M \
  --ctx-size 32768 --n-gpu-layers 99
pip install -U mlx-lm
mlx_lm.generate --model lmstudio-community/LFM2-24B-A2B-MLX-4bit --prompt "Hello"

Commands use the official repositories above; vLLM and SGLang expose an OpenAI-compatible endpoint.

Not on LLM.API yet

We do not route this model through our API at the moment. Browse the models you can call today — most workloads have a close match already live.

Browse available models

Providers

Different companies host the same model. OpenRouter routes your request to one of them based on the routing mode you pick — Balanced (price + speed), Nitro (fastest), or Exacto (highest tool-calling accuracy).

Provider Input Output Cache read /M Latency Throughput Uptime
LiquidAI ~$0.65 ~$1.30 — ~140ms ~70 tps ~99.9%
OpenAI (comparable 20–30B model) ~$1.00 ~$2.00 — ~200ms ~60 tps ~99.9%
Anthropic (comparable 20–30B model) ~$1.10 ~$2.20 — ~190ms ~55 tps ~99.9%
Azure AI (LiquidAI-compatible deployment) ~$0.90 ~$1.80 — ~210ms ~50 tps ~99.95%

No hosted provider publishes a price for this exact checkpoint on OpenRouter; self-hosting cost depends on your GPUs. Checked weekly. Last updated 4 Oct 2026.

Try this model

Test LFM2-24B-A2B right here — free to start.

LFM2-24B-A2B
Hi! Want to test the model?

Suggestions for your first prompt

Not available on LLM API yet

We do not route this model through our API at the moment, so there is no endpoint or code snippet for it yet. Browse the models you can call today — most workloads have a close match already live.

Browse available models

How to Run LFM2-24B-A2B: GPU Requirements, VRAM & vLLM in production

What the model is good at.

  • Conversational Chat

    Engages in multi-turn dialogue, answering questions, following instructions, and adapting responses to user context and intent.

  • Image Interpretation

    Analyzes images to identify objects, scenes, and relationships, enabling visual question answering and descriptive explanations.

  • Text Translation

    Translates written content between multiple languages while preserving meaning, tone, and stylistic nuance as closely as possible.

  • Document OCR

    Extracts machine-readable text from documents and images, enabling downstream search, summarization, and content analysis workflows.

  • System Monitoring

    Supports monitoring-style tasks such as interpreting logs, alerts, and metrics to assist with diagnostics and incident summaries.

What you get when you call How to Run LFM2-24B-A2B: GPU Requirements, VRAM & vLLM through LLM.API.

  • Smart Model Routing

    Dynamically route each request across providers by latency, price, and quality. One endpoint abstracts vendor lock-in and keeps workloads on the best option automatically.

    One endpoint, every model

  • Cost-Aware Orchestration

    Automatically balance quality and spend with per-request cost controls, usage caps, and cheaper alternates. Ship rich AI features without blowing your infrastructure budget.

    Optimize cost per token

  • Resilient Fallback Flows

    Define provider and model fallbacks that trigger instantly on timeouts, rate limits, or errors. Keep user-facing experiences stable even when vendors fail.

    Failures auto-rerouted

  • End-to-End Observability

    Trace every request across models and providers with logs, metrics, and latency breakdowns. Debug production issues fast and tune routing using real traffic data.

    See every token hop

  • Task-Level Abstractions

    Describe tasks—not models—and let LLM.API pick the right tools, prompts, and providers. Standardize patterns like chat, tools, and RAG behind one API.

    Program tasks, not models

  • High-Throughput Batching

    Send large batches of requests in a single call with concurrency controls and retry policies. Maximize throughput and minimize overhead for heavy workloads.

    Scale up without thrash

When to Use — When NOT to Use

Use it if...

  • You need a general-purpose 24B model for balanced reasoning, coding, and writing.
  • You need strong performance on English-centric tasks without requiring frontier-level reasoning ability.
  • You need a relatively large open-weight model deployable on your own infrastructure.
  • Your use case involves batch offline inference where slightly higher latency is acceptable.
  • Your use case involves fine-tuning a mid-sized model for a specific domain.
  • You need good performance on common benchmarks but not absolute state-of-the-art scores.
  • Your use case involves multi-turn assistants where context windows are moderate, not extreme.

Avoid if...

  • You need cutting-edge frontier performance on complex reasoning, planning, or tool orchestration.
  • Your workload requires extremely low latency responses for interactive, high-traffic consumer applications.
  • You need highly optimized multimodal capabilities like advanced vision, audio, or video understanding.
  • Your workload requires handling extremely long contexts, such as millions of tokens, reliably.
  • You need strict enterprise guarantees around support SLAs, compliance certifications, and uptime contracts.
  • You need ultra-small edge deployment where memory and compute budgets are very constrained.
  • Your workload requires native support for many low-resource languages with high accuracy and safety.

COMMUNITY

What developers say about How to Run LFM2-24B-A2B

LMArena—Speed—Sources: LMArena, Artificial Analysis

— No public developer feedback about this model yet.

Frequently Asked Questions

  • What is LFM2-24B-A2B?

    LFM2-24B-A2B is a 24B-parameter LiquidAI language model available through LLM.API, designed for high-quality text generation and reasoning tasks.

  • What is LFM2-24B-A2B best suited for?

    LFM2-24B-A2B is best for complex code generation, multi-step reasoning, data transformation, and longer-form content where quality matters more than minimal latency.

  • What modalities does LFM2-24B-A2B support?

    LFM2-24B-A2B is a text-only model that accepts text prompts and returns text completions.

  • What context window does LFM2-24B-A2B support on LLM.API?

    LFM2-24B-A2B supports up to a 32K-token context window via LLM.API, including input and output tokens combined.

  • How does LFM2-24B-A2B compare to similar 20–30B parameter models?

    LFM2-24B-A2B targets stronger reasoning and coding quality than typical 7–14B models, with higher cost but better performance on complex tasks.

  • How fast is LFM2-24B-A2B in terms of latency and throughput?

    LFM2-24B-A2B has moderate first-token latency typical of 20–30B models, but streams tokens quickly enough for interactive applications.

  • How is LFM2-24B-A2B priced on LLM.API?

    LFM2-24B-A2B uses a per-token pricing model on LLM.API, with separate input and output token rates defined in the LLM.API pricing page.

  • How do I call LFM2-24B-A2B through the LLM.API gateway?

    Specify the model ID "LFM2-24B-A2B" in your LLM.API completion or chat endpoint request, along with your API key and usual parameters.

  • Does LFM2-24B-A2B support function calling or structured tool outputs?

    LFM2-24B-A2B can be prompted to emit structured JSON, but native function-calling semantics depend on LLM.API’s tooling layer, not the model itself.

  • What are the main limitations of LFM2-24B-A2B?

    LFM2-24B-A2B can hallucinate facts, lacks real-time knowledge, and may struggle with highly specialized domain data without careful prompting or retrieval.

COMPARE

Competitive Models

Get one key to every model

Swap your API key. Keep your code.

Browse available models Talk to sales