New in llama.cpp: Model Management

New in llama.cpp: Model Management New in llama.cpp: Model Management Team Article Published December 11, 2025 Upvote 144 Xuan-Son Nguyen ngxson ggml-org Victor Mustar victor ggml-org llama.cpp server now ships with router mode , which lets you dynamically load, unload, and switch between multiple models without restarting. Reminder: llama.cpp server is a lightweight, OpenAI-compatible HTTP server for running LLMs locally.

This ModelRelease is relevant to the technology intelligence record because it involves OpenAI, GitHub, Anthropic, DeepSeek. The source article should remain the factual reference for follow-up coverage.

• New in llama.cpp: Model Management Team Article Published December 11, 2025 Upvote 144 Xuan-Son Nguyen ngxson ggml-org Victor Mustar victor ggml-org llama.cpp server now ships with router mode , which lets you dynamically load, unload, and switch between multiple models without restarting.
• Reminder: llama.cpp server is a lightweight, OpenAI-compatible HTTP server for running LLMs locally.
• This feature was a popular request to bring Ollama-style model management to llama.cpp.
• It uses a multi-process architecture where each model runs in its own process, so if one model crashes, others remain unaffected.
• Quick Start Start the server in router mode by not specifying a model : llama-server This auto-discovers models from your llama.cpp cache ( LLAMA_CACHE or ~/.cache/llama.cpp ).
• If you've previously downloaded models via llama-server -hf user/model , they'll be available automatically.