GPUStack is an open-source control plane for AI model serving and GPU instance provisioning. It manages GPU clusters across on-premises, Kubernetes and cloud environments, schedules accelerator capacity, and configures pluggable inference engines such as vLLM, SGLang and TensorRT-LLM rather than replacing those engines. The platform exposes compatible model APIs and adds authentication, access control, load balancing, monitoring, token and request metering, automated recovery, performance-oriented engine settings, and on-demand SSH-accessible GPU instances. Its current documentation lists support paths for NVIDIA, AMD, Ascend, Hygon, MThreads, Iluvatar, MetaX, Cambricon and T-Head accelerators, while worker nodes remain Linux-only. Maintainer claims about Day-0 model support and performance gains depend on the selected engine, model and hardware, so operators should validate compatibility and throughput against their own fleet. GPUStack is best suited to platform teams running multiple models, clusters or tenants; a single local model or a team seeking a fully managed serverless endpoint will usually need less operational machinery.
GPUStack
Open-source GPU control plane for scalable AI model serving
Open-source GPU cluster manager that configures vLLM, SGLang, TensorRT-LLM or custom engines, serves models through compatible APIs, and provisions SSH-accessible GPU instances across on-premises, Kubernetes and cloud environments.
Pricing
Free and open source under Apache-2.0; operators pay for their own GPU, compute, storage and operations. GPUStack Enterprise Edition is contact/demo-led and has no public price table verified as of 2026-08-12.
Platforms
Docker-deployable control plane with Linux GPU workers, multi-cluster scheduling, pluggable vLLM/SGLang/TensorRT-LLM engines, compatible model APIs, monitoring, metering and SSH-accessible GPU instances. Current verified release: v2.2.3.
Categories
Tags
Use Cases
Alternatives
AIBrix
Cloud-native control plane for scalable GenAI inference
Open-source Kubernetes-native building blocks for deploying, routing and scaling GenAI inference, including an LLM gateway, autoscaling, LoRA management and KV-cache offloading.
KubeAI
Kubernetes operator for serving AI inference workloads
KubeAI is an Apache-2.0 Kubernetes operator for deploying and scaling AI inference workloads, including LLMs, embeddings, reranking, and speech-to-text. It gives platform teams OpenAI-compatible endpoints, model proxy/controller primitives, model caching, scale-from-zero behavior, and cluster-native resource management for self-hosted inference on Kubernetes.
Dstack
Open-source control plane for AI workloads across multi-cloud GPU infrastructure
dstack is an open-source platform that orchestrates AI training and inference workloads across heterogeneous GPU infrastructure spanning multiple clouds, Kubernetes clusters, and bare-metal servers. It abstracts away cloud-specific APIs so teams define GPU requirements declaratively and dstack automatically provisions the cheapest available resources from AWS, GCP, Azure, Lambda, or on-premises hardware.
SkyPilot
Run AI workloads on any cloud with automatic cost optimization
SkyPilot is an open-source framework for running LLMs, AI, and batch jobs on any cloud with automatic cost optimization. It supports AWS, GCP, Azure, Lambda Cloud, and more, automatically selecting the cheapest available GPUs and managing spot instance preemption. Features include multi-cloud job scheduling, managed spot jobs with automatic recovery, and cluster autoscaling with 6,000+ GitHub stars.
Related Tools
Mooncake
Disaggregated KV cache storage and transfer for LLM serving
Open-source infrastructure for disaggregated LLM serving that pools KV caches across prefill and decode workers, with high-performance transfer, distributed storage and integrations for vLLM and SGLang.
LMCache
Reusable KV cache infrastructure for scalable LLM inference
Open-source KV cache management layer that persists, offloads and reuses model key-value caches across requests and serving engines to reduce repeated prefill work and improve inference throughput.
LitServe
Build custom AI inference servers in pure Python
Open-source, FastAPI-based serving engine from Lightning AI for building custom inference APIs — models, agents, RAG, and pipelines — with built-in batching, streaming, and multi-GPU autoscaling.
LLM Compressor
Quantize and compress LLMs for optimized vLLM deployment
Open-source, Hugging Face-compatible library for quantizing and compressing LLMs (W8A8, W4A16, NVFP4/MXFP4, KV-cache and attention quantization, expert pruning) into vLLM-ready `compressed-tensors` checkpoints.
Open Notebook
Private, self-hosted research notebooks with flexible AI models, source chat, and podcasts
Open Notebook is an MIT-licensed, self-hosted alternative to NotebookLM for collecting sources, chatting over research, generating reusable transformations, and producing multi-speaker podcasts. Its Docker stack keeps notebook data under the user's control while supporting 18-plus model providers, including local Ollama and LM Studio workflows.
Text Embeddings Inference
Hugging Face's open-source inference server for embeddings, rerankers, and classifiers
Text Embeddings Inference is Hugging Face's Apache-2.0 server for high-throughput embedding, reranking, and sequence-classification models. TEI packages token-based dynamic batching, optimized Transformers kernels, Safetensors loading, OpenAI-compatible embedding endpoints, Prometheus metrics, and configurable OpenTelemetry tracing in deployable CPU and GPU images.