Best tools for Self-Hosted Deployment

Open-source secrets management platform

Infisical is an open-source secrets management platform with 16K+ GitHub stars for syncing environment variables and secrets across teams, CI/CD pipelines, and infrastructure. Features end-to-end encryption, automatic secret rotation, dynamic secrets, access controls with audit logs, and native integrations with AWS, GCP, Azure, Kubernetes, Docker, GitHub Actions, and Vercel. Replaces scattered .env files with a centralized, encrypted secrets store accessible via dashboard, CLI, SDK, or API.

Cloud-native POSIX filesystem on object storage

JuiceFS is a high-performance distributed POSIX filesystem built on object storage like S3 and metadata engines like Redis or MySQL. It enables seamless data sharing across thousands of clients with low latency and elastic throughput. JuiceFS ships with a Kubernetes CSI driver, Hadoop SDK compatibility, and FUSE mount support for AI training, big data analytics, and shared storage workloads. Apache 2.0 licensed with 13K+ GitHub stars.

Kubernetes-native model inference platform

KServe is an open-source Kubernetes-native platform for deploying and managing ML model inference at scale. It provides standardized inference protocols, autoscaling including scale-to-zero, canary rollouts, A/B testing, and multi-model serving. KServe supports all major ML frameworks including TensorFlow, PyTorch, scikit-learn, XGBoost, and LLM runtimes like vLLM and Triton through pluggable serving runtimes.

Heterogeneous CPU-GPU inference and SFT for large MoE models

Open-source framework for running and fine-tuning large Mixture-of-Experts models with heterogeneous CPU-GPU execution, optimized kernels, limited VRAM and SGLang or LLaMA-Factory integrations.

Deploy web apps anywhere

Zero-downtime deployment tool created by 37signals (Basecamp/HEY) that deploys Docker containers to any VPS or bare metal server over SSH without requiring Kubernetes. Manages rolling deploys, health checks, SSL via Let's Encrypt, container registry, and load balancing with Traefik. Simple YAML-based config. Ideal for small teams deploying Rails, Node.js, or any Dockerized app. Ships as a Ruby gem. 12K+ GitHub stars and a growing alternative to complex container orchestration.

Open-source AI second brain with deep research and RAG

Khoj is an open-source personal AI app that serves as a self-hostable second brain. It connects to your documents — PDFs, Markdown, Notion, Word — and uses RAG to answer questions grounded in your knowledge base. Supports any local or cloud LLM including Llama, Claude, GPT, and Gemini. Features custom agents, scheduled automations, deep research mode, semantic search, and Obsidian, Emacs, and WhatsApp integrations. Over 33,000 GitHub stars, YC-backed.

Coding data, MCP integrations, and sandboxed tool-use environments

Klavis AI combines long-horizon coding and agentic data environments for post-training, Strata and MCP integrations for connecting AI agents to external APIs and SaaS tools, and MCP Sandbox for agent training and evaluation.

RAG-based document QA with multi-user support and agent reasoning

Kotaemon is an open-source RAG-powered document question-answering interface backed by Cinnamon AI. It supports multi-user workspaces with access controls, advanced retrieval pipelines including hybrid search and knowledge graph extraction, and agentic reasoning for complex multi-step queries. The web UI handles PDFs, Office documents, and images with citations pointing to exact source passages, making it suitable for both individual research and team knowledge management.

Kubernetes operator for serving AI inference workloads

KubeAI is an Apache-2.0 Kubernetes operator for deploying and scaling AI inference workloads, including LLMs, embeddings, reranking, and speech-to-text. It gives platform teams OpenAI-compatible endpoints, model proxy/controller primitives, model caching, scale-from-zero behavior, and cluster-native resource management for self-hosted inference on Kubernetes.

Modern application delivery platform for Kubernetes

KubeVela is a CNCF incubating project that provides a modern application delivery platform built on Kubernetes and the Open Application Model. It abstracts away infrastructure complexity by letting developers define applications declaratively with components, traits, and policies, while platform teams manage delivery workflows. KubeVela supports multi-cluster deployment, canary rollouts, GitOps integration, and extensible addon system.

Open-source toolkit for quantizing, deploying, and serving LLMs and vision-language models

LMDeploy is an Apache-2.0 toolkit for self-hosting LLM and vision-language model inference with TurboMind and PyTorch engines. It combines continuous batching, blocked KV cache, tensor parallelism, AWQ and KV-cache quantization with OpenAI-compatible APIs, multi-GPU distribution, offline pipelines, and production metrics.

Monorepo management for JavaScript

The original monorepo management tool for JavaScript. Used by Babel, Jest, React, and thousands of projects. Defined how the JS ecosystem thought about multi-package repos.

Google's production on-device LLM inference framework

LiteRT-LM is Google's official open-source framework for running large language models on-device across Android, iOS, Web, Desktop, and Raspberry Pi. Already deployed in Chrome and Pixel hardware, it provides production-grade on-device LLM inference with 1.4K+ GitHub stars. Apache 2.0 licensed.

Free, open-source local AI inference engine

LocalAI is an open-source local AI inference engine with 44K+ GitHub stars that runs LLMs, image generation, audio transcription, and embeddings entirely on consumer hardware without GPU requirements. Provides an OpenAI API-compatible REST endpoint as a drop-in replacement, supporting 1000+ models including LLaMA, Mistral, and Phi families. Features include text-to-speech, speech-to-text, function calling, constrained grammar output, and multi-modal capabilities all running locally.

Open-source cross-platform file sharing over local network

LocalSend is a free, open-source application for secure peer-to-peer file and message sharing between nearby devices over your local network. It works on Windows, macOS, Linux, Android, iOS, and Fire OS without requiring an internet connection or third-party servers. Each device generates TLS/SSL certificates for encrypted HTTPS communication, making it a privacy-first alternative to AirDrop that works across all operating systems.

Open-source auth infrastructure for modern apps

Logto is an open-source authentication and authorization platform built on OIDC and OAuth 2.1, serving as an alternative to Auth0, Cognito, and Firebase Auth. It provides pre-built sign-in flows with customizable UI, social login, Google One Tap, MFA, enterprise SSO via SAML, and role-based access control. SDKs cover 30+ frameworks including React, Next.js, Vue, Flutter, Go, and Python, with multi-tenancy support for SaaS applications.

macOS and Linux VM runtime for AI agents on Apple Silicon

Lume is an open-source CLI for creating and managing macOS and Linux virtual machines on Apple Silicon, built specifically for AI agent sandboxing, CI/CD pipelines, and desktop automation. Using Apple's native Virtualization.Framework for near-native performance, it provides the missing isolation layer for running coding agents safely — so an accidental destructive command doesn't affect your host machine.

IBM-backed ContextForge gateway for federating MCP, A2A, REST, and gRPC APIs

MCP Context Forge is IBM’s Apache-2.0 ContextForge project for operating a gateway, registry, and proxy across MCP servers, A2A agents, REST APIs, and gRPC services. It centralizes discovery, authentication, policy controls, federation, and observability, with deployment paths through PyPI, Docker, and Kubernetes.

Run LLMs natively on any device with ML compilation

MLC LLM is an open-source engine for deploying large language models natively across diverse platforms using machine learning compilation. It runs models on NVIDIA/AMD GPUs, Apple Silicon, mobile devices, and browsers via WebGPU without cloud dependencies. Features include OpenAI-compatible API, quantization support, and optimized backends for CUDA, Metal, Vulkan, and WebAssembly.

Run and fine-tune Vision Language Models locally on Mac

Open-source Python package for running and fine-tuning Vision Language Models locally on Mac using Apple's MLX framework. Supports multimodal inference with images, audio, and video across Qwen, DeepSeek, Phi, and Gemma architectures. Features OpenAI-compatible API server, Gradio chat UI, and KV cache optimization. 3.8K+ GitHub stars.

Enterprise-grade RAG and MCP knowledge base with one-click deployment

MaxKB is an enterprise-grade RAG platform with 21,000+ GitHub stars from the 1Panel team. It provides one-click deployment of knowledge bases with built-in LLM integration, MCP support, and a streamlined approach to document ingestion and retrieval that prioritizes operational simplicity over configuration complexity.

Declarative code-first ELT data integration

Meltano is a declarative, code-first data integration engine with 500+ Singer connectors for building ELT pipelines. It replaces custom API integration code with configuration-driven pipeline definitions that live in version control alongside application code. Integrates with dbt for transformation, supports scheduling and monitoring through a unified CLI, and powers production pipelines at scale.

High-performance S3-compatible object storage

MinIO is a high-performance, S3-compatible object storage server designed for AI, machine learning, and data-intensive workloads. Written in Go, it delivers industry-leading throughput for both read and write operations while maintaining full compatibility with the Amazon S3 API. MinIO includes an embedded web console for bucket management, a command-line client, and supports erasure coding, bitrot protection, and encryption at rest for enterprise-grade data durability.

Free MIT-licensed drop-in replacement for LocalStack

MiniStack is a free, MIT-licensed drop-in replacement for LocalStack that emulates 33 AWS services using real infrastructure — actual Postgres for RDS, real Redis for ElastiCache, real Docker for ECS — rather than faking API responses. Born from LocalStack's surprise paywall in March 2026, it starts in 2 seconds, idles at 30MB RAM versus LocalStack's ~500MB, and runs real services for accurate local AWS development.

Disaggregated KV cache storage and transfer for LLM serving

Open-source infrastructure for disaggregated LLM serving that pools KV caches across prefill and decode workers, with high-performance transfer, distributed storage and integrations for vLLM and SGLang.

Distributed inference orchestration above vLLM, SGLang and TensorRT-LLM

Open-source, datacenter-scale orchestration layer that coordinates vLLM, SGLang and TensorRT-LLM across nodes with disaggregated serving, KV-aware routing, multi-tier cache management and automatic scaling.

Secure sandboxed runtime for AI agent execution

NVIDIA OpenShell provides kernel-level isolation for AI agent workloads with Landlock, seccomp, and network namespace sandboxing. Announced at GTC 2026 with 17 enterprise partners including Adobe, Atlassian, SAP, and Salesforce, it offers declarative YAML policy enforcement, L7 HTTP inspection, and GPU passthrough — purpose-built to contain the blast radius when autonomous coding agents interact with filesystems and networks.

Open-source zero-trust networking with WireGuard

NetBird is an open-source zero-trust networking platform that creates encrypted WireGuard overlay networks between devices without opening ports or configuring firewalls. It provides peer-to-peer connectivity with NAT traversal, access control policies, DNS management, and a web dashboard for team management. NetBird replaces traditional VPNs with a simpler, more secure mesh networking approach for self-hosted infrastructure and remote teams.

Cross-platform on-device AI model runtime

Nexa SDK enables running frontier LLMs and multimodal models locally across PC, mobile, IoT, and wearables with automatic hardware acceleration for GPU, NPU, and CPU. It supports Qwen, Gemma, Llama, DeepSeek models with Python/C++ desktop SDKs, Android/iOS mobile SDKs, and Docker for edge deployment. Includes an OpenAI-compatible API server with chat and function calling support.

Extensibility-first open-source no-code/low-code platform for business apps

NocoBase is an open-source, self-hosted no-code and low-code platform built around a data-model-driven architecture and plugin-based microkernel. Unlike form-centric no-code tools, it separates data structure from UI, enabling unlimited block types and views per data collection. All functionality is delivered through plugins similar to WordPress. Features include AI Employees for in-app automation, visual workflow builder, RBAC permissions, and automatic REST/GraphQL API generation.

Low-code visual programming for event-driven apps

Node-RED is a flow-based, low-code development tool originally created by IBM in 2013 for wiring together hardware devices, APIs, and online services. Now an OpenJS Foundation project with nearly 23,000 GitHub stars, it provides a browser-based visual editor where flows are built by dragging and connecting nodes, then deployed to the Node.js runtime in a single click. Over 5,000 community-contributed nodes cover protocols like MQTT, Modbus, and OPC-UA alongside cloud APIs and databases.

Unified CI/CD and deployment platform with GPU support

Northflank is a unified platform for CI/CD, deployment, and infrastructure management that supports both traditional web applications and GPU-native AI workloads. It provides preview environments with full-stack database and AI model inclusion, BYOC deployment on any cloud, built-in observability, and automated scaling. Handles the complete deployment lifecycle from Git push to production with support for containers, databases, and GPU instances.

OpenAI API management gateway for 100+ LLM providers

One API is a self-hosted LLM API gateway that provides a unified OpenAI-compatible interface for managing multiple model providers including OpenAI, Azure, Anthropic, Google, and dozens of Chinese providers. It handles load balancing, quota management, rate limiting, token tracking, and channel-based routing through a web dashboard. Widely adopted in the Chinese developer ecosystem with over 18,000 GitHub stars.

Private, self-hosted research notebooks with flexible AI models, source chat, and podcasts

Open Notebook is an MIT-licensed, self-hosted alternative to NotebookLM for collecting sources, chatting over research, generating reusable transformations, and producing multi-speaker podcasts. Its Docker stack keeps notebook data under the user's control while supporting 18-plus model providers, including local Ollama and LM Studio workflows.

Linux Foundation fork of HashiCorp Vault for secrets management

OpenBao is the Linux Foundation's community-driven fork of HashiCorp Vault created after Vault's license change from open-source to BSL. It provides secrets management, encryption as a service, dynamic credentials, and PKI certificate management. Maintains API compatibility with Vault while developing under truly open-source governance with over 5,700 GitHub stars.

Rust-based agent OS with built-in security, WASM sandboxing, and multi-agent runtime

OpenFang is an open-source agent operating system built in Rust that provides a secure multi-agent runtime with WASM sandboxing, auditability layers, and multi-channel communication. It goes beyond typical orchestration SDKs by treating agent security and operational isolation as first-class concerns, making it suitable for teams deploying agents in environments where trust boundaries and audit trails matter.

Real-time usage metering and billing for AI, API, and DevOps products

OpenMeter is an Apache-2.0 open-source metering platform that ingests usage events, aggregates them into meters, and supports limits, entitlements, and usage-based billing. It fits AI, API, and DevOps products that need token-level reporting, customer usage visibility, or finance-ready chargeback data. Teams can self-host the core platform or evaluate OpenMeter's managed offering.

Open-source TypeScript headless CMS and app framework

Payload is an open-source TypeScript-first headless CMS and app framework with 32K+ GitHub stars. Built on Next.js with PostgreSQL or MongoDB, it provides auto-generated REST and GraphQL APIs, a customizable admin panel, access control, localization, versioning, and file uploads. Config-as-code defines collections in TypeScript for full type safety. Features live preview, draft system, block-based editor, and extensible plugins. Self-hostable with no vendor lock-in.

Single-binary open-source backend

PocketBase is an open-source backend in a single Go binary with 44K+ GitHub stars providing a real-time database, authentication, file storage, and admin dashboard. No dependencies, no Docker required — just download and run. Features collection-based data modeling, REST API, real-time subscriptions, OAuth2 authentication, file uploads with thumbnails, and a built-in admin UI. Extends with custom Go code or JavaScript hooks. Ideal for prototyping, MVPs, and small-to-medium applications.

Daemonless container engine — a drop-in Docker alternative that runs rootless.

Podman is a daemonless, open-source container engine developed by Red Hat as a secure alternative to Docker. It can run, build, and manage OCI containers and pods without requiring a daemon process or root privileges. CLI-compatible with Docker commands, making migration seamless.

Container management GUI for Docker and Kubernetes

Portainer is an open-source container management platform with 32K+ GitHub stars providing a web-based GUI for Docker, Docker Swarm, and Kubernetes. Simplifies container operations with visual management of containers, images, volumes, networks, and stacks without CLI expertise. Features user management with RBAC, environment templates, GitOps deployments, and edge computing support. Community Edition is free for up to 5 environments. Business Edition adds governance and support.

Heroku-like PaaS built on Kubernetes with YC backing

Porter is a YC-backed platform-as-a-service that provides a Heroku-like deployment experience on top of Kubernetes. It abstracts away cluster management while giving teams full access to underlying infrastructure when needed. Supports deploying from Git repos, Docker images, or Helm charts with automatic HTTPS, scaling, and preview environments. Runs on AWS, GCP, or Azure.

First commercially viable 1-bit LLMs that are 14x smaller and 8x faster

PrismML Bonsai delivers the first commercially viable 1-bit large language models with 8B, 4B, and 1.7B parameter variants. The 8B model runs in just 1GB of RAM versus 16GB for standard FP16 models, achieving 44 tokens per second on iPhone. Backed by $16.25M from Khosla Ventures and released under Apache 2.0, Bonsai makes capable LLMs practical for edge devices and resource-constrained environments.

Container-native local AI model serving with Podman

RamaLama is an open-source tool that containerizes AI model inference using Podman or Docker, eliminating host system configuration complexity. It auto-detects GPUs (NVIDIA, AMD, Intel, Apple Silicon), pulls models from HuggingFace, Ollama, and OCI registries, and runs them in isolated rootless containers with read-only mounts and network isolation. Developed under the Containers project (Red Hat ecosystem), it brings familiar container workflows to local LLM serving.

Kafka-compatible streaming platform, no JVM required

Redpanda is a Kafka-compatible streaming data platform written in C++ using the Seastar framework. It eliminates the need for ZooKeeper and the JVM, delivering up to 10x lower tail latencies and significantly reduced operational complexity. Redpanda ships as a single binary with a built-in schema registry, HTTP proxy, and message broker. It supports the Kafka wire protocol, so existing producers, consumers, and tools work without code changes. Backed by $165M+ in funding with 12.0K GitHub stars.

Cloud for the new default

Unified cloud platform for deploying web services, static sites, background workers, cron jobs, and managed databases with automatic scaling and zero DevOps. Auto-deploy from Git with preview environments, built-in DDoS protection, free TLS, and global CDN. Native runtimes for Node.js, Python, Go, Rust, Ruby, Elixir plus Docker. Managed PostgreSQL and Redis with daily backups, private networking, and infrastructure-as-code via Blueprints. Modern Heroku alternative with a free tier.

Edge AI deployment SDK for heterogeneous SoCs

Roofline AI is a contact-sales edge AI deployment toolkit built around an MLIR- and IREE-based compiler. Its SDK compiles models ahead of time, a lightweight C runtime executes them across CPUs, GPUs, and NPUs, and a performance dashboard tracks latency, throughput, memory use, and model coverage across devices.

GPU cloud platform for AI training and inference

RunPod is a GPU cloud platform providing on-demand and serverless GPU compute for AI training and inference workloads. It offers NVIDIA A100, H100, and RTX GPUs with per-second billing, serverless inference endpoints with auto-scaling, persistent storage, and Docker-based deployment. Popular with AI developers for its competitive pricing, fast provisioning, and developer-friendly API for deploying ML models at scale.