Best tools for Self-Hosted Deployment

Sandboxes for coding agents — Linux VMs, Git, and deploys in one box

Freestyle is YC-backed sandbox infrastructure built for AI coding agents, shipping secure Linux VMs with nested virtualization, Git servers, and one-click web deploys. It lets agents run real workloads, branch repos, and deploy apps under short-lived identities while billing only for active compute. Used in production by vly.ai, Rork, and Vibeflow.

Rust-native multi-agent orchestration for production

GraphBit is a Rust-native, multi-agent orchestration framework built for production. It targets the gap between Python-first frameworks like LangGraph and the operational expectations of enterprise systems — predictable memory, low latency, deterministic concurrency, and the ability to embed an agent runtime in services that already run Rust without dragging in a Python interpreter.

High-recall Postgres vector search at billion scale

VectorChord is a Postgres extension from TensorChord that brings high-recall vector search to PostgreSQL. As the spiritual successor to pgvecto.rs, it combines IVF indexes with RaBitQ quantization to deliver Pinecone-class performance at billion-vector scale while keeping all data inside a single Postgres database — no separate vector store, no two-system sync, no rewrites when the workload grows.

AI-native database for hybrid RAG retrieval

Infinity is an AI-native database from InfiniFlow that unifies dense vectors, sparse vectors, tensors, and full-text search in a single engine. Built for retrieval-augmented generation (RAG) at scale, it powers hybrid search workflows where lexical matching, semantic similarity, and reranking all happen against one storage layer instead of four loosely coupled services.

Open-source toolkit for building AI SRE incident response agents

OpenSRE is an open-source Python toolkit from Tracer Cloud for building AI SRE agents that investigate and respond to production incidents. It ships with connectors to Prometheus, Grafana, Kubernetes and incident platforms, plus a simulation harness that replays past incidents so teams can benchmark agent accuracy before trusting it on live pager rotations.

ML inference platform for production AI models

Baseten is the inference platform for deploying AI models at scale with dedicated and pre-optimized model APIs and performance-optimized infrastructure. Specializes in image generation, transcription, text-to-speech, LLM serving, embeddings, and compound AI workloads. Delivers 75% latency reduction with 415ms cold starts and 3000+ concurrent scaling. Available as managed cloud or self-hosted, trusted by Cursor, Notion, Descript, and Sourcegraph for production inference.

Kernel-space host intrusion detection system

Elkeid is ByteDance's open-source HIDS for hosts, containers, Kubernetes, and serverless workloads. Its kernel-level data collection via Kprobe hooks captures process lineage, privilege escalation attempts, file access patterns, and network connections with minimal overhead. Includes an Agent for telemetry, Detector for rule evaluation, Controller for policy management, and a Dashboard for alerts and investigation.

Declarative code-first ELT data integration

Meltano is a declarative, code-first data integration engine with 500+ Singer connectors for building ELT pipelines. It replaces custom API integration code with configuration-driven pipeline definitions that live in version control alongside application code. Integrates with dbt for transformation, supports scheduling and monitoring through a unified CLI, and powers production pipelines at scale.

Container-based CI/CD automation system

Concourse is an open-source CI/CD system built on composable primitives: resources for external artifacts, tasks for containerized work units, and jobs for orchestration. All pipelines are declarative YAML with version control, every task runs in an isolated container, and stateless workers enable horizontal scaling. Deployable via BOSH, Helm, Docker Compose, or standalone binary across any infrastructure.

Cross-platform on-device AI model runtime

Nexa SDK enables running frontier LLMs and multimodal models locally across PC, mobile, IoT, and wearables with automatic hardware acceleration for GPU, NPU, and CPU. It supports Qwen, Gemma, Llama, DeepSeek models with Python/C++ desktop SDKs, Android/iOS mobile SDKs, and Docker for edge deployment. Includes an OpenAI-compatible API server with chat and function calling support.

NVIDIA's optimized AI model serving platform

Triton Inference Server is NVIDIA's open-source inference serving platform that deploys AI models from TensorRT, PyTorch, ONNX, TensorFlow, OpenVINO, Python, and more across cloud, data center, and edge environments. It supports dynamic batching, model ensembles, concurrent model execution on GPUs and CPUs, and real-time, streaming, and batch inference patterns. Includes Model Analyzer for profiling and Model Navigator for automated optimization.

Open-source auth infrastructure for modern apps

Logto is an open-source authentication and authorization platform built on OIDC and OAuth 2.1, serving as an alternative to Auth0, Cognito, and Firebase Auth. It provides pre-built sign-in flows with customizable UI, social login, Google One Tap, MFA, enterprise SSO via SAML, and role-based access control. SDKs cover 30+ frameworks including React, Next.js, Vue, Flutter, Go, and Python, with multi-tenancy support for SaaS applications.

High-performance S3-compatible object storage

MinIO is a high-performance, S3-compatible object storage server designed for AI, machine learning, and data-intensive workloads. Written in Go, it delivers industry-leading throughput for both read and write operations while maintaining full compatibility with the Amazon S3 API. MinIO includes an embedded web console for bucket management, a command-line client, and supports erasure coding, bitrot protection, and encryption at rest for enterprise-grade data durability.

Lightweight OS for running AI agents in-process

agentOS is a portable open-source operating system for AI agents that delivers ~6ms cold starts at 32x lower cost than traditional sandboxes. Powered by WebAssembly and V8 isolates, it runs agents like Claude Code and Codex directly inside your process with granular permissions and host-managed tool access for S3, GitHub, and databases. Available as a simple npm package with no special infrastructure or vendor lock-in required.

Versatile microservice framework for any protocol

Armeria is an open-source microservice framework from the creator of Netty at LINE Corporation that supports gRPC, Thrift, REST, and GraphQL on a single server and port. It provides built-in decorators for metrics, distributed tracing, load balancing, authentication, rate limiting, circuit breakers, and automatic retries. The framework integrates seamlessly with Spring Boot, Dropwizard, and Reactive Streams while serving automated API documentation with interactive request testing.

Cloud-native POSIX filesystem on object storage

JuiceFS is a high-performance distributed POSIX filesystem built on object storage like S3 and metadata engines like Redis or MySQL. It enables seamless data sharing across thousands of clients with low latency and elastic throughput. JuiceFS ships with a Kubernetes CSI driver, Hadoop SDK compatibility, and FUSE mount support for AI training, big data analytics, and shared storage workloads. Apache 2.0 licensed with 13K+ GitHub stars.

Kafka-compatible streaming platform, no JVM required

Redpanda is a Kafka-compatible streaming data platform written in C++ using the Seastar framework. It eliminates the need for ZooKeeper and the JVM, delivering up to 10x lower tail latencies and significantly reduced operational complexity. Redpanda ships as a single binary with a built-in schema registry, HTTP proxy, and message broker. It supports the Kafka wire protocol, so existing producers, consumers, and tools work without code changes. Backed by $165M+ in funding with 12.0K GitHub stars.

Open-source deep learning text-to-speech toolkit

Coqui TTS is an open-source deep learning toolkit for text-to-speech synthesis, originally built by former Mozilla TTS engineers. It supports multi-speaker and multilingual synthesis, voice cloning from just six seconds of audio, and ships pre-trained models for 20+ languages. After Coqui shut down in 2023, the Idiap Research Institute forked and actively maintains it. With 45K+ GitHub stars, it remains the most popular open-source TTS framework in Python.

Modern application delivery platform for Kubernetes

KubeVela is a CNCF incubating project that provides a modern application delivery platform built on Kubernetes and the Open Application Model. It abstracts away infrastructure complexity by letting developers define applications declaratively with components, traits, and policies, while platform teams manage delivery workflows. KubeVela supports multi-cluster deployment, canary rollouts, GitOps integration, and extensible addon system.

Modern open-source server management panel

1Panel is a modern open-source Linux server management panel built with Go that provides a clean web interface for managing websites, databases, containers, and system resources. It features a marketplace with 165+ one-click app installs including Nextcloud and Bitwarden, automatic SSL provisioning with Let's Encrypt, visual Docker container management, and built-in firewall configuration. 1Panel also supports native AI agent deployment through Ollama integration.

Simple open-source personal cloud system

CasaOS is an elegant open-source personal cloud operating system that turns any hardware into a private home server with a one-line installation. It provides a beautiful web dashboard for managing Docker containers, a curated app store with one-click installs for tools like Nextcloud and Jellyfin, and built-in file management. CasaOS runs on Raspberry Pi, Intel NUC, old laptops, and cloud VMs with full support for Ubuntu, Debian, and Raspberry Pi OS.

Container-native local AI model serving with Podman

RamaLama is an open-source tool that containerizes AI model inference using Podman or Docker, eliminating host system configuration complexity. It auto-detects GPUs (NVIDIA, AMD, Intel, Apple Silicon), pulls models from HuggingFace, Ollama, and OCI registries, and runs them in isolated rootless containers with read-only mounts and network isolation. Developed under the Containers project (Red Hat ecosystem), it brings familiar container workflows to local LLM serving.

Open-source cross-platform file sharing over local network

LocalSend is a free, open-source application for secure peer-to-peer file and message sharing between nearby devices over your local network. It works on Windows, macOS, Linux, Android, iOS, and Fire OS without requiring an internet connection or third-party servers. Each device generates TLS/SSL certificates for encrypted HTTPS communication, making it a privacy-first alternative to AirDrop that works across all operating systems.

OAuth 2.1 provider framework for Cloudflare Workers

workers-oauth-provider is Cloudflare's official OAuth provider library for Workers. It implements the provider side of the OAuth 2.1 protocol with PKCE support and RFC 8414 compliance. Handles token management automatically via Cloudflare KV storage. Gives fullstack teams a path to implement OAuth at the edge without heavy auth servers. TypeScript-native with npm package available.