Braintrust

Lightweight server monitoring with Docker stats and alerts

Beszel is a lightweight, self-hosted server monitoring platform built in Go that tracks CPU, memory, disk, network, GPU, temperature, and Docker container metrics with historical data visualization and configurable alerts. Its simple hub-and-agent architecture deploys in minutes and consumes minimal resources compared to traditional monitoring stacks like Prometheus and Grafana.

Open-source LLM gateway with built-in optimization and A/B testing

TensorZero is an open-source LLMOps platform in Rust that unifies an LLM gateway, observability, prompt optimization, and A/B experimentation in a single binary. It routes requests across providers with sub-millisecond P99 latency at 10K+ QPS while capturing structured data for continuous improvement. Supports dynamic in-context learning, fine-tuning workflows, and production feedback loops. Backed by $7.3M seed funding, 11K+ GitHub stars.

Open-source LLM engineering platform for observability

Langfuse is an open-source LLM engineering platform with 29K+ GitHub stars for tracing, evaluating, and monitoring AI applications. Acquired by ClickHouse, it provides detailed traces of LLM calls, prompt management with versioning, dataset-based evaluation, user feedback collection, and cost tracking. Framework-agnostic with native integrations for LangChain, LlamaIndex, OpenAI SDK, and Vercel AI SDK. Offers both self-hosted deployment and a managed cloud service.

LLM application observability and evaluation platform

LangSmith is LangChain's platform for debugging, testing, evaluating, and monitoring LLM applications in production. Provides detailed tracing of every step in LLM chains and agent workflows, dataset management for regression testing, prompt versioning, and automated evaluation with custom metrics. Features an annotation queue for human feedback, online monitoring dashboards, and integration with LangChain, LangGraph, and any LLM framework via the Python/JS SDK. Essential for production LLM ops.

Open-source platform for the complete machine learning lifecycle.

MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. Covers experiment tracking, model packaging, model registry, and deployment. Created by Databricks and now a Linux Foundation project. Integrates with TensorFlow, PyTorch, scikit-learn, Hugging Face, and all major ML frameworks.

Open-source LLM observability through a single-line proxy

Helicone is an open-source LLM observability and AI gateway platform with proxy-based request logging, cost tracking, latency monitoring, caching, rate limits, user analytics, prompt tools, and HQL. It supports OpenAI, Anthropic, Azure, LiteLLM, Anyscale, Together AI, and OpenRouter integrations, and now presents itself as part of Mintlify while continuing managed and self-hosted gateway/observability workflows.

Better Stack is a hosted observability and incident-management platform that combines uptime monitoring, on-call workflows, status pages, logs, traces, metrics, error tracking, session replay, and an AI SRE interface. It is aimed at teams that want one SaaS control plane for telemetry and incident response.

HyperDX is the ClickStack UI for ClickHouse-backed observability. It provides a frontend for exploring logs, traces, metrics, session replay, dashboards, and alerts, with an OpenTelemetry-centered deployment path for teams that want a self-hosted or ClickHouse-aligned observability stack.

SigNoz is an OpenTelemetry-native observability platform for collecting and correlating logs, metrics, and traces. Teams can self-host it or use SigNoz Cloud, with dashboards, alerting, query workflows, and enterprise controls for cloud-native and AI application telemetry.

Sentry-style observability for AI agent conversations

Latitude is an agent observability platform for teams that need to inspect LLM traces, conversations, issues, and evaluation feedback in one workflow. Its public repo and docs position it as a Sentry-style monitor for AI agents, with semantic search, issue detection, annotations, MCP-assisted fixes, and cloud or self-hosted deployment paths for production debugging.

Session reports for Claude Code and Codex runs

Spotlight by Backplanes turns completed Claude Code and Codex sessions into concise reports for engineering, security, and spend review. The CLI installs on macOS, Linux, or WSL 2, watches sessions after they finish, redacts PII and credentials locally before upload, then summarizes files touched, commands run, external domains reached, scope drift, risky actions, and next-session improvements.

OpenTelemetry-native observability with AI tracing, logs, traces, metrics, and session replay — self-hosted in 90 seconds.

Traceway is an open-source, OpenTelemetry-native observability platform that combines logs, traces, metrics, exceptions, session replay, and AI tracing in a single self-hosted system. MIT licensed with no open-core restrictions, it deploys in 90 seconds via Docker Compose and accepts OTLP/HTTP from any OTel SDK without a Collector or per-language vendor SDK.