1,323 Developer Tools (2026)

LLM application observability and evaluation platform

LangSmith is LangChain's platform for debugging, testing, evaluating, and monitoring LLM applications in production. Provides detailed tracing of every step in LLM chains and agent workflows, dataset management for regression testing, prompt versioning, and automated evaluation with custom metrics. Features an annotation queue for human feedback, online monitoring dashboards, and integration with LangChain, LangGraph, and any LLM framework via the Python/JS SDK. Essential for production LLM ops.

Multi-agent simulation and LLM evaluation platform

Maxim AI is an evaluation and monitoring platform for LLM applications with multi-agent simulation capabilities. Features automated test generation from production traces, custom evaluation metrics, regression detection, and real-time monitoring dashboards. Supports simulating multi-turn conversations for testing chatbots and agents at scale. Provides prompt versioning, A/B testing, and integration with CI/CD pipelines for continuous quality assurance of AI outputs across deployment cycles.

Terminal-native coding agent by Mistral AI

Mistral Vibe is a native CLI coding agent from Mistral AI, powered by their Devstral coding model, that runs directly in the terminal with deep project awareness including automatic file structure scanning and Git status integration. Released December 2025 alongside Devstral 2, it features smart @-file references, shell command execution, and conversational workflows. Mistral's entry into the agentic CLI coding market alongside Claude Code and OpenCode.

Data and AI observability for enterprise teams

Monte Carlo is the leading data and AI observability platform using ML to monitor pipelines, warehouses, and lakes for quality issues. It detects freshness delays, volume anomalies, schema changes, and distribution shifts before they impact analytics. With 500+ deployments at Nasdaq, Honeywell, and Roche, it provides automated root cause analysis, field-level lineage, and incident management. Available on AWS and Azure Marketplace.

Visual IDE for React that designers can use

Onlook is an open-source visual editor for React and Next.js with 25.9K+ GitHub stars that lets designers edit real code-backed interfaces on an infinite canvas. It connects visual changes, AI chat, component and layer inspection, and versioned code workflows so teams can prototype and refine product UI without treating design files as the source of truth.

OpenTelemetry-native LLM observability instrumentation

OpenLLMetry by Traceloop is an open-source instrumentation library with 7,000+ GitHub stars that adds OpenTelemetry-native tracing to LLM and AI agent applications. It captures detailed traces of model calls including latency, token usage, costs, and error rates, exporting data to any OpenTelemetry-compatible backend like Grafana, Datadog, or Jaeger for vendor-neutral AI observability.

Open-source AI-powered PR review with slash commands

PR-Agent by Qodo is the most widely adopted open-source AI code review tool with 11,500+ GitHub stars and 207 contributors. It provides automated PR description generation, intelligent line-by-line code review, and actionable improvement suggestions through intuitive slash commands like /describe, /review, and /improve, integrating natively with GitHub Actions, GitLab CI, Bitbucket, and Azure DevOps.

Open-source AI coding agent with self-hosted deployment option

Refact.ai is an open-source/on-premise oriented AI coding agent for VS Code and JetBrains that supports BYOK model routing, codebase understanding, developer-tool integrations and self-hosted deployment. Its public site now warns that Refact Cloud is shutting down soon, so teams should treat hosted availability as a migration risk and validate the current enterprise support path.

Browser automation with AI vision — no XPath or DOM parsing needed

Skyvern automates browser-based workflows using LLMs and computer vision instead of brittle XPath or CSS selectors. It understands web pages visually, navigating forms, clicking buttons, and extracting data like a human would. Achieved 85.85% success rate on WebVoyager benchmark and SOTA on WRITE tasks for RPA. 21,000+ GitHub stars, AGPL-3.0 licensed. Skyvern Cloud offers managed usage-based hosting for teams that prefer not to self-host the infrastructure.

Official Slack MCP server for approved workspace search, messaging, canvas, and user-context actions.

Slack MCP Server is Slack’s official remote MCP layer for giving approved AI clients workspace context and controlled actions. It lets agents search messages, files, users, and channels, draft or send messages, read threads, manage canvases, and authenticate through Slack OAuth while workspace admins approve integrations and normal Slack rate limits still apply.

AI-powered pull request summaries and code review

WhatTheDiff is an AI tool that generates human-readable pull request summaries and suggests code improvements. It analyzes code diffs to explain what changed and why in plain language, helping reviewers understand PRs faster. Integrates with GitHub and supports automated refactoring suggestions through a /wtd command. Useful for teams wanting to improve PR review speed and maintain changelog quality.

Enterprise AI coding assistant with deep codebase understanding

AI coding assistant for professional software development built on Augment's Context Engine, which maintains a live understanding of your codebase, dependencies and history. It offers a coding agent, chat, CLI access, MCP and native tools, and Cosmos cloud compute, and is sold as flat-priced team plans with usage measured in dollars. Augment Code is SOC 2 Type II certified and offers ISO/IEC 42001 compliance and CMEK on enterprise plans.

AI-powered data quality for ML datasets

Cleanlab is a data-centric AI library that automatically detects and fixes label errors, outliers, and data quality issues in machine learning datasets. It works with any ML model and any data type including text, images, tabular, and audio by analyzing model predictions to identify mislabeled examples, near-duplicates, and ambiguous data points. Cleanlab helps teams improve model accuracy by cleaning training data rather than tuning model architecture.

Open-source dev environment management with AI integration

Daytona is secure, elastic infrastructure for running AI-generated code in isolated sandboxes. It gives agents and developer workflows programmable environments with dedicated kernel, filesystem, network, vCPU, memory, and disk, backed by OCI/Docker compatibility, SDK/API access, and under-90ms sandbox startup. The project has 72,000+ GitHub stars and is AGPL-3.0 licensed.

Autonomous AI software engineer by Cognition

Devin is Cognition's managed AI software engineer for delegating engineering tasks to cloud and desktop agents. It can plan work, navigate codebases, write and run code, test changes, open PRs, review/autofix issues, and collaborate through GitHub, GitLab, Bitbucket, Linear, Jira, Slack, and Teams. Current Devin surfaces include Devin Cloud, Devin Desktop, Devin CLI, Devin Review, Windows VM support, DeepWiki, Ask Devin, and team/enterprise controls.

AI assistant built into JetBrains IDEs

Native AI assistant integrated into all JetBrains IDEs with code completion, chat, and context-aware suggestions. Powered by multiple LLMs with deep IDE integration for refactoring, analysis, and documentation. Leverages JetBrains' rich code understanding to deliver more accurate suggestions than standalone AI tools, especially for Java, Kotlin, and Python projects.

Run LLMs as a single portable executable file

Llamafile by Mozilla packages a complete LLM — model weights, inference engine, and OpenAI-compatible API server — into a single executable file that runs on Mac, Windows, Linux, FreeBSD, and OpenBSD with no installation. Built on llama.cpp and Cosmopolitan Libc for cross-platform portability, it delivers GPU-accelerated inference when available and falls back to optimized CPU execution. Supports GGUF models with a built-in web chat UI and REST API for integration.

Unified AI code review and application security platform

Panto AI is a unified AI-driven code review and application security platform that combines static analysis, secrets detection, dependency scanning, and infrastructure-as-code security into a single PR workflow. It contextualizes issues based on repository structure and code history to deliver actionable insights directly within pull requests on GitHub, GitLab, and Bitbucket with zero-configuration onboarding and compliance-ready reporting for SOC 2, ISO, and PCI-DSS.

AI code integrity platform for test generation and quality

Qodo, formerly CodiumAI, is an AI code integrity platform focused on reviewing, testing, and improving code quality across the development lifecycle. It provides AI-powered code reviews, automated test generation, and context-aware suggestions that span IDE, pull request, and CI/CD workflows. Qodo distinguishes itself from general-purpose AI coding assistants by focusing on quality assurance rather than code generation alone.

Evaluation framework for RAG pipelines

RAGAS is an Apache-2.0 open-source evaluation framework with 14K+ GitHub stars that provides standardized metrics for assessing RAG pipeline quality. It measures faithfulness, answer relevancy, context precision, and context recall to identify whether retrieval, generation, or both are failing. It is framework-agnostic, supports LLM-as-judge evaluation, and its README discloses minimal anonymized Open Analytics with a RAGAS_DO_NOT_TRACK opt-out.

MIT-licensed autonomous coding-agent reference, now superseded for many new uses by mini-swe-agent.

SWE-agent is an MIT-licensed autonomous coding-agent reference from Princeton and Stanford researchers that takes GitHub issues and attempts fixes with a bring-your-own language model. Its agent-computer interface remains foundational for repository navigation, editing, and test execution. The README now says development has shifted to mini-swe-agent, which supersedes SWE-agent and is generally recommended going forward.

Serverless vector database with pay-as-you-go API pricing

Upstash Vector is a managed serverless vector database for RAG, semantic search, and embedding lookup. It is separate from the existing Upstash platform record in the aicoolies catalog: this slug covers the Vector product line, not the broader Redis, Kafka, or QStash platform.

Open-source, local-first Notion alternative with built-in AI

AppFlowy is an open-source workspace platform offering documents, wikis, databases, and project management as a privacy-first alternative to Notion. Built with Rust and Flutter for native performance, it runs fully offline with local data storage by default. AI features include writing assistance, summarization, and translation powered by configurable LLM providers. 72.5K+ GitHub stars, backed by a growing contributor community with self-hosted and cloud deployment options.

Git-friendly offline API client

Open-source API client for testing REST, GraphQL, and gRPC APIs as a lightweight, privacy-focused alternative to Postman. Stores API collections as plain files in your Git repository for easy version control and team collaboration. Features environment variables, scripting, collection runner, and a declarative syntax called Bru. Offline-first with no cloud account required. Available on macOS, Windows, and Linux. 44K+ GitHub stars and growing rapidly among developers leaving Postman.

Edge-native vector database for Workers and AI applications

Cloudflare Vectorize is Cloudflare’s managed vector database for Workers and edge AI applications. It is distinct from the existing Cloudflare Workers tool page: Workers is the compute runtime, while Vectorize is the embedding index and vector-query layer used to add semantic retrieval to Cloudflare-hosted apps.

AI-native data application framework with SQL generation and agents

DB-GPT is an open-source AI-native data app framework combining SQL generation, database chat, RAG, and multi-agent orchestration for data-centric workflows. It supports natural language to SQL conversion, automated data analysis, and custom data app development. Integrates with MySQL, PostgreSQL, SQLite, and more. 19,000+ GitHub stars, MIT licensed. Positioned as an alternative to MindsDB for teams building AI-powered data applications and internal database tools.

Drag-and-drop LLM flow builder

Flowise is an open-source, low-code UI and API platform for building customized LLM orchestration flows, multi-agent systems, and autonomous AI applications using drag-and-drop node graphs.

Multi-agent CLI pair programmer with FORGE, MUSE, and SAGE agents

Model-agnostic terminal coding tool with 3 specialized agents: FORGE for code editing, MUSE for planning and review, and SAGE for research. Connects to hundreds of LLM providers and models with local-first privacy and conversational Git integration. Apache 2.0 licensed. A thoughtfully designed multi-agent approach that separates concerns between coding, thinking, and information gathering for more reliable results.

Google's enterprise AI coding assistant powered by Gemini

Gemini Code Assist is Google's business AI coding assistant for VS Code, JetBrains IDEs, Cloud Workstations, GitHub, and Google Cloud workflows. Standard and Enterprise plans include Gemini 3, a 1M-token context window, Gemini CLI access, preview agent mode, code completion, chat, transformation, PR review, and enterprise governance controls for GCP-heavy teams.

Self-evolving local computer agent with a reusable skill tree

GenericAgent is a minimal, self-evolving autonomous agent from a 3.3K-line seed and ~3K core loop that gives LLMs system-level control of a local computer. It writes files, runs shell commands, browses the web, and uses keyboard/mouse/screen/mobile tools, while skill crystallization saves successful runs into a reusable skill tree that cuts token cost on repeats.

Delightful JavaScript testing

Jest is Meta's open-source JavaScript and TypeScript testing framework with batteries-included support for unit, integration, and snapshot tests. Zero-config for most projects, it bundles its own test runner, assertion library, mocking utilities, code coverage, and parallel execution. The de facto standard for React, Node.js, Next.js, and modern JS apps, and a common target for AI-generated test suites.

AI coding agent by Moonshot AI

Kimi Code is Moonshot AI's AI coding agent, available as an open-source (MIT) terminal CLI, a desktop app and IDE extensions. It reads and edits code, runs shell commands, searches files, fetches web pages and plans multi-step work, and now runs Kimi K3.1 Pro with a 1M-token context window. Sign in with a Kimi membership (Plus and above) or a Kimi API key; the CLI can also use other compatible providers, supports MCP servers and plugins, and accepts video input.

AI agent testing and LLM evaluation platform

LangWatch is an AI agent testing, LLM evaluation, and observability platform for production AI teams. It combines traces, evaluations, scenario simulations, guardrails, prompt management, and Optimization Studio/DSPy workflows, with cloud and self-managed options for teams that need release-quality feedback loops.

AI-powered SRE agent for Kubernetes troubleshooting

Metoro is an AI SRE platform for Kubernetes that combines observability with autonomous troubleshooting. Its Guardian agent monitors cluster health, correlates metrics, logs, and traces to identify root causes, and suggests remediation actions. Features an MCP server for integration with AI coding agents and natural language querying of infrastructure state.

Federated SQL and semantic search across live data for AI agents

MindsDB Query Engine is a self-hosted, SQL-compatible federated query layer for AI agents. It connects live databases, warehouses, SaaS apps, documents, and vector stores without requiring a separate ETL pipeline; adds knowledge bases for semantic and hybrid retrieval; and exposes the result through SQL, MCP, HTTP, and MySQL/PostgreSQL-compatible interfaces. It is separate from MindsHub, the same team's hosted agent workspace.

Framework for evaluating LLM and agent performance

OpenAI Evals is an open-source framework and benchmark registry for evaluating LLM performance on custom tasks. It provides infrastructure for writing evaluation prompts, running them against models, and recording results in a structured format for comparison. The hosted Evals API on the OpenAI platform adds managed run tracking, dataset management, and programmatic access to evaluation pipelines. With 17,700+ GitHub stars, it serves as a foundation for systematic LLM quality measurement.

All-in-one open-source observability — logs, metrics, traces, RUM

OpenObserve is an open-source observability platform that unifies logs, metrics, traces, and real user monitoring in a single binary. It claims 140x lower storage costs than Elasticsearch through columnar storage and compression, with native OpenTelemetry support, a built-in query UI, dashboards, and alerts. Designed for AI and cloud-native workloads at petabyte scale. Over 15,000 GitHub stars.

Industry-standard incident management and on-call alerting platform

PagerDuty is an enterprise incident management platform for on-call scheduling, alert routing, escalation policies, and incident response orchestration. It integrates with 750+ monitoring, ticketing, cloud, and collaboration tools, including Datadog, Slack, Jira, AWS, and Microsoft Teams. Current pricing lists a Free tier for up to 5 users, Professional from $21/user/mo annually ($25 monthly), Business from $41/user/mo annually ($49 monthly), and Enterprise custom.

Prompt registry, observability, and evaluation workflows for LLM applications.

PromptLayer is a prompt management, observability, and evaluation platform for LLM applications. Teams use its Prompt Registry, visual editor, request logs, Tables, evaluations, Tool Registry, and Skill Collections to version prompts, replay requests, compare variants, run datasets, and ship prompt changes without redeploying code. Pricing starts with Free $0 for 5 users and 2.5K requests/month, Pro $49/month, Team $500/month, and Enterprise custom.

MCP server registry and hosting

Registry and management platform for Model Context Protocol (MCP) servers that helps teams securely discover, install, deploy, and connect MCP servers for AI assistants. Smithery combines a searchable catalog, CLI setup, hosted deployments, namespaces, connection APIs, and scoped service-token flows for clients such as Claude, Cursor, Windsurf, and Codex.

AI-native IDE by ByteDance

TRAE is an AI development environment with a Desktop app (an Agent window plus an IDE window with IDE and SOLO modes), a web app and a mobile app. Agents plan and run tasks locally or in the cloud, and CUE handles completion and next-edit prediction. Built-in models include GPT-6, GPT-5.x, Gemini 3.x, Seed, GLM, DeepSeek, Kimi and MiniMax; other models, including Claude, can be added with your own API key.

Kubernetes-native framework for DevOps AI agents

kagent is a Kubernetes-native AI agent framework developed at Solo.io and accepted into the CNCF sandbox. It provides a structured environment for running DevOps-focused agents directly within Kubernetes clusters, with a dedicated kmcp toolkit for cloud-native operations. Unlike general-purpose agent frameworks, kagent targets platform engineers and SREs who need AI assistance with cluster management, troubleshooting, and infrastructure automation workflows.

Open-source no-code automation platform, self-hosted Zapier alternative

Activepieces is an open-source business automation platform and self-hostable Zapier/Make alternative. It offers a visual flow builder, TypeScript code steps, AI-native blocks, and MCP-oriented pieces for services such as Gmail, Slack, Notion, and OpenAI. With 23,000+ GitHub stars, Docker deployment, and core/open MIT Expat licensing plus separately licensed enterprise directories, it suits teams seeking data sovereignty and familiar no-code workflow automation.

Self-hosted proxy API for routing AI CLI accounts into OpenAI-compatible endpoints

CLIProxyAPI is an open-source Go proxy server that wraps Gemini CLI, Claude Code, OpenAI Codex, Grok Build, and related CLI account flows behind OpenAI/Gemini/Claude-compatible API endpoints. Use it carefully: it can touch OAuth sessions, auth files, logs, and provider account policies, so production use needs credential and ToS review.

Free cloud-hosted Jupyter notebooks with GPU access

Google Colab is a free cloud-hosted Jupyter notebook environment providing access to GPUs and TPUs for machine learning, data analysis, and education. Requires no setup — notebooks run in the browser with pre-installed ML libraries including TensorFlow, PyTorch, scikit-learn, and pandas. Features Google Drive integration for persistent storage, collaborative editing, and sharing. Free tier includes limited GPU access; Colab Pro provides faster GPUs, more memory, and longer runtimes.

Open-source toolkit for building AI SRE incident response agents

OpenSRE is Tracer Cloud’s open-source public-alpha Python toolkit for building AI SRE agents that investigate and respond to production incidents. It ships 60+ tools across observability, databases, incident management, communications, deployment and protocol integrations, plus simulation/evaluation workflows for benchmarking agent accuracy before live pager use.

AI-powered code review and refactoring assistant

Automated code reviewer that integrates with GitHub PRs. Detects code quality issues, suggests refactoring, and auto-generates reviews. Python specialist with growing JS/TS support and 1.8k+ GitHub stars. Enforces team coding standards consistently and helps developers learn best practices through inline suggestions on every pull request.

AWS AI assistant for terminal and code

Amazon Q Developer CLI is an agentic AI assistant from AWS that enhances the terminal experience with deep AWS integration, code generation, and natural-language command translation. Completes multi-step development tasks, troubleshoots AWS services, writes infrastructure code, and handles documentation lookups directly from the shell. Included with AWS accounts and integrates with existing AWS credentials.