# LLM

Self-hosted AI platform with ChatGPT-like interface for local and cloud LLMs.

Extensible, self-hosted AI platform with 290M+ Docker pulls and 124K+ GitHub stars. Supports Ollama, OpenAI-compatible APIs, and any Chat Completions backend. Features built-in RAG, multi-user RBAC, voice/video calls, Python function workspace, model builder, and web browsing. Runs entirely offline with enterprise features including SSO and audit logging.

2x faster LLM fine-tuning with 70% less VRAM on a single GPU

Unsloth is an open-source framework for fine-tuning large language models up to 2x faster while using 70% less VRAM. Built with custom Triton kernels, it supports 500+ model architectures including Llama 4, Qwen 3, and DeepSeek on consumer NVIDIA GPUs. Unsloth Studio adds a no-code web UI for dataset creation, training observability, model comparison, and GGUF export for Ollama and vLLM deployment.

Self-hosted AI platform with RAG, agents, and 40+ connectors

Onyx is an open-core, self-hostable AI knowledge platform for enterprise search, RAG chat, deep research, custom agents, and workplace connectors. It connects to 40+ apps, supports permission-aware retrieval, and offers Cloud, Docker/Kubernetes, and enterprise deployment paths for teams that need controlled internal AI search.

Google's official CLI agent for coding with Gemini

Gemini CLI is Google's open-source terminal agent for coding with Gemini models, shell/file tools, web fetching, Google Search grounding, and MCP extensions. As of 18 June 2026, unpaid tier and Google One users are being moved to Antigravity CLI; supported Standard, Enterprise, and Google Cloud access paths remain the safer fit for teams.

Run local LLMs with an intuitive desktop GUI and OpenAI-compatible API server.

Free desktop application by Element Labs for discovering, downloading, and running open-source LLMs locally. Features a curated Hugging Face model browser, side-by-side model comparison, parameter tuning, and an OpenAI-compatible API server on localhost:1234. Powered by llama.cpp with Metal acceleration for Apple Silicon.

Google async coding agent for GitHub tasks, plans, and PRs

Jules is Google's async coding agent for GitHub repositories. Users start work from a prompt, GitHub issue label, scheduled task, or opt-in Suggested Task; Jules runs in a Google Cloud VM, proposes a plan, and opens PR-ready diffs. Free Jules offers 15 tasks/day and 3 concurrent tasks on Gemini 2.5 Pro; Pro/Ultra raise limits and start with Gemini 3 Pro access.

Evaluation-first LLM and agent observability

Confident AI is an evaluation-first observability platform that scores every trace and span with 50+ metrics, alerting on quality drops in LLM and agent applications. It goes beyond traditional APM by treating evaluation as core observability, providing actionable insights that help teams understand not just whether their AI applications are running but whether they are producing correct and useful outputs.

Open-source ML and LLM monitoring with 100+ metrics

Evidently AI is an open-source platform with 100+ pre-built metrics for monitoring data quality, model performance, and data drift in AI/ML pipelines. Available under Apache 2.0 with a cloud version, it helps teams detect when production data shifts away from training distributions, LLM output quality degrades, or feature pipelines introduce anomalies that silently degrade model accuracy.

CyberArk's open-source LLM fuzzing framework for AI security testing

FuzzyAI is CyberArk's Apache-2.0 framework for fuzzing LLM APIs to identify jailbreaks and related security vulnerabilities. Current README examples cover Ollama/local models, OpenAI, Anthropic, custom REST endpoints, and attacks such as ManyShot, Taxonomy, and ArtPrompt. Use it as a repeatable security-testing starting point, not a complete AI risk-management system.

Rust-native multi-agent orchestration for production

GraphBit is a Rust-native, multi-agent orchestration framework built for production. It targets the gap between Python-first frameworks like LangGraph and the operational expectations of enterprise systems — predictable memory, low latency, deterministic concurrency, and the ability to embed an agent runtime in services that already run Rust without dragging in a Python interpreter.

Autonomous Kubernetes management and predictive scaling

Sedai provides an autonomous control layer for Kubernetes that right-sizes workloads, remediates anomalies, and performs predictive autoscaling ahead of traffic demand. Sedai says it manages large enterprise cloud environments for customers including Palo Alto Networks and builds behavioral models to scale pods before demand arrives rather than reacting after performance degrades.

Open-source Kubernetes security platform for risk analysis and compliance

Kubescape is a CNCF-backed open-source Kubernetes security platform that scans clusters, manifests, and container images for vulnerabilities, misconfigurations, and compliance violations. It checks against NSA-CISA, MITRE ATT&CK, and CIS benchmarks, integrates into CI/CD pipelines, and provides runtime threat detection via eBPF. Supports SBOM generation and vulnerability scanning. Used by ARMO with growing enterprise adoption in cloud-native security.

Observability and lifecycle management for AI agents

AgentOps is an observability platform for monitoring, debugging, and managing AI agents. It provides session replay, event timelines, tool-call and LLM tracing, cost tracking, and dashboards for multi-agent workflows. Integrations span OpenAI, CrewAI, AutoGen, LangChain, LangGraph, LlamaIndex, Google ADK, OpenAI Agents, xAI, and 400+ LLMs/frameworks.

Microsoft's conversational multi-agent framework

AutoGen is an open-source programming framework from Microsoft Research for building AI agents and facilitating cooperation among multiple agents to solve complex tasks through multi-turn conversations. Pioneered conversable agents that interact, use tools, and involve humans in the loop for multi-agent workflows. v0.4 features a redesigned async event-driven architecture with stronger observability, flexible collaboration patterns, and reusable components.

AI code review that groups and orders PR changes

Cubic is an AI-powered code review platform used by teams at cal.com and n8n that organizes complex PRs by grouping files logically (Backend → API → UI) and provides an AI walkthrough of changes. At paid/team pricing should be verified in Cubic’s current purchase flow; it solves the jumpiness problem in large pull requests where reviewers lose context hopping between unrelated files, offering one-click fixes and custom rule enforcement.

Self-evolution engine for AI agents with auditable updates

Evolver is an open-source self-evolution engine for AI agents that turns run logs into auditable, reviewable updates via its Genome Evolution Protocol. Instead of ad hoc prompt tweaking, teams collect traces and Evolver proposes versioned diffs to prompts, tools and workflows that engineers can approve, reject or roll back like code.

OpenTelemetry-native LLM observability instrumentation

OpenLLMetry by Traceloop is an open-source instrumentation library with 7,000+ GitHub stars that adds OpenTelemetry-native tracing to LLM and AI agent applications. It captures detailed traces of model calls including latency, token usage, costs, and error rates, exporting data to any OpenTelemetry-compatible backend like Grafana, Datadog, or Jaeger for vendor-neutral AI observability.

Browser automation with AI vision — no XPath or DOM parsing needed

Skyvern automates browser-based workflows using LLMs and computer vision instead of brittle XPath or CSS selectors. It understands web pages visually, navigating forms, clicking buttons, and extracting data like a human would. Achieved 85.85% success rate on WebVoyager benchmark and SOTA on WRITE tasks for RPA. 21,000+ GitHub stars, AGPL-3.0 licensed. Skyvern Cloud offers managed usage-based hosting for teams that prefer not to self-host the infrastructure.

AI-powered pull request summaries and code review

WhatTheDiff is an AI tool that generates human-readable pull request summaries and suggests code improvements. It analyzes code diffs to explain what changed and why in plain language, helping reviewers understand PRs faster. Integrates with GitHub and supports automated refactoring suggestions through a /wtd command. Useful for teams wanting to improve PR review speed and maintain changelog quality.

Enterprise AI coding assistant with deep codebase understanding

AI coding assistant for professional software development that uses a proprietary Context Engine to maintain live understanding of your entire stack — code, dependencies, architecture, and git history. Achieved a record-breaking SWE-bench score and a 70% win rate over GitHub Copilot in head-to-head evaluations. Agent plans, creates files, and ships features with a 200K-token context and 89% multi-file refactor accuracy. First AI coding assistant with ISO/IEC 42001 certification.

Self-evolving local computer agent with a reusable skill tree

GenericAgent is a minimal, self-evolving autonomous agent from a 3.3K-line seed and ~3K core loop that gives LLMs system-level control of a local computer. It writes files, runs shell commands, browses the web, and uses keyboard/mouse/screen/mobile tools, while skill crystallization saves successful runs into a reusable skill tree that cuts token cost on repeats.

Federated SQL and semantic search across live data for AI agents

MindsDB Query Engine is a self-hosted, SQL-compatible federated query layer for AI agents. It connects live databases, warehouses, SaaS apps, documents, and vector stores without requiring a separate ETL pipeline; adds knowledge bases for semantic and hybrid retrieval; and exposes the result through SQL, MCP, HTTP, and MySQL/PostgreSQL-compatible interfaces. It is separate from MindsHub, the same team's hosted agent workspace.

Framework for evaluating LLM and agent performance

OpenAI Evals is an open-source framework and benchmark registry for evaluating LLM performance on custom tasks. It provides infrastructure for writing evaluation prompts, running them against models, and recording results in a structured format for comparison. The hosted Evals API on the OpenAI platform adds managed run tracking, dataset management, and programmatic access to evaluation pipelines. With 17,700+ GitHub stars, it serves as a foundation for systematic LLM quality measurement.

Open-source no-code automation platform, self-hosted Zapier alternative

Activepieces is an open-source business automation platform and self-hostable Zapier/Make alternative. It offers a visual flow builder, TypeScript code steps, AI-native blocks, and MCP-oriented pieces for services such as Gmail, Slack, Notion, and OpenAI. With 23,000+ GitHub stars, Docker deployment, and core/open MIT Expat licensing plus separately licensed enterprise directories, it suits teams seeking data sovereignty and familiar no-code workflow automation.

Open-source toolkit for building AI SRE incident response agents

OpenSRE is Tracer Cloud’s open-source public-alpha Python toolkit for building AI SRE agents that investigate and respond to production incidents. It ships 60+ tools across observability, databases, incident management, communications, deployment and protocol integrations, plus simulation/evaluation workflows for benchmarking agent accuracy before live pager use.

Autonomous scientific discovery via agentic tree search

AI Scientist v2 is Sakana AI's source-available system distributed under the AI Scientist Source Code License for fully autonomous scientific research using LLM-powered agentic tree search. It generates hypotheses, designs experiments, writes and executes code, analyzes results, and produces publishable manuscripts without human intervention. The system uses progressive exploration with backtracking to navigate the research space efficiently.

English-to-SQL with 100K+ users

AI2SQL converts plain English into optimized SQL queries with automatic schema detection, serving over 100,000 users worldwide. It features SQL explanation for learning, a formula generator for spreadsheets, and extensions for Slack and Chrome, providing a comprehensive ecosystem for non-technical users and developers who want to query databases faster using natural language input.

Cloud-native control plane for scalable GenAI inference

Open-source Kubernetes-native building blocks for deploying, routing and scaling GenAI inference, including an LLM gateway, autoscaling, LoRA management and KV-cache offloading.

Multi-model coding subscription by Alibaba Cloud

Alibaba Cloud Coding Plan is a flat-rate subscription that bundles access to multiple AI coding models — Qwen3.5-Plus, Qwen3-Coder-Next, GLM-4.7, and Kimi-K2.5 — under a single monthly fee, replacing unpredictable pay-per-token API pricing. It integrates with popular AI coding tools including Cline, Claude Code, and OpenCode, giving developers and small teams enterprise-grade Chinese AI models at dramatically lower price points than Western competitors.

Chat with your database in natural language

AskYourDatabase lets users chat with their databases using natural language, automatically generating SQL queries and visualizations from conversational questions. It autonomously creates dashboards and charts from queried data, serving as a self-serve reporting tool that reduces the business intelligence workload on engineering teams by letting non-technical users access data directly.

One desktop app for every LLM — private, cross-platform, extensible

Chatbox is a cross-platform desktop AI client supporting OpenAI, Claude, Gemini, DeepSeek, and local models via Ollama. All chat data stays on-device, making it ideal for privacy-conscious developers. Features include document analysis, code assistance with syntax highlighting, image generation, web search, and a local knowledge base for private Q&A. Available on Windows, macOS, Linux, Android, iOS, and web.

Heads-up display plugin for Claude Code

Real-time visibility layer for Claude Code sessions that surfaces context window usage, active tools, token consumption, and task status directly in your terminal. Turns the opaque agentic loop into a transparent, monitorable process. Particularly useful during long-running agent sessions where understanding resource consumption and progress helps you decide when to intervene, redirect, or let the agent keep working.

Multi-provider AI coding assistant with BYOK model access

CodeGPT is an AI coding assistant for VS Code and JetBrains IDEs that connects to multiple AI providers including OpenAI, Anthropic, Google, Mistral, and local models via Ollama using your own API keys. It offers code generation, explanation, refactoring, documentation writing, bug detection, and an agent marketplace with pre-built assistants for common tasks. The BYOK approach gives developers full cost control, no rate limits, and complete data ownership over their AI interactions.

Community cursor rules directory

Community-maintained collection of .cursorrules files that customize Cursor IDE's AI behavior for specific frameworks, languages, and project types. Define coding conventions, preferred libraries, architectural patterns, and style guidelines that the AI follows consistently. Popular rules exist for Next.js, React, Python, TypeScript, Tailwind, and more. Hosted on cursor.directory with 1-click installation. Essential for getting consistent, project-aware AI completions in Cursor.

OpenAI's custom chatbot builder and GPT Store

Create personalized GPT assistants with custom instructions, knowledge files, and tool integrations including browsing, DALL-E, and code interpreter. Publish to the GPT Store or keep private with no coding required. Enables anyone to build specialized AI assistants for specific domains, workflows, or audiences using OpenAI's consumer-friendly builder interface.

Cost-effective AI inference platform with 86+ models from $0.02/M tokens

DeepInfra is an AI inference platform offering 86+ LLM models with pricing starting at $0.02 per million tokens. Backed by $20.6M in funding including an $18M Series A from Felicis Ventures, it provides OpenAI-compatible endpoints for models including DeepSeek, Llama, and Mistral with pay-as-you-go pricing.

State-of-the-art open-source code language models

DeepSeek Coder is a family of open-source code language models trained from scratch on 2 trillion tokens of code and natural language data. Available in sizes from 1B to 33B parameters, these models support 80+ programming languages with 16K context windows and fill-in-the-blank capabilities. DeepSeek Coder outperforms CodeLlama-34B on HumanEval and MBPP benchmarks while being commercially licensable under MIT.

Instant AI API documentation from code snippets

DevDocsAI is a fast-utility AI documentation generator where developers paste code snippets and receive structured API documentation in 30 seconds. It targets individual makers and small teams who need quick documentation for specific code segments without the overhead of setting up a full documentation platform, providing rapid turnaround for function and endpoint documentation.

Open-source IDE for Java and enterprise development

Long-standing open-source IDE primarily for Java development, backed by the Eclipse Foundation. Features powerful Java tooling with refactoring, debugging, JUnit integration, and Maven/Gradle support. Extensible via a massive plugin marketplace covering C/C++, Python, PHP, and web development. Includes built-in Git support, terminal, and XML/JSON editors. Used extensively in enterprise Java, Android (legacy), and embedded systems development. Free and cross-platform.

AI-powered DAST platform specializing in API and GraphQL security

Escape is an AI-powered dynamic application security testing platform focused on API security including REST, GraphQL, and gRPC endpoints. It automatically discovers and tests API endpoints for vulnerabilities without requiring source code access. Features business logic testing that goes beyond OWASP patterns, CI/CD integration for shift-left security, and detailed remediation guidance for developers.

Open-source GPU control plane for scalable AI model serving

Open-source GPU cluster manager that configures vLLM, SGLang, TensorRT-LLM or custom engines, serves models through compatible APIs, and provisions SSH-accessible GPU instances across on-premises, Kubernetes and cloud environments.

Run open-source LLMs on your phone, fully offline and private

Google AI Edge Gallery is an open-source mobile app that lets you download and run large language models like Gemma directly on Android and iOS devices with zero cloud dependency. Built on MediaPipe and LiteRT, it features AI chat with reasoning mode, multimodal image analysis, real-time audio transcription, and autonomous agent skills—all running entirely on-device for complete privacy. A reference implementation for developers building offline-first AI experiences.

Constrained generation that guarantees valid LLM outputs every time

Guidance is Microsoft's structured generation library that enforces output constraints directly within LLM decoding. It supports JSON schemas, regex patterns, grammars, and interleaved generation-and-control flow to guarantee valid outputs from any compatible model. Works with local models via llama.cpp, Transformers, and remote APIs including OpenAI and Anthropic. Eliminates retry loops and post-processing for structured data extraction.

NLP and RAG pipeline framework by deepset

Haystack is an open-source AI orchestration framework by deepset for building production-ready LLM applications with explicit control over retrieval, routing, memory, and generation pipelines. Its component-based architecture lets developers chain specialized pieces into branching, looping pipelines for semantic search, RAG, QA, and autonomous agents. Integrates with OpenAI, Anthropic, Mistral, Cohere, Hugging Face, Azure, AWS Bedrock, and major vector stores.

Electron-based terminal with plugin ecosystem

Terminal emulator built on web technologies (HTML/CSS/JS) using Electron. Extensible via npm packages with a vibrant plugin and theme ecosystem. Created by Vercel with 43k+ GitHub stars. Offers split panes, configurable hotkeys, and a fully customizable UI that appeals to developers who want a modern, hackable terminal experience.

Open-source secrets management platform

Infisical is an open-source secrets management platform with 16K+ GitHub stars for syncing environment variables and secrets across teams, CI/CD pipelines, and infrastructure. Features end-to-end encryption, automatic secret rotation, dynamic secrets, access controls with audit logs, and native integrations with AWS, GCP, Azure, Kubernetes, Docker, GitHub Actions, and Vercel. Replaces scattered .env files with a centralized, encrypted secrets store accessible via dashboard, CLI, SDK, or API.

Open-source post-building layer for agents — tracing, evals, and online monitoring

Judgeval is the open-source post-building layer for AI agents from Judgment Labs, providing OpenTelemetry-based tracing, hosted and custom evaluation scorers, and online behavior monitoring for LLM-powered applications. Instrument any function with a single decorator, score live production traffic against faithfulness and instruction-adherence checks, and feed real-world failures back into reinforcement learning or supervised fine-tuning loops.

Heterogeneous CPU-GPU inference and SFT for large MoE models

Open-source framework for running and fine-tuning large Mixture-of-Experts models with heterogeneous CPU-GPU execution, optimized kernels, limited VRAM and SGLang or LLaMA-Factory integrations.