# LLM

Budget coding subscription by Moonshot AI

Kimi Coding Plan covers Moonshot AI's consumer subscription tiers and API pricing for accessing Kimi's AI coding capabilities, powered by the Kimi K2.5 model that scores 76.8% on SWE-Bench Verified. Includes a free Adagio tier with unlimited basic conversations, paid Andante and Presto tiers with higher K2.5 quotas, and pay-as-you-go platform API pricing for integrating Kimi into custom workflows.

Kubernetes operator for serving AI inference workloads

KubeAI is an Apache-2.0 Kubernetes operator for deploying and scaling AI inference workloads, including LLMs, embeddings, reranking, and speech-to-text. It gives platform teams OpenAI-compatible endpoints, model proxy/controller primitives, model caching, scale-from-zero behavior, and cluster-native resource management for self-hosted inference on Kubernetes.

Quantize and compress LLMs for optimized vLLM deployment

Open-source, Hugging Face-compatible library for quantizing and compressing LLMs (W8A8, W4A16, NVFP4/MXFP4, KV-cache and attention quantization, expert pruning) into vLLM-ready `compressed-tensors` checkpoints.

Input and output security scanners for LLM applications

LLM Guard is an open-source security toolkit by Protect AI that provides 15 input scanners and 20 output scanners to protect LLM applications from prompt injection, PII leakage, toxic content, secrets exposure, and data exfiltration. Each scanner is modular and independent — pick the ones you need, configure thresholds, and chain them into a pipeline. The library works with any LLM and has been downloaded over 2.5 million times. MIT licensed, Python 3.9+.

Reusable KV cache infrastructure for scalable LLM inference

Open-source KV cache management layer that persists, offloads and reuses model key-value caches across requests and serving engines to reduce repeated prefill work and improve inference throughput.

AI localization engine for software and docs

Lingo.dev is a Y Combinator F24-backed AI localization engine that automates translation of software interfaces and technical documentation into multiple languages while maintaining technical context. It preserves code variables, formatting tokens, and technical terminology that generic translation tools mangle, solving the global reach problem for developer-facing products and documentation.

Open-source AI-powered log analysis by Salesforce

LogAI is an open-source log analysis platform by Salesforce Research that uses deep learning to detect anomalies in large-scale system logs. It provides research-backed autonomous log troubleshooting capabilities, applying ML models to identify patterns, cluster log events, and surface anomalies that would be invisible in manual log review across high-volume production environments.

Unified Python/.NET framework for multi-agent AI

Microsoft Agent Framework is Microsoft's official unified SDK for building multi-agent AI workflows in Python and .NET. It consolidates Semantic Kernel and AutoGen into a single framework with MCP tool integration, graph-based workflows, human-in-the-loop patterns, and multi-agent orchestration. The framework reached Release Candidate status in February 2026 and is Microsoft's recommended path for production agent development.

Multi-model coding subscription by Minimax

The MiniMax Coding Plan is a subscription service providing developers with flat-rate access to MiniMax's AI models for coding tasks, with three monthly tiers designed to offer dramatically more value than per-token pricing from Anthropic or OpenAI. MiniMax positions its coding plans as delivering capacity equivalent to Claude Code Max at a fraction of the cost, making high-performance AI coding assistance accessible to a broader range of developers.

Disaggregated KV cache storage and transfer for LLM serving

Open-source infrastructure for disaggregated LLM serving that pools KV caches across prefill and decode workers, with high-performance transfer, distributed storage and integrations for vLLM and SGLang.

Distributed inference orchestration above vLLM, SGLang and TensorRT-LLM

Open-source, datacenter-scale orchestration layer that coordinates vLLM, SGLang and TensorRT-LLM across nodes with disaggregated serving, KV-aware routing, multi-tier cache management and automatic scaling.

Programmable safety rails for LLM applications

NeMo Guardrails is NVIDIA's open-source toolkit for adding programmable safety rails to LLM applications. It supports five guardrail types — input, dialog, retrieval, execution, and output rails — covering content safety, jailbreak detection, topic control, PII masking, hallucination detection, and fact-checking. The toolkit uses Colang, a domain-specific language for defining conversational constraints, and integrates with OpenAI, Azure, Anthropic, HuggingFace, and LangChain/LangGraph.

OpenAI API management gateway for 100+ LLM providers

One API is a self-hosted LLM API gateway that provides a unified OpenAI-compatible interface for managing multiple model providers including OpenAI, Azure, Anthropic, Google, and dozens of Chinese providers. It handles load balancing, quota management, rate limiting, token tracking, and channel-based routing through a web dashboard. Widely adopted in the Chinese developer ecosystem with over 18,000 GitHub stars.

Fork, customize, and ship AI agents on Vercel in minutes

Open Agents is a Vercel Labs open-source template for building and deploying cloud-hosted AI agents. It provides a production-ready Next.js starter with built-in tool use, streaming responses, multi-model support, and deployment on Vercel infrastructure. Developers can fork, customize agent behavior and tools, then ship agent-backed apps in minutes with automatic scaling and edge routing.

Open-source async coding agent you can run in your own sandbox

Open-source framework from LangChain AI for building your organization's internal coding agent — the same pattern Stripe's Minions, Ramp's Inspect, and Coinbase's Cloudbot follow. Built on LangGraph and Deep Agents, Open SWE runs each task in an isolated cloud sandbox (Modal, Daytona, Runloop, or LangSmith), invokes from Slack, Linear, or GitHub, orchestrates subagents, and opens pull requests autonomously — customizable end-to-end for your codebase and conventions.

Linux Foundation fork of HashiCorp Vault for secrets management

OpenBao is the Linux Foundation's community-driven fork of HashiCorp Vault created after Vault's license change from open-source to BSL. It provides secrets management, encryption as a service, dynamic credentials, and PKI certificate management. Maintains API compatibility with Vault while developing under truly open-source governance with over 5,700 GitHub stars.

Low-cost multi-model coding subscription

Budget-friendly AI coding plan featuring GLM-5, Kimi K2.5, and MiniMax M2.5/M2.7 models at $10/mo with a $12/5h usage cap. Hosted in US, EU, and Singapore for global low-latency access. Compatible with OpenCode and any OpenAI-compatible coding agent, offering an affordable alternative to premium API-based coding subscriptions.

Open-source Kubernetes cost monitoring (CNCF)

OpenCost is a CNCF-certified open-source tool for real-time Kubernetes cost monitoring that maps cloud spend directly to namespaces, deployments, pods, and labels. It provides granular cost allocation across teams and projects without vendor lock-in, supporting AWS, GCP, Azure, and on-premises clusters as the industry standard for open-source FinOps visibility in cloud-native environments.

Lightweight eval library for LLM applications

OpenEvals is a lightweight evaluation library from the LangChain team for testing LLM application quality using LLM-as-judge patterns. It provides pre-built prompt sets and evaluation functions that score model outputs against criteria like accuracy, relevance, coherence, and safety without requiring complex infrastructure. Available as both Python and JavaScript packages, OpenEvals complements OpenAI Evals with a simpler, framework-agnostic approach to quality measurement in agentic workflows.

Spec-driven development for AI coding assistants

OpenSpec is an open-source spec-driven development framework by Fission-AI for AI coding assistants. It adds a lightweight spec layer so developers and AI agents agree on what to build before any code is written. OpenSpec organizes changes in dedicated folders for proposals, specs, design docs, and tasks, and works across 21+ AI coding tools including Claude Code, Cursor, Windsurf, Continue, Gemini CLI, GitHub Copilot, and Amazon Q — bringing structured planning to agentic development.

Conversational data analysis with natural language queries over databases

PandasAI enables natural-language queries against databases, data lakes, CSVs, and parquet files using LLMs and RAG pipelines. With 23,400+ GitHub stars, it bridges the gap between database tools and AI by letting developers and analysts interact with data conversationally, supporting SQL, PostgreSQL, and various file formats.

Build and evaluate LLM apps end-to-end

Prompt Flow is Microsoft's open-source development suite for building, testing, evaluating, and deploying LLM-based applications end-to-end. It links LLM calls, prompts, Python code, and other tools into executable flows defined in YAML, with a VS Code extension providing a visual flow designer. The tool supports tracing LLM interactions for debugging, running batch evaluations with quality metrics against larger datasets, and integrating tests into CI/CD pipelines before production deployment.

Enterprise middleware for securing AI applications against prompt attacks

Prompt Security provides enterprise security middleware that protects AI applications from prompt injection, data leakage, jailbreaks, and toxic content generation. It sits between users and LLM APIs to inspect, filter, and sanitize inputs and outputs in real-time. Supports deployment as a proxy, SDK integration, or browser extension with customizable security policies and compliance reporting.

Governed analytics with a unified semantic layer

Querio turns plain English into SQL queries with a focus on governed analytics, connecting to live data warehouses and enforcing a unified semantic layer across the organization. It ensures that AI-generated queries remain consistent with business logic definitions, preventing the common problem where different teams get different answers to the same question from the same data.

Cursor for business workflow automation

Rebolt is a Y Combinator W25 company marketed as Cursor for business workflow automation, focusing on generating the business logic and internal UIs needed for complex company operations. It targets the internal tooling problem with an agentic automation approach, helping teams build custom workflow applications that connect existing business systems without traditional development cycles.

DRM and IP protection for AI model weights

RefortifAI is a Y Combinator P2026 batch company that provides DRM and intellectual property protection for AI models by obfuscating model weights so they only run inside a hardened runtime. It solves the critical problem of model weight protection for companies distributing custom LLMs to untrusted environments, preventing IP theft while maintaining inference performance.

Build modular, scalable LLM applications in Rust

Open-source Rust library for building scalable, modular, and ergonomic LLM-powered applications. Rig unifies 20+ model providers (OpenAI, Anthropic, Mistral, DeepSeek, Ollama, and more) and 10+ vector stores behind one trait-based interface, supports completion and embedding workflows, multi-turn streaming, and transcription/audio/image generation, with full GenAI Semantic Convention compatibility and WASM-ready core library — production agentic infra for Rust teams.

Agentic AI for software teams by Atlassian

AI-powered coding agent from Atlassian, deeply integrated with Jira, Bitbucket, and Confluence so it can validate code changes against acceptance criteria and plan multi-step development workflows aligned with team goals. Achieved 41.98% on SWE-bench full leaderboard at release. Available as both CLI and IDE integration, connecting project management and development within the Atlassian ecosystem.

LLM-powered web scraping with graph-based extraction pipelines

ScrapeGraphAI is a Python library that uses LLMs and graph-based logic to build automated, self-healing web scraping pipelines. Developers describe desired data in natural language and ScrapeGraphAI constructs a processing graph that extracts structured information from any website. It supports multiple LLM providers, achieves 96%+ accuracy on semantic extraction benchmarks, and adapts to layout changes automatically. Over 20,000 GitHub stars.

Microsoft's AI orchestration SDK for .NET, Python, and Java

Microsoft's open-source AI SDK that lets you combine AI models with conventional programming. Supports plugins, planners, memory, and function calling with availability for .NET, Python, and Java. Designed for enterprise developers building AI-powered applications within the Microsoft ecosystem, offering deep integration with Azure AI services and existing business logic.

Enterprise software composition analysis for supply chain security

Sonatype Lifecycle is an enterprise software composition analysis platform that identifies vulnerabilities, license risks, and quality issues in open-source dependencies throughout the development lifecycle. It integrates with IDEs, CI/CD pipelines, and artifact repositories to block risky components before they enter the codebase. Backed by the largest vulnerability database with proprietary research beyond public CVE data.

AI schema generator from natural language

Structa is an AI tool that turns natural language descriptions into production-ready database schemas and SQL in under 45 seconds, focused on PostgreSQL and Supabase. Recognized as an editor's choice for AI schema generation, it is the fastest idea-to-schema tool available, generating optimized table structures, relationships, indexes, and constraints from plain English project requirements.

Open-source library for generating synthetic tabular data

Synthetic Data Vault (SDV) is an MIT-backed open-source Python library for generating synthetic tabular, relational, and time-series data. It learns statistical patterns from real datasets and produces synthetic versions that preserve distributions, correlations, and referential integrity. Supports single-table, multi-table, and sequential data with built-in privacy and quality metrics.

Mozilla-backed browser infrastructure for AI agents

Tabstack is Mozilla's browser infrastructure service for AI agents, providing clean markdown extraction, structured JSON data, and automated browser actions through a fast API. With two-tier fetch escalation that achieves sub-600ms latency for static pages, robots.txt compliance, and ephemeral data handling, it offers an ethical alternative to aggressive web scraping tools — complete with an MCP server for Claude and Cursor integration.

Self-hosted AI accounting for freelancers and small teams

TaxHacker is an open-source, self-hosted AI accounting app that automatically extracts financial data from receipts, invoices, and bank statements using LLMs. It supports 170+ currencies and 14 cryptocurrencies with historical exchange rate conversion, multi-project accounting, and custom AI extraction fields. Works with OpenAI, Gemini, Mistral, or local models via Ollama—deploy with Docker and keep all financial data under your control.

Open-source LLM gateway with built-in optimization and A/B testing

TensorZero is an open-source LLMOps platform in Rust that unifies an LLM gateway, observability, prompt optimization, and A/B experimentation in a single binary. It routes requests across providers with sub-millisecond P99 latency at 10K+ QPS while capturing structured data for continuous improvement. Supports dynamic in-context learning, fine-tuning workflows, and production feedback loops. Backed by $7.3M seed funding, 11K+ GitHub stars.

Spec-driven development to keep AI agents on rails

Tessl is a spec-driven development platform founded by Snyk creator Guy Podjarny that keeps AI coding agents reliable by defining software through structured specifications before any code is written. The Tessl Framework guides agents to create plans, specs with linked tests, and API definitions — then generates and validates code against them. The Spec Registry provides 10,000+ specs for open-source libraries, preventing API hallucinations and version mixups in agent-generated code.

Secure text-to-SQL with schema-only processing

Text2SQL.ai is a secure text-to-SQL platform that uses a schema-only model where actual database data never leaves the developer's infrastructure. It provides an affordable solution for teams building internal analytics tools with a public API for embedding SQL generation into custom applications, prioritizing data privacy for organizations in regulated industries.

Open-source observability and self-healing layer for AI agents

TraceRoot is a YC S25-backed open-source observability platform purpose-built for AI agents and LLM apps. It combines OpenTelemetry-compatible tracing with an agentic debugging runtime that reads your source code, correlates failures with recent commits, and proposes fix PRs automatically. BYOK support spans seven LLM providers; the entire stack runs self-hosted via Docker Compose, with TraceRoot Cloud available for managed deployments.

Plan-first AI coding agent that designs before implementing

Traycer is a YC-backed AI coding agent that takes a plan-first approach to software development. It creates detailed implementation plans with architectural decisions and dependency analysis before writing code. The agent reasons about project structure and constraints to produce coherent, well-organized changes rather than incremental patches. Focused on reducing rework from poorly planned AI-generated code.

Multi-agent AI coding platform

Verdant (also known as Verdent) is an AI-native coding tool by Verdent AI that orchestrates multiple specialized AI agents working in parallel to handle software development workflows. One agent plans architecture, another writes code, a third reviews quality — functioning like a small AI development team. Founded by Zhijie Chen (ex-ByteDance algorithms lead), Verdant achieves 76.1% on SWE-bench Verified, among the top coding agents.

GLM-powered coding subscription by Zhipu AI

AI coding subscription built on Kimi K2.5 model. Provides competitive pricing tiers for code generation, completion, and chat-based development assistance. Compatible with major IDE integrations and OpenAI-compatible coding agents. Positioned as a cost-effective alternative for developers seeking strong reasoning capabilities at lower price points than leading providers.

Persistent memory layer for AI coding agents — keeps Claude Code, Codex, Cursor, and any MCP agent in context across sessions

agentmemory is an open-source MCP server that gives AI coding agents persistent, cross-session memory. Built on hybrid vector-graph search, it achieves 95.2% recall on the LongMemEval-S benchmark while using up to 92% fewer context tokens than naive context injection. Works out of the box with Claude Code, Codex, Cursor, Windsurf, Cline, OpenCode, Kilo Code, Hermes, and any MCP client through 51 MCP tools plus 12 hooks and 4 skills.

CLI that writes your git commit messages with AI

aicommits is a lightweight CLI tool that generates git commit messages using AI by analyzing your staged changes. Supports OpenAI, Anthropic, and local models via multiple providers. Configure message style, language, and conventional commit format. 8,800+ GitHub stars, MIT licensed. Built by Nutlope (Hassan El Mghari), known for popular open-source AI projects. Install via npm and run 'aicommits' to generate a message from your current diff.

AI chatbot framework for WeChat with multi-model and plugin support

chatgpt-on-wechat is an open-source framework for deploying AI chatbots on WeChat, the dominant messaging platform in China. It supports OpenAI, Claude, Gemini, Qwen, and local models through a plugin architecture. Features group chat management, image generation, voice messages, and knowledge base integration. Over 42,700 GitHub stars reflecting massive adoption in the Chinese developer community.

NVIDIA's LLM vulnerability scanner and red-teaming tool

garak is NVIDIA's open-source LLM vulnerability scanner for red-teaming AI models and applications. Probes for prompt injection, data leakage, hallucination, toxicity, encoding-based attacks, and dozens of other vulnerability categories. Runs automated attack sequences against any LLM endpoint and generates detailed vulnerability reports. Features a modular probe/detector architecture that is extensible with custom attack patterns. Named after the Star Trek character known for deception.

Lightweight C++ inference for Google Gemma models

gemma.cpp is Google's standalone C++ inference engine built specifically for running Gemma language models without Python or CUDA dependencies. It provides optimized CPU inference using SIMD instructions and Highway library, supports Gemma 2 and Gemma 3 models, and runs on x86 and ARM architectures. Designed for embedded systems, edge devices, and server deployments needing minimal overhead.

Find which AI models actually run on your hardware in one command

llmfit is a Rust-based terminal tool that matches over 200 LLM models from 30+ providers against your exact hardware specs. The interactive TUI scores each model on fit, speed, VRAM usage, and context length, helping you avoid downloading models that won't run on your machine. It supports Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio backends.