# Python

Distributed AI compute engine for scaling Python and ML workloads

Ray is an open-source distributed computing framework built for scaling AI and Python applications from a laptop to thousands of GPUs. It provides libraries for distributed training, hyperparameter tuning, model serving, reinforcement learning, and data processing under a single unified API. Ray's public site highlights OpenAI and other enterprise users. Maintained by Anyscale with Apache-2.0 open-source licensing.

Unified framework for fine-tuning 100+ large language models

LLaMA-Factory is an open-source toolkit providing a unified interface for fine-tuning over 100 LLMs and vision-language models. It supports SFT, RLHF with PPO and DPO, LoRA and QLoRA for memory-efficient training, and continuous pre-training. The LLaMA Board web UI enables no-code configuration, while CLI and YAML workflows serve advanced users. Integrates with Hugging Face, ModelScope, vLLM, and SGLang for model deployment.

Ultra-fast SIMD-accelerated embedded vector database with TurboQuant compression

High-performance Rust and Python vector search index utilizing TurboQuant 2-to-4-bit data-oblivious quantization for ultra-compact memory footprint and zero-training ANN retrieval.

High-throughput LLM serving engine

vLLM is an Apache-2.0 LLM inference and serving engine focused on high-throughput self-hosted model APIs. It combines PagedAttention, continuous batching, prefix caching, quantization options, OpenAI-compatible serving, structured outputs, metrics, Docker/Kubernetes deployment guidance and integrations with agent and LLM frameworks.

Build and share ML web apps in Python with a few lines of code

Gradio is a 42K+ star Apache-2.0 Python library for building interactive web interfaces around machine-learning functions and apps. It supports 40+ components for text, images, audio, video, 3D, chat, plots, JSON, and dataframes. Apps can be hosted free on Hugging Face Spaces, shared through public links, or extended with server-side rendering, streaming, API clients, and MCP server support.

The GitHub of ML — model hub, datasets, and inference

Open-source platform for building, sharing, and deploying machine learning models and datasets. Hosts 500k+ models, 100k+ datasets, and Spaces for interactive demos. The central hub of the open-source AI ecosystem, providing model discovery, inference APIs, and collaborative tools that make it the GitHub of machine learning for researchers and developers worldwide.

Security scanner for MCP servers against tool poisoning attacks

MCP-Scan is a security tool that scans MCP servers for vulnerabilities including tool poisoning, prompt injection, cross-origin escalation, and rug pull attacks. Acquired by Snyk in 2026, it is the first dedicated security scanner for the MCP ecosystem. It analyzes tool descriptions, permissions, and behavior patterns to detect malicious or compromised MCP servers before they can exploit AI agents.

Serverless GPU compute platform for AI inference and training

Modal is a serverless compute platform that lets developers run AI workloads on GPUs with a Python-first SDK. Functions deploy with decorators, auto-scale from zero to thousands of containers, and bill per second. It supports LLM inference, fine-tuning, batch jobs, and sandboxes, with current GPU options including B200, H200, H100, A100, L40S, A10, L4, and T4. Modal’s 2026 Series C valued the company at $4.65B.

Microsoft's distributed approximate nearest neighbor search library and billion-scale vector engine

Microsoft's distributed approximate nearest neighbor search library and online vector serving engine, utilizing space-partition trees and relative neighborhood graphs for billion-scale semantic retrieval.

Open-source post-building layer for agents — tracing, evals, and online monitoring

Judgeval is the open-source post-building layer for AI agents from Judgment Labs, providing OpenTelemetry-based tracing, hosted and custom evaluation scorers, and online behavior monitoring for LLM-powered applications. Instrument any function with a single decorator, score live production traffic against faithfulness and instruction-adherence checks, and feed real-world failures back into reinforcement learning or supervised fine-tuning loops.

Apache-2.0 Python framework for repeatable LLM, RAG, agent, MCP, and safety evaluation workflows.

DeepEval is an Apache-2.0 Python framework for evaluating LLM apps, RAG systems, agents, MCP workflows, and safety behavior with repeatable test cases. It works locally and in CI/CD, then connects to Confident AI for hosted reports, observability, red teaming, and governance when teams need shared evidence instead of ad-hoc prompt reviews and manual QA.

Turn websites into LLM-ready structured data

Firecrawl is a Y Combinator-backed API that crawls websites and converts them into clean, LLM-ready Markdown or structured JSON. Handles JavaScript rendering, pagination, sitemaps, and anti-bot measures automatically. Designed for RAG pipelines, AI agents, and data extraction workflows. Features batch crawling, scheduled scraping, webhook notifications, and custom extraction schemas. Processes content for direct ingestion into vector databases and LLM context windows.

Zig-built headless browser engineered for AI agent workloads

Open-source headless browser written in Zig for AI agents, crawling, and automation. Lightpanda omits graphical rendering, keeps DOM and JavaScript execution, exposes CDP for Puppeteer/Playwright/chromedp, and adds Agent, PandaScript, and MCP workflows. Current public benchmarks claim about 9x faster execution and 16x less memory than Chrome.

Intelligent memory layer for AI agents and assistants

Mem0 is an open-source intelligent memory layer for AI agents with 51K+ GitHub stars providing persistent, adaptive memory across sessions. It manages working, short-term, and long-term memory types, enabling personalized AI experiences that improve over time. Features automatic memory extraction from conversations, semantic search over stored memories, multi-format support, and integration with 100+ frameworks. Simple API for adding memory to any LLM-powered application or agent.

Open-source product analytics, session replay, and feature flags

PostHog is an open-source product and data tools platform for analytics, session replay, feature flags, experiments, surveys, error tracking, web analytics, data warehouse, CDP and LLM observability workflows. It suits developer-led teams that want one integrated product OS instead of many separate tools.

Run and deploy ML models via API with simple pricing

Cloud platform that lets developers run thousands of open-source and proprietary public ML models through a simple API without managing GPUs or infrastructure. Replicate hosts models for image, text, audio, and video, supports Cog-based custom deployments and private models, and now operates as a distinct Cloudflare brand with pay-by-time or input/output pricing depending on the model.

Open-source LLM observability and evaluation

Phoenix by Arize is an open-source AI observability platform for tracing, evaluating, and debugging LLM applications. It captures prompt-response pairs, retrieval context, agent tool calls, and latency data through OpenTelemetry-based instrumentation. Provides experiment tracking, dataset management, and evaluation frameworks for systematically improving AI application quality. 10K+ GitHub stars.

Open-source identity provider for self-hosted SSO and access management

Authentik is an open-source Identity Provider supporting SAML, OAuth2/OIDC, LDAP, RADIUS, and SCIM for self-hosted single sign-on. It provides customizable authentication flows, multi-factor authentication, user management, and proxy-based SSO for applications without native support. Positioned as a modern Keycloak alternative with 22K+ GitHub stars, free Open Source use, and paid Enterprise/Enterprise Plus plans.

Secure cloud sandboxes for AI agents

E2B provides secure cloud sandboxes that let AI agents execute code, run terminal commands, and interact with filesystems in isolated environments. Each sandbox spins up in ~150ms with its own OS, giving agents a safe space to run untrusted code. Supports Python, JavaScript, and any language via custom Dockerfiles. Used by AI coding assistants, data analysis agents, and code interpreters. SDK available for Python and JavaScript with a simple API for programmatic sandbox control.

Apache-2.0 Python framework for building MCP servers, clients, apps, and deployable agent tool surfaces.

FastMCP is an Apache-2.0 Python framework for building MCP servers, clients, and apps with decorators, type hints, transports, auth patterns, deployment docs, and telemetry hooks. The project now resolves to PrefectHQ/fastmcp, with Prefect Horizon available as a separate enterprise MCP gateway for identity, RBAC, audit, monitoring, and server governance.

Open-source LLM engineering platform for observability

Langfuse is an open-source LLM engineering platform with 29K+ GitHub stars for tracing, evaluating, and monitoring AI applications. Acquired by ClickHouse, it provides detailed traces of LLM calls, prompt management with versioning, dataset-based evaluation, user feedback collection, and cost tracking. Framework-agnostic with native integrations for LangChain, LlamaIndex, OpenAI SDK, and Vercel AI SDK. Offers both self-hosted deployment and a managed cloud service.

Data framework for LLM applications

Leading Python framework for building LLM-powered applications with focus on data-aware and agentic workflows. Provides tools for RAG (Retrieval-Augmented Generation), document indexing, vector store integrations, query engines, and multi-agent orchestration. 150+ data connectors for various sources. Works with OpenAI, Anthropic, local models, and more. Includes LlamaHub for community tools and LlamaCloud for managed RAG pipelines. 50K+ GitHub stars.

Fast open-source SAST with custom rules

Semgrep is an AppSec platform with a widely used open-source engine for readable code rules plus commercial SAST, supply-chain and secrets workflows. Current product positioning emphasizes AI-assisted detection, triage and remediation, CI/pull-request integration and managed governance for security teams.

Microsoft's open-source frontier voice AI for long-form multi-speaker audio

VibeVoice is Microsoft's open-source voice AI family with both TTS and speech recognition models. The TTS model generates up to 90 minutes of expressive multi-speaker audio with 4 distinct voices. VibeVoice-ASR transcribes 60-minute recordings in a single pass with speaker identification and timestamps. Built on continuous speech tokenizers at 7.5 Hz and next-token diffusion, it compresses audio 80x more efficiently than Encodec while preserving fidelity.

All-in-one self-hosted AI app with RAG, agents, and multi-user support

AnythingLLM is an open-source, privacy-first AI application that turns any document into an interactive knowledge base. It bundles document ingestion, vector storage (built-in LanceDB), RAG pipelines, AI agents, and multi-user access into a single deployable package. Supports 30+ LLM providers including OpenAI, Anthropic, Ollama, and local models. With 62K+ GitHub stars and MIT license, it runs as a desktop app or Docker container with zero configuration required out of the box.

LLM evaluation and prompt engineering platform

Braintrust is an AI observability and evaluation platform for tracing LLM applications, building datasets, running prompt/model experiments, scoring outputs and turning production feedback into regression tests. It fits teams that need repeatable quality gates for AI releases rather than one-off prompt demos.

Open-source sandboxes and SDKs for AI agents that control desktops

Open-source computer-use infrastructure for agents that need to drive desktop environments in the background. CUA includes Cua Driver, Sandbox, Run, Bench, and Verified Data across Linux, Windows, macOS, and Android, with MCP and CLI surfaces for screenshots, accessibility trees, keyboard/mouse actions, shell commands, task evaluation, and fleet execution.

Embedded vector database for multimodal AI with petabyte scale

LanceDB is an open-source embedded vector database built on the Lance columnar format for multimodal AI. It delivers near in-memory performance from disk with zero-copy architecture, supporting vector search, full-text search, and SQL. Native SDKs for Python, TypeScript, and Rust integrate with LangChain, LlamaIndex, and DuckDB. Backed by a $30M Series A, used by Harvey AI and Runway, with 18,000+ GitHub stars.

Stateful agent orchestration framework by LangChain

LangGraph is LangChain's framework for building stateful, multi-actor AI agent applications as controllable graphs. It models workflows as nodes and edges, enabling cycles, branching, and human-in-the-loop patterns that simple chains cannot express. Features built-in persistence for conversation memory, streaming support, and fault tolerance. Provides fine-grained control over execution flow while supporting single-agent and multi-agent architectures with shared or independent state.

Modular graph-based RAG pipeline using hierarchical knowledge graph community summaries

Microsoft GraphRAG is an open-source retrieval framework that transforms unstructured text into structured knowledge graphs, clusters entities hierarchically using the Leiden algorithm, and generates dataset-wide summaries alongside entity-level local search for multi-hop reasoning.

Official Python SDK for OpenAI agents

OpenAI's Python framework for building multi-agent AI applications with GPT models. Provides primitives for creating agents with tool calling, handoffs between specialized agents, guardrails for input/output validation, and tracing for observability. Supports building complex workflows where agents collaborate on tasks. Includes built-in tools for file search, code execution, and web browsing. Designed for production agent systems with structured output and error recovery patterns.

Infrastructure as Code using real programming languages — TypeScript, Python, Go, C#, Java.

Pulumi is a modern Infrastructure as Code platform that lets teams define cloud infrastructure using familiar programming languages instead of DSLs. Supports TypeScript, Python, Go, C#, Java, and YAML across major clouds, Kubernetes, and a broad Pulumi Registry with Terraform-derived provider coverage. Offers testing, IDE autocomplete, reusable components, and Pulumi Cloud state/governance features.

Application monitoring and error tracking that helps developers fix issues faster.

Sentry is the leading error tracking and performance monitoring platform for developers. Captures and aggregates errors with full stack traces, breadcrumbs, and context across 100+ platforms. Used by over 100,000 organizations. Features session replay, performance tracing, and code-level profiling. A self-hosted deployment is available under FSL-1.1-Apache-2.0.

AI agent framework for web browser automation

Browser Use is an open-source AI agent framework with 99K+ GitHub stars enabling LLMs to control web browsers via natural language. Y Combinator-backed, it lets agents navigate sites, fill forms, extract data, and complete multi-step tasks autonomously. Built on Playwright with vision-based element detection, multi-tab management, cookie persistence, and self-correcting actions. Supports OpenAI, Anthropic, and local models with a simple Python API for building custom browser agents.

AI code review with full codebase context

Greptile is a Y Combinator-backed AI code review tool that builds a semantic graph of your codebase's functions, classes, and dependencies. Unlike diff-only reviewers, it is built to catch cross-file issues, architectural drift, and convention violations. Supports GitHub, GitLab, Jira, Slack, and VS Code. Offers cloud and self-hosted VPC deployment with SOC2 Type II compliance. Used by 250+ companies including Stripe and Amazon.

UK AI Security Institute framework for LLM safety evaluations

Inspect AI is an MIT-licensed framework from the UK AI Security Institute for running large language model evaluations, including tool use, multi-turn dialogue, model-graded scoring, and reusable evaluation tasks.

Official Python SDK for building MCP servers

The official Python SDK for the Model Context Protocol, enabling developers to build MCP servers and clients with asyncio support. Provides type-safe tool definitions, resource management, and all standard MCP transports. The most popular Python package for MCP development with comprehensive documentation.

AI gateway with observability, routing, and guardrails

Portkey is an AI gateway and observability platform providing a unified API for 200+ LLM providers with intelligent routing, caching, rate limiting, and guardrails. Route requests across OpenAI, Anthropic, Google, and more with automatic failover, load balancing, and cost optimization. Features request logging, prompt management, evaluation tools, and real-time monitoring. The open-source gateway can be self-hosted; Portkey Cloud adds managed observability and team features.

Python agent framework by Pydantic team

Agent framework built on Pydantic for type-safe AI applications. Provides structured outputs, dependency injection, and multi-model support. Created by the Pydantic team, it brings the same validation and typing philosophy that made Pydantic essential for Python APIs to the world of AI agents, ensuring reliable data flow between LLMs and application logic.

Property-based API fuzz testing from OpenAPI and GraphQL schemas

Schemathesis automatically generates test cases from OpenAPI and GraphQL schemas to find crashes, validation errors, and specification violations in APIs. It uses property-based testing and fuzzing techniques to explore edge cases that manual test writing misses. CLI tool and Python library with CI/CD integration. 3.4K+ GitHub stars with support for authentication, custom checks, stateful testing, JUnit XML, and Allure reports.

Adaptive web scraping library with anti-bot evasion and smart selectors

Scrapling is a Python web scraping library that uses adaptive selectors and anti-bot evasion techniques to extract data from websites reliably. It generates selectors that survive website layout changes by understanding element context rather than relying on brittle CSS paths. Features stealth browser automation, automatic retry logic, and proxy rotation. 65K+ GitHub stars.

Open-source authentication with self-hosting

SuperTokens is an open-source authentication solution with 14K+ GitHub stars providing email/password, passwordless, social login, MFA, session management, and user management. Designed for self-hosting with Docker — full control over user data with no vendor lock-in. Pre-built UI components for React, Vue, and vanilla JS. Backend SDKs for Node.js, Python, and Go. Managed cloud option also available. Emphasizes security with rotating refresh tokens and anti-CSRF protection by default.

Self-hosted AI coding assistant

Tabby is an open-source, self-hosted AI coding assistant that serves as a privacy-focused alternative to GitHub Copilot. It provides code completion, chat-based coding assistance, and IDE integrations while running entirely on your own infrastructure with support for consumer-grade GPUs. Tabby supports multiple LLM backends and offers extensions for VS Code, JetBrains, Vim, and other editors, giving teams full control over their AI coding tools without sending code to external services.

Open-source embedding database — the AI-native way to store and query embeddings.

Chroma is an open-source embedding database designed for simplicity and developer experience. Runs in-memory, as a Python library, or as a client-server deployment. Popular for prototyping RAG applications, local development, and lightweight vector search. Integrates natively with LangChain, LlamaIndex, and OpenAI.

Zero-instrumentation Kubernetes observability powered by eBPF

Coroot is an open-source observability platform that uses eBPF to automatically instrument Kubernetes applications without code changes. It provides application maps, latency analysis, log correlation, and continuous profiling with automatic anomaly detection. Replaces the need for manual instrumentation with agents that capture metrics, traces, and logs at the kernel level.

Modern data orchestration for ML and analytics

Dagster is an open-source data orchestration platform with 15K+ GitHub stars combining pipeline scheduling with software-defined assets, built-in data quality checks, and a modern developer experience. Defines data assets declaratively rather than imperatively. Features asset lineage visualization, partitioned processing, sensor-based triggers, comprehensive testing, and integrated observability. A modern alternative to Airflow for teams wanting asset-centric orchestration.

Automated code quality platform with AI autofix

DeepSource is a code quality, security, and AI review platform for repositories across GitHub, GitLab, Bitbucket, and Azure DevOps. It combines static analysis, SCA, coverage, license compliance, quality gates, Autofix, and AI Review. Team is listed at $24/user/month yearly; Open Source is limited to public repositories with 1,000 PR reviews/month, while AI Review/Autofix use credits or pay-as-you-go.

Distributed search and analytics engine for all types of data.

Elasticsearch is the world's most popular open-source search and analytics engine, powering search experiences for companies like Wikipedia, GitHub, Netflix, and Uber. Built on Apache Lucene, it provides near-real-time search, structured and unstructured data analysis, and machine learning capabilities. Part of the Elastic Stack (ELK), it handles log analytics, application search, security analytics, and observability at scale. Supports vector search for AI/RAG applications.