OpenRouter

Open-weight inference, fine-tuning, and GPU-cloud platform

Together AI is a cloud platform for running, fine-tuning, batching, and training open-weight AI models. It supports serverless inference, dedicated endpoints, LoRA and full fine-tuning, GPU clusters, code-execution sandboxes, and async batch jobs up to 30B tokens per model. Current docs list fast-moving families such as Qwen, Kimi, GLM, GPT-OSS, DeepSeek, Llama, MiniMax, and Mistral.

Production-grade inference with serverless and on-demand GPUs

High-performance inference platform serving open-source and custom AI models at global scale, processing 13+ trillion tokens daily at ~180K requests per second. Fireworks AI delivers 1,000+ tokens per second on large models through quantization-aware tuning and adaptive speculation, with serverless, fine-tuning, and dedicated GPU options across text, image, and audio modalities.

Managed foundation models on AWS

Fully managed AWS service providing enterprise access to 100+ foundation models from Anthropic, Meta, Mistral, Cohere, and Amazon's Nova family through a single API. Bedrock includes AgentCore for agent runtime, Knowledge Bases for RAG, Guardrails blocking 88% of harmful content, plus Model Distillation, Prompt Caching, and Intelligent Prompt Routing for cost optimization.

Open-source LLM gateway with built-in optimization and A/B testing

TensorZero is an open-source LLMOps platform in Rust that unifies an LLM gateway, observability, prompt optimization, and A/B experimentation in a single binary. It routes requests across providers with sub-millisecond P99 latency at 10K+ QPS while capturing structured data for continuous improvement. Supports dynamic in-context learning, fine-tuning workflows, and production feedback loops. Backed by $7.3M seed funding, 11K+ GitHub stars.

Smart LLM router that cuts inference costs up to 70%

Manifest is an open-source smart model router that intelligently routes LLM requests to the cheapest capable model, reducing inference costs by up to 70% without sacrificing output quality. It uses a 23-dimension scoring algorithm to evaluate 300+ models across providers including OpenAI, Anthropic, Google, and DeepSeek, with automatic fallbacks and budget controls. Manifest can be deployed as a cloud service, local plugin, or self-hosted Docker container with transparent routing logic.

Heterogeneous CPU-GPU inference and SFT for large MoE models

Open-source framework for running and fine-tuning large Mixture-of-Experts models with heterogeneous CPU-GPU execution, optimized kernels, limited VRAM and SGLang or LLaMA-Factory integrations.

Hugging Face's open-source inference server for embeddings, rerankers, and classifiers

Text Embeddings Inference is Hugging Face's Apache-2.0 server for high-throughput embedding, reranking, and sequence-classification models. TEI packages token-based dynamic batching, optimized Transformers kernels, Safetensors loading, OpenAI-compatible embedding endpoints, Prometheus metrics, and configurable OpenTelemetry tracing in deployable CPU and GPU images.

Open-source toolkit for quantizing, deploying, and serving LLMs and vision-language models

LMDeploy is an Apache-2.0 toolkit for self-hosting LLM and vision-language model inference with TurboMind and PyTorch engines. It combines continuous batching, blocked KV cache, tensor parallelism, AWQ and KV-cache quantization with OpenAI-compatible APIs, multi-GPU distribution, offline pipelines, and production metrics.

Multi-agent model API that orchestrates frontier models behind one OpenAI-compatible endpoint

Sakana Fugu is a hosted model-provider API that exposes a learned multi-agent system as one OpenAI-compatible model. It dynamically routes coding, code review, research, and reasoning tasks across a frontier-model pool, with Fugu for lower-latency work and Fugu Ultra for harder workloads where answer quality matters more than cost or speed.

Lifelike AI voice generation, cloning, and voice agents

ElevenLabs is an AI voice platform for text-to-speech, voice cloning, and conversational AI agents, built on models like Multilingual v2 and the low-latency Flash v2.5 and Turbo v2.5. Developers call its API to generate lifelike narration, clone voices from short audio samples, dub content across 30+ languages, add sound effects, and deploy real-time voice agents for customer service, IVR, and interactive apps, with SDKs for Python, JavaScript, and more.

Official Python SDK for the xAI API

The xAI Python SDK is the official Python client for the xAI API, giving developers a direct way to build Grok-powered apps without relying on community proxies or unofficial wrappers. It supports synchronous and asynchronous Python clients for chat completions, streaming responses, function/tool calling, and multimodal workflows, making it a clean fit for backend services, agents, notebooks, and developer tools that need programmatic xAI access.