# llama

Wafer-scale inference at thousands of tokens per second

Cerebras Inference serves open-weight LLMs like Llama, Qwen, and GPT-OSS on wafer-scale CS-3 chips through an OpenAI-compatible API, benchmarking between 1,800 and 2,600 output tokens per second on Llama 3.1 8B and several hundred on 70B models. A free tier offers one million tokens per day with no credit card, while paid pay-per-token pricing starts at $0.04 per million tokens for the smaller Llama models.

Managed foundation models on AWS

Fully managed AWS service providing enterprise access to 100+ foundation models from Anthropic, Meta, Mistral, Cohere, and Amazon's Nova family through a single API. Bedrock includes AgentCore for agent runtime, Knowledge Bases for RAG, Guardrails blocking 88% of harmful content, plus Model Distillation, Prompt Caching, and Intelligent Prompt Routing for cost optimization.

Production-grade inference with serverless and on-demand GPUs

High-performance inference platform serving open-source and custom AI models at global scale, processing 13+ trillion tokens daily at ~180K requests per second. Fireworks AI delivers 1,000+ tokens per second on large models through quantization-aware tuning and adaptive speculation, with serverless, fine-tuning, and dedicated GPU options across text, image, and audio modalities.

Ultra-fast LPU inference with fastest token generation

AI inference company building the Language Processing Unit (LPU), purpose-built silicon that delivers the fastest LLM token generation speeds available. GroqCloud serves popular open-source models like Llama at 300+ tokens per second with sub-millisecond latency — roughly 10x faster than NVIDIA H100 GPU clusters — through a simple API without infrastructure management.

Fast inference platform for open-source models

Cloud platform for running, fine-tuning, and training open-source AI models with optimized inference speeds up to 4x faster than traditional deployments. Together AI supports serverless endpoints and dedicated GPUs, fine-tuning of 100B+ parameter models like DeepSeek-V3 and Qwen3-235B, plus async batch processing scaling to 30B tokens for cost-effective large workloads.

Run LLMs locally with one command

Tool for running large language models locally on your machine with a simple CLI interface. Download and run Llama 3, Mistral, Gemma, Phi, Code Llama, and dozens of other open-source models with a single command. Features model management, GPU acceleration (NVIDIA/AMD/Apple Silicon), OpenAI-compatible API server, Modelfile for customization, and multi-model switching. Ideal for offline AI development, privacy-sensitive use cases, and local testing. 120K+ GitHub stars.