gemma.cpp

Lightweight C++ inference for Google Gemma models

gemma.cpp is Google's standalone C++ inference engine built specifically for running Gemma language models without Python or CUDA dependencies. It provides optimized CPU inference using SIMD instructions and Highway library, supports Gemma 2 and Gemma 3 models, and runs on x86 and ARM architectures. Designed for embedded systems, edge devices, and server deployments needing minimal overhead.

Rate

Website

Is this your product? →

About gemma.cpp

gemma.cpp is Google DeepMind's purpose-built inference engine that strips away the overhead of Python runtimes and heavy ML frameworks to run Gemma models with maximum efficiency on CPU hardware. Unlike general-purpose inference engines like llama.cpp that support many model architectures, gemma.cpp is optimized specifically for the Gemma model family, enabling architecture-specific optimizations that would not be possible in a generic framework. The result is faster inference with lower memory usage for Gemma-specific deployments.

The engine leverages Google's Highway library for portable SIMD operations, automatically selecting the best instruction set available on the target CPU — AVX-512, AVX2, SSE4, or NEON for ARM. This makes it suitable for deployment across x86 servers, Apple Silicon Macs, Raspberry Pi devices, and other ARM hardware without code changes. gemma.cpp supports the complete Gemma model family including Gemma 2 and Gemma 3 variants, with quantized model formats that reduce memory requirements while preserving quality.

With over 6,800 GitHub stars and Google's direct maintenance, gemma.cpp serves developers who need to deploy Gemma models in environments where Python is unavailable, undesirable, or too slow. Use cases include embedded systems, mobile applications via native code, IoT edge devices, and high-throughput server deployments. The Apache-2.0 license and Google's active development ensure the engine stays current with new Gemma model releases and architectural improvements.

Pricing & Platform Specs

Pricing Summary

100% free and open source under the Apache-2.0 license ($0 software cost). Minimal, zero-dependency C++ inference engine with Google Highway portable SIMD acceleration for Google Gemma and Gemma 2 models.

full pricing breakdown →

Supported Platforms

x86 and ARM CPUs — no Python or CUDA required

Explore categories, tags & use cases

Use Cases

AI Model Training Mobile Development

Alternatives

All gemma.cpp alternatives →

llama.cpp

High-performance local LLM inference in C/C++

llama.cpp is the foundational C/C++ library with 75K+ GitHub stars powering local LLM inference on consumer hardware. Provides optimized CPU and GPU inference for quantized models in GGUF format. Supports LLaMA, Mistral, Phi, Gemma, and most open-weight families. Features 2-8 bit quantization for reduced memory, multi-GPU support, context extension, grammar-constrained output, and an OpenAI-compatible API server. The engine behind Ollama and LM Studio.

Open Source

Ollama

Run LLMs locally with one command

Tool for running large language models locally on your machine with a simple CLI interface. Download and run Llama 3, Mistral, Gemma, Phi, Code Llama, and dozens of other open-source models with a single command. Features model management, GPU acceleration (NVIDIA/AMD/Apple Silicon), OpenAI-compatible API server, Modelfile for customization, and multi-model switching. Ideal for offline AI development, privacy-sensitive use cases, and local testing. 120K+ GitHub stars.

Open Source

Discover more tools in this category

Related Tools

computed discovery: shared active categories

Ray

Distributed AI compute engine for scaling Python and ML workloads

Ray is an open-source distributed computing framework built for scaling AI and Python applications from a laptop to thousands of GPUs. It provides libraries for distributed training, hyperparameter tuning, model serving, reinforcement learning, and data processing under a single unified API. Ray's public site highlights OpenAI and other enterprise users. Maintained by Anyscale with Apache-2.0 open-source licensing.

freemiumOpen Source

LLaMA-Factory

Unified framework for fine-tuning 100+ large language models

LLaMA-Factory is an open-source toolkit providing a unified interface for fine-tuning over 100 LLMs and vision-language models. It supports SFT, RLHF with PPO and DPO, LoRA and QLoRA for memory-efficient training, and continuous pre-training. The LLaMA Board web UI enables no-code configuration, while CLI and YAML workflows serve advanced users. Integrates with Hugging Face, ModelScope, vLLM, and SGLang for model deployment.

Open Source

Tavily

Real-time search API built for AI agents

Tavily is an AI-native search API that provides real-time web search, content extraction, and crawling capabilities specifically designed for LLM applications and autonomous agents. It returns structured, citation-ready results optimized for RAG workflows with built-in safety features including prompt injection protection and PII leak prevention. Acquired by Nebius in 2026, Tavily integrates with LangChain, LlamaIndex, and major agent frameworks, serving over one million developers worldwide.

freemium

Firecrawl

Turn websites into LLM-ready structured data

Firecrawl is a Y Combinator-backed API that crawls websites and converts them into clean, LLM-ready Markdown or structured JSON. Handles JavaScript rendering, pagination, sitemaps, and anti-bot measures automatically. Designed for RAG pipelines, AI agents, and data extraction workflows. Features batch crawling, scheduled scraping, webhook notifications, and custom extraction schemas. Processes content for direct ingestion into vector databases and LLM context windows.

freemiumOpen Source

Unsloth

2x faster LLM fine-tuning with 70% less VRAM on a single GPU

Unsloth is an open-source framework for fine-tuning large language models up to 2x faster while using 70% less VRAM. Built with custom Triton kernels, it supports 500+ model architectures including Llama 4, Qwen 3, and DeepSeek on consumer NVIDIA GPUs. Unsloth Studio adds a no-code web UI for dataset creation, training observability, model comparison, and GGUF export for Ollama and vLLM deployment.

Open Source

VibeVoice

Microsoft's open-source frontier voice AI for long-form multi-speaker audio

VibeVoice is Microsoft's open-source voice AI family with both TTS and speech recognition models. The TTS model generates up to 90 minutes of expressive multi-speaker audio with 4 distinct voices. VibeVoice-ASR transcribes 60-minute recordings in a single pass with speaker identification and timestamps. Built on continuous speech tokenizers at 7.5 Hz and next-token diffusion, it compresses audio 80x more efficiently than Encodec while preserving fidelity.

Open Source

Community experience

Sources & verification

gemma.cpp Official Repository & Documentation

Sources checked: September 5, 2026
Content verified: September 6, 2026

Verification dates are editorial checks. Routine CMS saves and automatic updatedAt timestamps do not advance them.

FAQ

What is gemma.cpp?

Is gemma.cpp free?

Yes — gemma.cpp is open source and free to use. 100% free and open source under the Apache-2.0 license ($0 software cost). Minimal, zero-dependency C++ inference engine with Google Highway portable SIMD acceleration for Google Gemma and Gemma 2 models.

Is gemma.cpp open source?

Yes — gemma.cpp is open source.

Is gemma.cpp still maintained?

Yes — gemma.cpp is active. Its listing was last verified on September 6, 2026.

What are the best gemma.cpp alternatives?

The first editor-selected gemma.cpp alternatives are llama.cpp, Ollama.

gemma.cpp

About gemma.cpp

Pricing & Platform Specs

Pricing Summary

Supported Platforms

Categories

Tags

Use Cases

Alternatives

llama.cpp

Ollama