LLM Compressor is an open-source library (maintained under the vllm-project org, with Red Hat / Neural Magic behind it) for shrinking large language models so they run cheaper and faster at inference time. It offers a broad set of quantization algorithms and transforms — weight, activation, KV-cache, and attention quantization across precisions like W8A8 (int8/fp8), W4A16, and microscale formats (NVFP4, MXFP4, MXFP8) — plus structural techniques such as REAP expert pruning for Mixture-of-Experts models. It integrates directly with Hugging Face models and saves in the compressed-tensors format that vLLM loads natively, and supports DDP and disk offloading to compress very large models. Vendor framing (labeled): the project cites checkpoints (e.g., large MoE models) compressed with ">70% size reduction while maintaining accuracy recovery" — these are producer-reported results on specific models/benchmarks; expected accuracy/size trade-offs must be validated per model and per task. Positioning note: this is a pre-deployment optimization pipeline that produces artifacts consumed by the vLLM engine — it is not the vLLM engine itself, nor vllm-production-stack. Best fit for teams self-hosting open-weight models who want to fit larger models on fewer/cheaper GPUs. Requires Python, a calibration dataset, and GPU capacity for the compression run.
LLM Compressor
Quantize and compress LLMs for optimized vLLM deployment
Open-source, Hugging Face-compatible library for quantizing and compressing LLMs (W8A8, W4A16, NVFP4/MXFP4, KV-cache and attention quantization, expert pruning) into vLLM-ready `compressed-tensors` checkpoints.
Pricing
Free and open-source (Apache-2.0). No paid tier; only your own GPU/compute for the compression run.
Platforms
Python library (`pip install llmcompressor`). Linux + GPU for calibration/compression. Integrates with Hugging Face Transformers; outputs `compressed-tensors` checkpoints for vLLM. DDP + disk-offload for very large models.
Categories
Tags
Alternatives
Related Tools
LMDeploy
Open-source toolkit for quantizing, deploying, and serving LLMs and vision-language models
LMDeploy is an Apache-2.0 toolkit for self-hosting LLM and vision-language model inference with TurboMind and PyTorch engines. It combines continuous batching, blocked KV cache, tensor parallelism, AWQ and KV-cache quantization with OpenAI-compatible APIs, multi-GPU distribution, offline pipelines, and production metrics.
OpenMeter
Real-time usage metering and billing for AI, API, and DevOps products
OpenMeter is an Apache-2.0 open-source metering platform that ingests usage events, aggregates them into meters, and supports limits, entitlements, and usage-based billing. It fits AI, API, and DevOps products that need token-level reporting, customer usage visibility, or finance-ready chargeback data. Teams can self-host the core platform or evaluate OpenMeter's managed offering.
Flexprice
Usage metering and billing infrastructure for AI, API, and SaaS products
Flexprice is an AGPL-3.0 open-source platform for real-time usage metering, usage-based pricing, credits, entitlements, and billing workflows. It helps engineering and finance teams turn token, API, and feature events into billable usage across managed-cloud or self-hosted deployments. Use it when a product needs finance-grade chargeback and customer usage controls, not only LLM traces.
MCP Context Forge
IBM-backed ContextForge gateway for federating MCP, A2A, REST, and gRPC APIs
MCP Context Forge is IBM’s Apache-2.0 ContextForge project for operating a gateway, registry, and proxy across MCP servers, A2A agents, REST APIs, and gRPC services. It centralizes discovery, authentication, policy controls, federation, and observability, with deployment paths through PyPI, Docker, and Kubernetes.
Mergify
Merge queue, CI insights, flaky-test controls, and stacked pull requests for GitHub teams
Mergify is a pull request automation platform that keeps main branches green with merge queue batching, merge protections, CI Insights, flaky-test detection, and stacked pull requests. Its Stacks workflow turns commits on one local branch into focused PR chains, helping teams review large AI-generated or feature-heavy changes without losing queue safety.
Terrateam
Open-source GitOps automation for Terraform and OpenTofu pull requests
Terrateam is open-source GitOps infrastructure orchestration for Terraform and OpenTofu pull requests. It automates plans and applies in GitHub workflows, supports monorepos and many workspaces, and adds apply-only locks, OPA/Rego policy checks, cost and drift signals, and approval controls without forcing teams into a separate IaC platform.