Text Embeddings Inference, usually shortened to TEI, is Hugging Face's open-source serving engine for embedding, reranking and sequence-classification workloads. It is deliberately separate from Text Generation Inference: TEI turns text into vectors or relevance scores for retrieval, semantic search, clustering, recommendations and RAG pipelines, while TGI focuses on generating text. Official documentation lists support across BERT-family models, Nomic, E5, GTE, Qwen, ModernBERT, Gemma and several reranker families. The server can expose OpenAI-compatible embedding endpoints, load private or gated Hub models with an HF token, and run from published Docker images or local builds.
TEI's production value comes from its serving controls rather than a claim that one embedding model is universally best. It uses token-based dynamic batching, Safetensors weight loading, optimized Transformers code, Flash Attention, Candle and cuBLASLt where supported. Operators can set maximum concurrent requests, batch-token budgets, request size, input truncation, pooling strategy, API-key authorization and model revision. Prometheus metrics and optional OpenTelemetry export support observability without requiring a Hugging Face hosted endpoint. Current hardware docs include CPU images and NVIDIA generations from Turing through Blackwell, with explicit caveats: Volta-class CUDA devices are unsupported and some newer architecture images remain experimental.
TEI is a strong fit when a team wants a self-hosted, repeatable embedding or reranking service that integrates cleanly with Hugging Face model artifacts. The Apache-2.0 engine is free, but the organization still pays for compute and must comply with each selected model's license. Hugging Face Inference Endpoints can run TEI as a separate managed, usage-priced service; those endpoint charges are not a license fee for TEI itself. Teams should compare TEI with general serving engines such as vLLM or Ollama when they need mixed generation workloads, and with Text Generation Inference when the primary requirement is token generation rather than vector extraction or reranking.
