# Embeddings

The GitHub of ML — model hub, datasets, and inference

Open-source platform for building, sharing, and deploying machine learning models and datasets. Hosts 500k+ models, 100k+ datasets, and Spaces for interactive demos. The central hub of the open-source AI ecosystem, providing model discovery, inference APIs, and collaborative tools that make it the GitHub of machine learning for researchers and developers worldwide.

API for GPT-5 family models, multimodal generation, embeddings, and agents

Official API platform for the GPT-5 family, reasoning/thinking variants, multimodal generation, speech, embeddings, and agent workflows. Features the Responses API, tool calling, structured outputs, batch processing, fine-tuning, and SDK support. It remains one of the most widely integrated AI APIs in the developer ecosystem, but model choice, retention settings, rate limits, and pricing tiers require active governance in production.

Data framework for LLM applications

Leading Python framework for building LLM-powered applications with focus on data-aware and agentic workflows. Provides tools for RAG (Retrieval-Augmented Generation), document indexing, vector store integrations, query engines, and multi-agent orchestration. 150+ data connectors for various sources. Works with OpenAI, Anthropic, local models, and more. Includes LlamaHub for community tools and LlamaCloud for managed RAG pipelines. 50K+ GitHub stars.

High-recall Postgres vector search at billion scale

VectorChord is a Postgres extension from the supervc-stack/VectorChord project that brings high-recall vector search to PostgreSQL. As the spiritual successor to pgvecto.rs, it combines IVF indexes with RaBitQ quantization to deliver Pinecone-class performance at billion-vector scale while keeping all data inside a single Postgres database — no separate vector store, no two-system sync, no rewrites when the workload grows.

AI-native database for hybrid RAG retrieval

Infinity is an AI-native database from InfiniFlow that unifies dense vectors, sparse vectors, tensors, and full-text search in a single engine. Built for retrieval-augmented generation (RAG) at scale, it powers hybrid search workflows where lexical matching, semantic similarity, and reranking all happen against one storage layer instead of four loosely coupled services.

DiskANN-powered vector search extension for PostgreSQL

pgvectorscale is an open-source PostgreSQL extension from Timescale that complements pgvector with DiskANN-based approximate vector search. It is useful for teams that want faster embedding retrieval while keeping vectors, filters, and application data inside the Postgres ecosystem instead of adopting a separate hosted vector database.

Framework for LLM applications

The most widely-used framework for building LLM-powered applications, available in Python and JavaScript. Provides abstractions for chains, agents, RAG, memory, tool usage, and structured output. Integrates with 100+ LLM providers, vector stores, document loaders, and tools. LangSmith offers tracing and evaluation. LangGraph enables stateful, multi-agent workflows with cycles. 100K+ GitHub stars. The de facto standard for LLM application development despite growing alternatives like LlamaIndex.

Unified API proxy for 100+ LLMs

Drop-in OpenAI-compatible proxy supporting 100+ LLM providers with load balancing, spend tracking, rate limiting, and fallback routing. Acts as a unified gateway for all your AI model calls, letting teams switch between providers, enforce budgets, and add reliability layers without changing application code. Essential infrastructure for multi-model AI architectures.

Library for efficient similarity search and clustering of dense vectors at billion-scale.

FAISS is Meta AI Research's open-source library for efficient similarity search and clustering of dense vectors. It implements approximate nearest-neighbor algorithms designed to scale to billions of vectors, with optimized indexes that fit in RAM and GPU acceleration for the largest workloads. Engineering teams use FAISS as the retrieval primitive underneath custom RAG pipelines, recommendation systems, and large-scale embedding search infrastructure.

Serverless vector database with pay-as-you-go API pricing

Upstash Vector is a managed serverless vector database for RAG, semantic search, and embedding lookup. It is separate from the existing Upstash platform record in the aicoolies catalog: this slug covers the Vector product line, not the broader Redis, Kafka, or QStash platform.

Edge-native vector database for Workers and AI applications

Cloudflare Vectorize is Cloudflare’s managed vector database for Workers and edge AI applications. It is distinct from the existing Cloudflare Workers tool page: Workers is the compute runtime, while Vectorize is the embedding index and vector-query layer used to add semantic retrieval to Cloudflare-hosted apps.

Enterprise AI for text generation, search, and RAG

Enterprise-focused AI platform from former Google Brain researchers offering Command (chat), Embed (semantic search), and Rerank (result ordering) model families. Cohere Embed v4 supports 100+ languages with multimodal text/image inputs, North agent workspace processes documents and spreadsheets, and Model Vault enables secure VPC or on-premises deployment for regulated enterprises.

AI data runtime for multimodal datasets and vector search

Deep Lake is an open-source AI data runtime from Activeloop for storing, versioning, and querying multimodal data and embeddings. It fits teams building RAG, training, evaluation, or dataset-heavy agent workflows that need a bridge between vector search, structured metadata, and large image, text, audio, or video collections.

Open-source toolkit for curating datasets and evaluating visual AI models

FiftyOne is an open-source Python toolkit from Voxel51 for building high-quality datasets and better computer-vision and multimodal AI models. It pairs a browser-based visualization App with programmatic dataset curation, embeddings, similarity search, and model-evaluation workflows.

Embedding-first search and discovery engine for AI-powered product experiences.

Marqo is an open-source tensor search engine that combines embedding generation and vector search in a single API, removing the need to manage separate embedding pipelines and vector databases. Built for product discovery and multi-modal search, it lets teams index text, images, and structured data together, returning ranked results based on semantic similarity rather than keyword overlap.

All-in-one multimodal RAG framework

RAG-Anything is an all-in-one multimodal RAG framework from the University of Hong Kong that processes text, images, tables, and equations through a unified pipeline built on LightRAG. It constructs multi-modal knowledge graphs by extracting multimodal entities and establishing cross-modal relationships. The VLM-Enhanced Query mode integrates visual content into large language models for deeper document understanding beyond plain text retrieval.

Build modular, scalable LLM applications in Rust

Open-source Rust library for building scalable, modular, and ergonomic LLM-powered applications. Rig unifies 20+ model providers (OpenAI, Anthropic, Mistral, DeepSeek, Ollama, and more) and 10+ vector stores behind one trait-based interface, supports completion and embedding workflows, multi-turn streaming, and transcription/audio/image generation, with full GenAI Semantic Convention compatibility and WASM-ready core library — production agentic infra for Rust teams.

Hugging Face's open-source inference server for embeddings, rerankers, and classifiers

Text Embeddings Inference is Hugging Face's Apache-2.0 server for high-throughput embedding, reranking, and sequence-classification models. TEI packages token-based dynamic batching, optimized Transformers kernels, Safetensors loading, OpenAI-compatible embedding endpoints, Prometheus metrics, and configurable OpenTelemetry tracing in deployable CPU and GPU images.

Fast embeddable vector search engine

USearch is a high-performance vector search engine implementing HNSW algorithms for approximate nearest neighbor queries across C++, Python, JavaScript, Rust, Java, Go, and more. It supports user-defined distance metrics, memory-mapped persistence for datasets larger than RAM, and filtered search with predicates. Used by YugabyteDB and ScyllaDB as their production vector indexing backend.

Header-only C++ implementation of HNSW for fast approximate nearest-neighbor search.

hnswlib is a header-only C++ library implementing the Hierarchical Navigable Small World (HNSW) graph algorithm for approximate nearest-neighbor search, with Python bindings and a tiny dependency footprint. Originally developed by the nmslib team, it has become the default HNSW implementation embedded inside many vector databases and search products. Engineers use it directly when they want HNSW retrieval without pulling in a heavyweight vector DB.

Vector search extension for SQLite that runs anywhere

sqlite-vec is a lightweight vector search extension for SQLite written in pure C with zero dependencies. It brings nearest-neighbor search capabilities directly into SQLite databases, enabling AI applications to store and query embeddings without running a separate vector database. The extension works everywhere SQLite runs including Linux, macOS, Windows, WebAssembly in browsers, and even Raspberry Pi devices. Sponsored by Mozilla Builders, Fly.io, and Turso.