aicoolies logo
DVC logo

DVC

Git-based version control for ML data and pipelines

Share
open-sourceOpen Source
Visit Website →

DVC (Data Version Control) is a free open-source tool that brings Git-like version control to datasets, ML models, and experiment pipelines. It stores pointer files in Git while keeping large data in remote storage like S3, GCS, or Azure. Features include reproducible ML pipelines with DAG-based dependency tracking, experiment management, metrics comparison, and a VS Code extension for visual experiment tracking.

DVC extends Git workflows to handle the unique challenges of machine learning projects — tracking large datasets, versioning model artifacts, and reproducing experiments. With over 15,000 GitHub stars, DVC has become a standard tool for data scientists who want reproducibility without abandoning their existing Git workflows. It works by storing lightweight .dvc pointer files in your Git repository while the actual data lives in configurable remote storage backends including Amazon S3, Google Cloud Storage, Azure Blob, SSH servers, and local network drives.

The pipeline system lets teams define multi-stage ML workflows in dvc.yaml files, creating directed acyclic graphs of dependencies between data, code, and outputs. Running dvc repro intelligently re-executes only the stages affected by changes, saving significant compute time. The experiment tracking system enables comparing parameters, metrics, and plots across runs without leaving the terminal or VS Code, making it easy to iterate on model development and share findings with teammates.

Originally created by Iterative.ai, DVC was acquired by lakeFS in November 2025, uniting two data version control pioneers. DVC remains free and open-source under Apache 2.0, with the lakeFS platform providing enterprise-scale data versioning for teams needing petabyte-level multimodal object store management. DVC supports any programming language and ML framework, integrating with Python, R, Julia, PyTorch, TensorFlow, and CI/CD systems for fully automated MLOps workflows.

Pricing

Free and open-source (Apache 2.0); lakeFS Enterprise available

Platforms

CLI + VS Code extension — Linux, macOS, Windows

Categories

Tags

Use Cases

Alternatives

Related Tools

Deep Lake logo

Deep Lake

AI data runtime for multimodal datasets and vector search

Deep Lake is an open-source AI data runtime from Activeloop for storing, versioning, and querying multimodal data and embeddings. It fits teams building RAG, training, evaluation, or dataset-heavy agent workflows that need a bridge between vector search, structured metadata, and large image, text, audio, or video collections.

open-sourceOpen Source
SeekDB logo

SeekDB

AI-native state store with hybrid vector and full-text search

SeekDB is an open-source AI-native state store from the OceanBase ecosystem that combines MySQL-compatible data access with hybrid vector and full-text retrieval. It targets agent and AI application teams that need embedded or server deployment, copy-on-write style sandboxes, and searchable state without gluing together several separate storage layers.

open-sourceOpen Source
Marqo logo

Marqo

Embedding-first search and discovery engine for AI-powered product experiences.

Marqo is an open-source tensor search engine that combines embedding generation and vector search in a single API, removing the need to manage separate embedding pipelines and vector databases. Built for product discovery and multi-modal search, it lets teams index text, images, and structured data together, returning ranked results based on semantic similarity rather than keyword overlap.

freemium
Magika logo

Magika

AI-powered file-type detection at Google scale

Open-source AI-powered file-type detection tool from Google that uses a custom deep-learning model under a few megabytes to identify more than 200 binary and textual content types in milliseconds, even on a single CPU. Magika ships as a CLI, Python package, JavaScript/TypeScript library, and an ONNX model, achieves around 99% accuracy on its test set, and is already used at Google scale across Gmail, Drive, and Safe Browsing as well as by VirusTotal and abuse.ch.

freeOpen Source
Zep logo

Zep

Context engineering platform for AI agents with temporal knowledge graphs

Zep is a context engineering platform that assembles relationship-aware context for AI agents from conversations, business data, documents, and events. It maintains a temporal knowledge graph that automatically extracts entities and relationships, tracking how context evolves over time. Zep delivers formatted context blocks optimized for LLMs with sub-200ms latency, integrating with LangChain, LlamaIndex, AutoGen, and Google ADK through Python, TypeScript, and Go SDKs.

freemium
Hindsight logo

Hindsight

Agent memory system that learns, not just remembers

Hindsight is an agent memory system that enables AI agents to learn from experience rather than just store conversations. It organizes memories into three biomimetic categories: World knowledge for facts, Experiences for agent events, and Mental Models for learned understanding. The system provides retain, recall, and reflect operations backed by a temporal knowledge graph with parallel retrieval strategies including semantic, keyword, graph traversal, and temporal search.

freemiumOpen Source

Comparisons