Best tools for Local AI Workflows

Google's lightweight ML framework for mobile and embedded

TensorFlow Lite is Google's lightweight ML framework for deploying models on mobile and embedded devices. It supports quantization, GPU/NPU delegation, and runs on Android, iOS, Linux, and microcontrollers. Provides pre-trained models, model conversion tools from TensorFlow and JAX, and hardware acceleration via GPU, Hexagon DSP, and CoreML delegates. Powers on-device ML in billions of Google app installations.

AI-powered flaky test detection for Playwright and CI

TestDino is an AI-powered platform for detecting and managing flaky tests, with deep Playwright integration. It uses ML to analyze CI test results, classify failure root causes like network timeouts and race conditions, and provides an MCP server for conversational CI debugging. Auto-groups failures by cause and tracks flakiness trends across test suites.

Free local AI code completion for VSCode

Twinny is a free, open-source AI code completion extension for VS Code that works with any OpenAI API-compatible endpoint including local models via Ollama. It provides real-time inline suggestions, a sidebar chat for code discussion, workspace embeddings for context-aware completions, and a decentralized P2P inference network. A privacy-first alternative to GitHub Copilot with zero licensing cost.

One vector-search extension across PostgreSQL, DuckDB and SQLite

MIT-licensed vector-search extension for PostgreSQL, DuckDB and SQLite that shares one graph-index core with PQ/RaBitQ quantization, persistent indexes and metadata filtering; SQLite packages cover Linux, macOS, iOS, Android and WASM, so it runs inside existing databases instead of as a separate vector service.

Offline speech recognition for 20+ languages

Vosk is an offline speech recognition toolkit supporting 20+ languages with compact 50MB models that run on Raspberry Pi, Android, iOS, and servers. It provides streaming API with zero-latency response, speaker identification, and reconfigurable vocabulary. Vosk offers bindings for Python, Java, Node.js, C#, Go, and Rust. Unlike cloud-based alternatives, all processing happens locally with no internet required. Apache 2.0 licensed with 14K+ GitHub stars.

Tokenizer-free multilingual TTS with voice cloning

VoxCPM is an open-source text-to-speech system from OpenBMB generating continuous speech across 30 languages without traditional tokenization. Its 2B parameter end-to-end diffusion architecture produces 48kHz studio-quality audio with natural prosody and emotion. Key capabilities include voice design from text descriptions, few-shot voice cloning, and multilingual synthesis without language-specific modules. The Apache 2.0 project has 8,700 GitHub stars.

OpenAI's open-source speech recognition model for any language

Whisper is OpenAI's open-source automatic speech recognition model trained on 680,000 hours of multilingual audio data. It supports transcription and translation across 99 languages with robust handling of accents, background noise, and technical vocabulary. Available in multiple model sizes from tiny (39M) to large (1.5B parameters) for balancing accuracy and speed.

Sentry-maintained MCP server and CLI for Xcode builds, simulators, and tests

XcodeBuildMCP is a Sentry-maintained, MIT-licensed MCP server and CLI for agent-assisted iOS and macOS development. It lets MCP-compatible coding agents run Xcode build and test workflows, manage simulators, inspect failures, and work through Homebrew, npm, or on-demand client configuration, with documented Sentry telemetry controls for teams that need an opt-out.

Local model inference engine with OpenAI-compatible API and web UI

Xinference is a local inference engine that runs LLMs, embedding models, image generation, and audio models with an OpenAI-compatible API. It provides a web dashboard for model management, supports vLLM, llama.cpp, and transformers backends, and handles multi-GPU deployment automatically. Supports 100+ models including Qwen, Llama, Mistral, and DeepSeek with over 9,200 GitHub stars.

Persistent memory layer for AI coding agents — keeps Claude Code, Codex, Cursor, and any MCP agent in context across sessions

agentmemory is an open-source MCP server that gives AI coding agents persistent, cross-session memory. Built on hybrid vector-graph search, it achieves 95.2% recall on the LongMemEval-S benchmark while using up to 92% fewer context tokens than naive context injection. Works out of the box with Claude Code, Codex, Cursor, Windsurf, Cline, OpenCode, Kilo Code, Hermes, and any MCP client through 51 MCP tools plus 12 hooks and 4 skills.

One-command local coding agent that auto-detects your hardware and picks the best model

hf-agents is a Hugging Face CLI extension that detects your hardware, recommends the best GGUF model using llmfit, and launches a local coding agent in a single command. It collapses the multi-step local LLM setup into hf agents run pi, automatically handling hardware profiling, model download, inference server startup, and coding agent activation.

Hot-swap between local LLM models via OpenAI-compatible API

llama-swap is an open-source tool that manages multiple local LLM models behind a single OpenAI-compatible API endpoint. It automatically loads and unloads models on demand, letting developers hot-swap between different models without restarting services. With 3.1K+ GitHub stars, it solves the common pain point of running multiple specialized models on limited hardware.

High-performance local LLM inference in C/C++

llama.cpp is the foundational C/C++ library with 75K+ GitHub stars powering local LLM inference on consumer hardware. Provides optimized CPU and GPU inference for quantized models in GGUF format. Supports LLaMA, Mistral, Phi, Gemma, and most open-weight families. Features 2-8 bit quantization for reduced memory, multi-GPU support, context extension, grammar-constrained output, and an OpenAI-compatible API server. The engine behind Ollama and LM Studio.

Find which AI models actually run on your hardware in one command

llmfit is a Rust-based terminal tool that matches over 200 LLM models from 30+ providers against your exact hardware specs. The interactive TUI scores each model on fit, speed, VRAM usage, and context length, helping you avoid downloading models that won't run on your machine. It supports Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio backends.

Lossy image-token proxy for reducing Claude Code request context costs

pxpipe is a local proxy for Claude Code and compatible agent workflows that turns bulky request context—system prompts, tool docs, old history, code and JSON—into dense PNG image blocks before forwarding the call. The project README reports workload-dependent 59–70% end-to-end bill reduction, while warning exact byte values should stay as text.

Terminal AI chatbot — no API key required

Cross-platform terminal GPT client written in Go that provides free access to AI chatbots (KoboldAI, Blackbox AI, OpenAI) without API keys or account creation. Offers interactive multi-turn chat, shell command generation and execution, code generation, and image generation via Pollinations. Distributed as a single binary with no dependencies, making installation trivial on macOS, Linux, and Windows. Ideal for developers who want quick AI in the terminal without signup friction.

All-in-one embeddings database with RAG, search, and agent capabilities

txtai is a self-contained AI search and RAG platform that combines vector embeddings, semantic search, LLM pipelines, and agent workflows in a single Python library. It handles embedding generation, similarity search, extractive QA, summarization, translation, and custom pipelines without external dependencies. Runs locally with over 12,400 GitHub stars and Apache 2.0 license.