Best tools for Local AI Workflows

Persistent memory plugin for Claude Code with automatic context injection

Claude-Mem is a persistent memory plugin for Claude Code with 44,000+ GitHub stars that captures session context and injects it into future sessions. It features progressive disclosure with token cost visibility, automatic compression, and privacy controls with private tags to manage what gets remembered across coding sessions.

Open-source deep learning text-to-speech toolkit

Coqui TTS is an open-source deep learning toolkit for text-to-speech synthesis, originally built by former Mozilla TTS engineers. It supports multi-speaker and multilingual synthesis, voice cloning from just six seconds of audio, and ships pre-trained models for 20+ languages. After Coqui shut down in 2023, the Idiap Research Institute forked and actively maintains it. With 45K+ GitHub stars, it remains the most popular open-source TTS framework in Python.

AI test writing with self-healing and visual editor

DogQ is a testing tool that writes, heals, and suggests tests automatically through a visual interface for managing the entire testing lifecycle. It provides a low-code alternative for teams without dedicated SDETs (Software Development Engineers in Test), automating test creation from user interactions and maintaining test health as the application evolves with AI-driven self-healing.

PyTorch on-device AI for mobile and edge devices

ExecuTorch is PyTorch's official solution for deploying AI models on mobile, embedded, and edge devices. It features a 50KB base runtime, 12+ hardware backends including Apple CoreML, Qualcomm QNN, ARM, and Vulkan, and native PyTorch export without format conversions. Powers Meta's on-device AI across Instagram, WhatsApp, Quest 3, and Ray-Ban Smart Glasses, supporting LLMs, vision, speech, and multimodal models.

No-code knowledge base platform with visual AI workflow and built-in RAG

FastGPT is an open-source no-code AI knowledge base platform with 27,000+ GitHub stars and 500,000+ users worldwide. It combines visual workflow orchestration, built-in RAG pipelines, QA-pair extraction, and API-aligned completions into a single deployable stack that runs on just 2GB RAM via Docker one-liner deployment.

AI QA agent specialized for mobile apps

FinalRun is a specialized AI QA agent for mobile applications that automates testing of complex mobile gestures and flows on both iOS and Android platforms. It addresses the specific mobile QA gap that generic web-testing tools miss, handling touch interactions, swipe gestures, device rotation, and platform-specific behavior that require mobile-native understanding.

Multilingual emotional text-to-speech with 80+ language support

Fish Speech is an open-source text-to-speech system supporting 80+ languages with emotional expression, zero-shot voice cloning, and real-time streaming. It generates natural speech with controllable emotions, speaking styles, and prosody. Features a web interface, API server, and integration with AI agent frameworks for voice-enabled applications. Over 29,000 GitHub stars.

Blazing fast Solidity development toolkit

Foundry is a blazing-fast portable toolkit for Ethereum smart contract development written in Rust. It provides Forge for testing and deploying Solidity contracts with native Solidity tests that run orders of magnitude faster than JavaScript-based alternatives, Cast for interacting with EVM chains from the command line, Anvil as a local testnet node, and Chisel as a Solidity REPL. Foundry has become the standard toolchain for serious Solidity development.

AI-powered resilient mobile test flows

GPT Driver is a mobile app QA tool that uses generative AI to create resilient test flows with self-healing capabilities and broad framework support for cross-platform applications. It leverages AI to handle the non-deterministic nature of mobile UI where elements shift between releases, generating tests that adapt to layout changes rather than breaking on every app update.

Open-source voice cloning and text-to-speech with few-shot learning

GPT-SoVITS is an open-source voice cloning and text-to-speech system that generates natural-sounding speech from just a few seconds of reference audio. It combines GPT-style language modeling with SoVITS voice synthesis for zero-shot and few-shot voice cloning across multiple languages. Supports Chinese, English, Japanese, Korean, and Cantonese with over 56,000 GitHub stars.

Codeless browser testing with visual test recorder and scheduling

Ghost Inspector provides codeless browser testing through a visual recorder that captures user interactions and converts them into automated test suites. Tests run on managed infrastructure with scheduled execution, CI/CD integration, and Slack notifications. Features visual comparison for UI regression detection, API testing, and test organization with folders and tags for managing large test suites.

Run open-source LLMs on your phone, fully offline and private

Google AI Edge Gallery is an open-source mobile app that lets you download and run large language models like Gemma directly on Android and iOS devices with zero cloud dependency. Built on MediaPipe and LiteRT, it features AI chat with reasoning mode, multimodal image analysis, real-time audio transcription, and autonomous agent skills—all running entirely on-device for complete privacy. A reference implementation for developers building offline-first AI experiences.

Offline-first AI assistant for local inference

Jan is an open-source offline-first AI assistant with 25K+ GitHub stars running LLMs locally without sending data externally. Features a ChatGPT-like interface with one-click model downloads from Hugging Face, conversation management, customizable prompts, and an OpenAI-compatible local API server. Supports GGUF models via llama.cpp with GPU acceleration on NVIDIA and Apple Silicon. Built with Electron for macOS, Windows, and Linux with full data privacy.

Heterogeneous CPU-GPU inference and SFT for large MoE models

Open-source framework for running and fine-tuning large Mixture-of-Experts models with heterogeneous CPU-GPU execution, optimized kernels, limited VRAM and SGLang or LLaMA-Factory integrations.

Open-source AI second brain with deep research and RAG

Khoj is an open-source personal AI app that serves as a self-hostable second brain. It connects to your documents — PDFs, Markdown, Notion, Word — and uses RAG to answer questions grounded in your knowledge base. Supports any local or cloud LLM including Llama, Claude, GPT, and Gemini. Features custom agents, scheduled automations, deep research mode, semantic search, and Obsidian, Emacs, and WhatsApp integrations. Over 33,000 GitHub stars, YC-backed.

RAG-based document QA with multi-user support and agent reasoning

Kotaemon is an open-source RAG-powered document question-answering interface backed by Cinnamon AI. It supports multi-user workspaces with access controls, advanced retrieval pipelines including hybrid search and knowledge graph extraction, and agentic reasoning for complex multi-step queries. The web UI handles PDFs, Office documents, and images with citations pointing to exact source passages, making it suitable for both individual research and team knowledge management.

Kubernetes operator for serving AI inference workloads

KubeAI is an Apache-2.0 Kubernetes operator for deploying and scaling AI inference workloads, including LLMs, embeddings, reranking, and speech-to-text. It gives platform teams OpenAI-compatible endpoints, model proxy/controller primitives, model caching, scale-from-zero behavior, and cluster-native resource management for self-hosted inference on Kubernetes.

Google's production on-device LLM inference framework

LiteRT-LM is Google's official open-source framework for running large language models on-device across Android, iOS, Web, Desktop, and Raspberry Pi. Already deployed in Chrome and Pixel hardware, it provides production-grade on-device LLM inference with 1.4K+ GitHub stars. Apache 2.0 licensed.

Free, open-source local AI inference engine

LocalAI is an open-source local AI inference engine with 44K+ GitHub stars that runs LLMs, image generation, audio transcription, and embeddings entirely on consumer hardware without GPU requirements. Provides an OpenAI API-compatible REST endpoint as a drop-in replacement, supporting 1000+ models including LLaMA, Mistral, and Phi families. Features include text-to-speech, speech-to-text, function calling, constrained grammar output, and multi-modal capabilities all running locally.

Open-source MCP bridge between AI assistants and the Unity Editor

MCP for Unity is CoplayDev’s MIT-licensed bridge between MCP-compatible AI assistants and the Unity Editor. It exposes tools for assets, scenes, GameObjects, scripts, tests, profiling, and build-oriented workflows. The community project supports Unity 2021.3 LTS through 6.x and is explicitly not affiliated with Unity Technologies.

Run LLMs natively on any device with ML compilation

MLC LLM is an open-source engine for deploying large language models natively across diverse platforms using machine learning compilation. It runs models on NVIDIA/AMD GPUs, Apple Silicon, mobile devices, and browsers via WebGPU without cloud dependencies. Features include OpenAI-compatible API, quantization support, and optimized backends for CUDA, Metal, Vulkan, and WebAssembly.

Run and fine-tune Vision Language Models locally on Mac

Open-source Python package for running and fine-tuning Vision Language Models locally on Mac using Apple's MLX framework. Supports multimodal inference with images, audio, and video across Qwen, DeepSeek, Phi, and Gemma architectures. Features OpenAI-compatible API server, Gradio chat UI, and KV cache optimization. 3.8K+ GitHub stars.

Lightweight mobile and edge AI inference engine

MNN is a lightweight, high-performance deep learning inference engine developed by Alibaba and battle-tested across 30+ Alibaba apps including Taobao, DingTalk, and Youku. It supports TensorFlow, ONNX, PyTorch, and Caffe models with optimized backends for CPU, GPU, and NPU on mobile and edge devices. MNN includes on-device LLM inference, an OpenCV-like image processing library, and Python bindings for rapid prototyping. Apache 2.0 licensed with 15K+ stars.

Low-code AI test automation for modern teams

Mabl is a low-code AI test automation platform for end-to-end testing of web apps, APIs, mobile, and accessibility. Uses machine learning for auto-healing tests that adapt to application changes, reducing flaky test maintenance. Features a no-code visual builder, parallel cross-browser execution, performance testing, and native CI/CD integration. Provides unified reporting with insights into test coverage and quality trends. Integrates with Jira, Slack, GitHub, and major CI/CD tools.

Human-in-the-loop web agent you can co-pilot in real time

Magentic-UI is a Microsoft Research web agent with a human-in-the-loop interface for browsing, coding, and file tasks. It plans multi-step actions, asks for approval before executing, and lets users co-pilot by taking over the browser mid-task. Built on AutoGen, it runs a team of specialized agents for web browsing, file handling, and code execution with full action transparency and safety guardrails.

Enterprise-grade RAG and MCP knowledge base with one-click deployment

MaxKB is an enterprise-grade RAG platform with 21,000+ GitHub stars from the 1Panel team. It provides one-click deployment of knowledge bases with built-in LLM integration, MCP support, and a streamlined approach to document ingestion and retrieval that prioritizes operational simplicity over configuration complexity.

Single-file memory layer replacing complex RAG for AI agents

Memvid is an open-source single-file memory system for AI agents with 13,700+ GitHub stars. It replaces complex RAG infrastructure with instant retrieval from portable .mv2 files, claiming 35% accuracy improvement over state-of-the-art on LoCoMo benchmarks with 0.025ms P50 latency. Available for Python, Node.js, Rust, and CLI.

YC-backed multimodal RAG platform for documents, images, and video

Morphik is a YC-backed multimodal RAG platform that ingests and retrieves information from documents, images, tables, and video content. It processes complex document layouts including charts, diagrams, and multi-column formats that traditional text-only RAG systems handle poorly. Provides API-first integration for building knowledge bases that understand visual as well as textual information.

High-performance mobile neural network inference

NCNN is Tencent's high-performance neural network inference framework optimized for mobile and embedded platforms. It features pure C++ with zero dependencies, ARM NEON assembly optimization, Vulkan GPU acceleration, and sophisticated memory management for minimal footprint. Supports importing models from PyTorch, ONNX, Caffe, TensorFlow, and Keras with 8-bit quantization and half-precision storage for efficient on-device deployment across Android, iOS, and Linux.

Cross-platform on-device AI model runtime

Nexa SDK enables running frontier LLMs and multimodal models locally across PC, mobile, IoT, and wearables with automatic hardware acceleration for GPU, NPU, and CPU. It supports Qwen, Gemma, Llama, DeepSeek models with Python/C++ desktop SDKs, Android/iOS mobile SDKs, and Docker for edge deployment. Includes an OpenAI-compatible API server with chat and function calling support.

Cross-platform high-performance ML inference engine

ONNX Runtime is Microsoft's open-source inference engine for machine learning models in ONNX format. It delivers cross-platform acceleration via execution providers for NVIDIA CUDA, TensorRT, DirectML, CoreML, OpenVINO, and more. Supports training acceleration, quantization, and GenAI workloads. Used in production across Windows, Azure, Office 365, and thousands of applications with pip-installable Python and native C++/C#/Java APIs.

Private, self-hosted research notebooks with flexible AI models, source chat, and podcasts

Open Notebook is an MIT-licensed, self-hosted alternative to NotebookLM for collecting sources, chatting over research, generating reusable transformations, and producing multi-speaker podcasts. Its Docker stack keeps notebook data under the user's control while supporting 18-plus model providers, including local Ollama and LM Studio workflows.

Intel's open-source AI inference optimization toolkit

OpenVINO is Intel's open-source toolkit for optimizing and deploying AI inference across CPUs, GPUs, and NPUs. It supports models from PyTorch, TensorFlow, ONNX, and TFLite, providing graph optimizations, quantization, and hardware-specific acceleration. The toolkit includes a GenAI API for LLM deployment and runs on Intel, ARM, and x86 platforms for edge, desktop, and cloud inference workloads.

Structured generation for LLMs

Outlines is an open-source Python library for structured text generation that guarantees LLM outputs conform to a defined schema or format. It constrains the model's token selection at each step so only tokens leading to valid output are considered, eliminating fragile post-processing. Supports multiple-choice constraints, regex patterns, JSON Schema, and type-safe Pydantic models — helping teams extract reliable structured data from any LLM.

First commercially viable 1-bit LLMs that are 14x smaller and 8x faster

PrismML Bonsai delivers the first commercially viable 1-bit large language models with 8B, 4B, and 1.7B parameter variants. The 8B model runs in just 1GB of RAM versus 16GB for standard FP16 models, achieving 44 tokens per second on iPhone. Backed by $16.25M from Khosla Ventures and released under Apache 2.0, Bonsai makes capable LLMs practical for edge devices and resource-constrained environments.

Modern test management with AI organization

QA Sphere is a modern, clutter-free test management system that uses AI to organize test cases and track coverage across complex projects. It streamlines the management side of QA that is often handled through messy spreadsheets, providing structured test case repositories, execution tracking, and coverage analytics for technical project managers and QA leads.

On-device hybrid search engine for your docs and notes

QMD is an on-device search engine built by Tobi Lütke (Shopify CEO) that indexes markdown notes, meeting transcripts, and documentation locally. It combines BM25 full-text search, vector semantic search, and LLM-powered re-ranking into a single hybrid pipeline. Ships with a built-in MCP server for seamless integration with Claude Code, Cursor, and other AI editors. All processing happens on your machine via node-llama-cpp with GGUF models — zero cloud dependency.

Optimize and deploy AI models on Snapdragon devices

Qualcomm AI Hub is a platform for optimizing and deploying AI models on Snapdragon-powered devices with NPU acceleration. It provides pre-optimized models, profiling tools, and the SNPE SDK for compiling models to run efficiently on Qualcomm's Hexagon DSP and AI Engine. Supports hundreds of model architectures with on-device benchmarking across real Snapdragon chipsets for mobile, IoT, and XR applications.

Production RAG engine with hybrid search and knowledge graphs

R2R is a production-grade RAG engine from SciPhi AI that combines hybrid search with knowledge graph extraction and agentic retrieval capabilities. It provides a complete pipeline from document ingestion through retrieval and generation, supporting vector, keyword, and graph-based search strategies. The managed API and self-hosted options make it accessible for both rapid prototyping and production deployments requiring advanced retrieval beyond simple vector similarity.

All-in-one multimodal RAG framework

RAG-Anything is an all-in-one multimodal RAG framework from the University of Hong Kong that processes text, images, tables, and equations through a unified pipeline built on LightRAG. It constructs multi-modal knowledge graphs by extracting multimodal entities and establishing cross-modal relationships. The VLM-Enhanced Query mode integrates visual content into large language models for deeper document understanding beyond plain text retrieval.

On-device AI inference for React Native apps

Declarative framework for running AI models on-device in React Native applications, powered by Meta ExecuTorch runtime. Supports LLMs including Llama 3.2, computer vision, OCR, embeddings, and vision-language models on iOS 17+ and Android 13+. Developed by Software Mansion with pre-built optimized models, custom model export support, and privacy-first inference without any cloud dependency for mobile AI development.

DeepSeek-native terminal coding agent with a Go rewrite and MCP support

Reasonix is an open-source terminal coding agent built around DeepSeek workflows, with a newer Go-based 1.0 line, MCP integration, repository-aware code understanding, and BYOK model usage. It fits developers who want a DeepSeek-first CLI agent rather than a Claude- or OpenAI-native workflow.

Test generation from real user behavior sessions

Relicx uses generative AI to conduct visual and functional testing based on real user behavior captured from production sessions. It prioritizes the user experience in testing rather than just code coverage, generating regression suites from actual user journeys to ensure that the flows real people use most frequently are always covered by automated tests.

Cross-platform on-device AI inference SDK

RunAnywhere SDK is a production-ready toolkit for running AI models entirely on-device across iOS, macOS, Android, Web, React Native, and Flutter. It provides a unified C++ core with platform-specific bindings for LLM text generation via llama.cpp, vision-language models, Whisper speech-to-text, Piper text-to-speech, and on-device image generation. All processing stays local with zero cloud dependency, ensuring privacy and low latency for mobile and edge AI applications.

Apple's Safari-native MCP server for web debugging agents

Safari MCP Server is Apple's safaridriver-based MCP server in Safari Technology Preview, giving compatible coding agents local access to Safari page content, console logs, network requests, screenshots, JavaScript evaluation, interactions, viewport controls, and accessibility/performance checks.

Rust-powered coding agent and pair programmer by OVH

ShAI (Shell AI) is a CLI that brings natural language to the command line, turning plain English descriptions into executable shell commands tailored to your OS and shell. Offers command generation plus command explanation (paste an unfamiliar command, get a breakdown). Adapts syntax for macOS, Linux, and Windows across bash, zsh, fish, and PowerShell. The OVH shai variant extends further into a full pair-programming coding agent with support for multiple LLM providers.

AI-powered command-line productivity tool

ShellGPT (sgpt) is a command-line productivity tool powered by OpenAI models that generates shell commands, code snippets, and documentation directly in the terminal. Integrates with Bash, Zsh, PowerShell, and CMD; places AI-suggested commands into the terminal input via hotkeys. Supports persistent chat sessions, a REPL mode, and custom Python functions for extending capabilities. Accepts stdin and arguments for pipe-based workflows where command output can be piped to the AI.

Mutation testing framework to validate test quality

Stryker Mutator is an open-source mutation testing framework for JavaScript, TypeScript, C#, and Scala that measures the real effectiveness of your test suite. It introduces small code mutations and checks whether tests catch them, revealing gaps that line coverage metrics miss. Supports incremental mode for CI/CD integration.