Best tools for Local AI Workflows

Unified framework for fine-tuning 100+ large language models

LLaMA-Factory is an open-source toolkit providing a unified interface for fine-tuning over 100 LLMs and vision-language models. It supports SFT, RLHF with PPO and DPO, LoRA and QLoRA for memory-efficient training, and continuous pre-training. The LLaMA Board web UI enables no-code configuration, while CLI and YAML workflows serve advanced users. Integrates with Hugging Face, ModelScope, vLLM, and SGLang for model deployment.

High-throughput LLM serving engine

vLLM is an Apache-2.0 LLM inference and serving engine focused on high-throughput self-hosted model APIs. It combines PagedAttention, continuous batching, prefix caching, quantization options, OpenAI-compatible serving, structured outputs, metrics, Docker/Kubernetes deployment guidance and integrations with agent and LLM frameworks.

Run LLMs locally with one command

Tool for running large language models locally on your machine with a simple CLI interface. Download and run Llama 3, Mistral, Gemma, Phi, Code Llama, and dozens of other open-source models with a single command. Features model management, GPU acceleration (NVIDIA/AMD/Apple Silicon), OpenAI-compatible API server, Modelfile for customization, and multi-model switching. Ideal for offline AI development, privacy-sensitive use cases, and local testing. 120K+ GitHub stars.

Self-hosted AI platform with ChatGPT-like interface for local and cloud LLMs.

Extensible, self-hosted AI platform with 290M+ Docker pulls and 124K+ GitHub stars. Supports Ollama, OpenAI-compatible APIs, and any Chat Completions backend. Features built-in RAG, multi-user RBAC, voice/video calls, Python function workspace, model builder, and web browsing. Runs entirely offline with enterprise features including SSO and audit logging.

2x faster LLM fine-tuning with 70% less VRAM on a single GPU

Unsloth is an open-source framework for fine-tuning large language models up to 2x faster while using 70% less VRAM. Built with custom Triton kernels, it supports 500+ model architectures including Llama 4, Qwen 3, and DeepSeek on consumer NVIDIA GPUs. Unsloth Studio adds a no-code web UI for dataset creation, training observability, model comparison, and GGUF export for Ollama and vLLM deployment.

Open-source AI agent framework with persistent memory, reusable skills, tools, and messaging gateways

Hermes Agent is an open-source AI agent framework with persistent memory, reusable skills, 40+ tools, cron jobs, and messaging gateways.

Self-hosted multi-model AI chat platform

LibreChat is an open-source ChatGPT-like interface with 35K+ GitHub stars supporting multiple AI providers in a single self-hosted platform. Connect OpenAI, Anthropic, Google, Mistral, local models via Ollama, and custom endpoints simultaneously. Features conversation branching, file uploads, code interpreter, plugins, presets, multi-user support with RBAC, and LDAP/SSO authentication. Privacy-focused alternative to commercial AI chat services with full data ownership.

Plain English QA testing for rapid MVPs

Stably enables developers to create QA tests in plain English using a no-code editor, with AI ensuring tests remain valid as the application evolves through self-healing locators and assertions. It lowers the barrier to high-quality QA for startups by eliminating the need for scripting knowledge, automatically adapting test steps when UI elements change position or structure.

Memory engine and context API for AI assistants

Supermemory is a memory and context platform for AI assistants and agents. It ranks #1 on LongMemEval, LoCoMo, and ConvoMem, supports MCP for Claude/Cursor-style clients, provides plugins for developer tools, and combines memory extraction, user profiles, hybrid search, connectors, and RAG in one API.

Microsoft's open-source frontier voice AI for long-form multi-speaker audio

VibeVoice is Microsoft's open-source voice AI family with both TTS and speech recognition models. The TTS model generates up to 90 minutes of expressive multi-speaker audio with 4 distinct voices. VibeVoice-ASR transcribes 60-minute recordings in a single pass with speaker identification and timestamps. Built on continuous speech tokenizers at 7.5 Hz and next-token diffusion, it compresses audio 80x more efficiently than Encodec while preserving fidelity.

All-in-one self-hosted AI app with RAG, agents, and multi-user support

AnythingLLM is an open-source, privacy-first AI application that turns any document into an interactive knowledge base. It bundles document ingestion, vector storage (built-in LanceDB), RAG pipelines, AI agents, and multi-user access into a single deployable package. Supports 30+ LLM providers including OpenAI, Anthropic, Ollama, and local models. With 62K+ GitHub stars and MIT license, it runs as a desktop app or Docker container with zero configuration required out of the box.

AI-native end-to-end testing with plain English test authoring

Momentic is an AI-native testing platform that lets teams write end-to-end tests in plain English. It features auto-healing test selectors that adapt to UI changes, instant mobile device emulators, built-in visual regression testing, and AI-powered flaky test handling. Backed by $15M Series A from Standard Capital, it eliminates brittle test maintenance through intelligent element identification and self-repairing test flows.

Self-hosted AI platform with RAG, agents, and 40+ connectors

Onyx is an open-core, self-hostable AI knowledge platform for enterprise search, RAG chat, deep research, custom agents, and workplace connectors. It connects to 40+ apps, supports permission-aware retrieval, and offers Cloud, Docker/Kubernetes, and enterprise deployment paths for teams that need controlled internal AI search.

Open-source multi-model AI chat framework with plugin ecosystem

LobeChat is a source-available AI chat and agent workspace for OpenAI, Claude, Gemini, Ollama, DeepSeek, and Qwen. It includes RAG, 10,000+ MCP-compatible plugins, Agent Groups, TTS/STT, Vercel/Docker self-hosting, and 79K+ GitHub stars.

Self-hosted AI coding assistant

Tabby is an open-source, self-hosted AI coding assistant that serves as a privacy-focused alternative to GitHub Copilot. It provides code completion, chat-based coding assistance, and IDE integrations while running entirely on your own infrastructure with support for consumer-grade GPUs. Tabby supports multiple LLM backends and offers extensions for VS Code, JetBrains, Vim, and other editors, giving teams full control over their AI coding tools without sending code to external services.

Codebase knowledge graph MCP server for AI coding agents

Codebase Memory MCP is an MIT-licensed MCP server that turns a repository into a persistent code knowledge graph for AI coding agents. It gives Claude Code, Cursor, Codex-style agents, and other MCP clients structural queries for functions, classes, call chains, routes, and architecture, helping them explore large projects without repeatedly rereading files or relying only on broad search.

Run local LLMs with an intuitive desktop GUI and OpenAI-compatible API server.

Free desktop application by Element Labs for discovering, downloading, and running open-source LLMs locally. Features a curated Hugging Face model browser, side-by-side model comparison, parameter tuning, and an OpenAI-compatible API server on localhost:1234. Powered by llama.cpp with Metal acceleration for Apple Silicon.

AMD's open-source local LLM server with GPU and NPU acceleration

Lemonade is AMD's open-source local AI serving platform for LLMs, image generation, speech recognition, and text-to-speech on your own hardware. Built in lightweight C++, it can detect CPU, GPU, and NPU backends and is extra optimized for Ryzen AI, Radeon, and Strix Halo PCs. Lemonade exposes OpenAI, Anthropic, and Ollama-compatible APIs, ships with a desktop model manager, and supports source-confirmed GGUF, FLM, and ONNX models across Windows, Linux, macOS, and Docker.

Open-source terminal AI agent by Alibaba

Apache-2.0 terminal coding agent with Qwen, OpenAI, Anthropic, Gemini, third-party, and local-model support. Qwen OAuth free access ended on April 15, 2026; use Alibaba ModelStudio, a provider API key, or a self-hosted endpoint.

Source-available LLM app development platform

Source-available LLM application development platform combining a visual no-code canvas with backend capabilities for building AI workflows, RAG pipelines, and agent systems from prototype to production. Integrates hundreds of models from dozens of providers, with PDF/PPT ingestion, ReAct agents with 50+ tool integrations, and multi-step orchestration. Used by both technical and non-technical teams to ship GenAI apps like chatbots and Q&A systems.

Context compression for LLM apps and coding agents

Headroom is an Apache-2.0 context compression layer for LLM apps and coding agents. It compresses tool output, logs, files, RAG chunks, and agent history through a local library, proxy, wrapper, or MCP server, with retrieval hooks for bringing originals back when needed. Treat its savings numbers as Headroom-reported benchmarks, not independent aicoolies measurements.

Knowledge graph-powered RAG framework from HKU

LightRAG is a research-backed RAG framework from Hong Kong University that combines knowledge graph structures with vector search for more contextual retrieval. Published at EMNLP 2025, it extracts entities and relationships from documents to build a structured knowledge graph, then uses dual-level retrieval across both graph and vector representations with five query modes: naive, local, global, hybrid, and mix.

Open-source AI coding agent for the terminal

Open-source terminal-based AI coding agent built in Go by the SST team, with a rich TUI (Bubble Tea) supporting 75+ model providers including OpenAI, Anthropic, Gemini, Bedrock, Groq, and OpenRouter. Features vim-like editing, persistent SQLite sessions, and LSP integration for 40+ languages. Fully free with no vendor lock-in, it has rapidly grown to 95k+ GitHub stars.

Autonomous Java unit test generation in CI

Diffblue Testing Agent orchestrates verified regression unit test generation for Java and Python projects through existing AI coding platforms such as GitHub Copilot CLI and Claude Code. It measures baseline coverage, generates tests, verifies that they compile and pass, and charges for net new coverage lines added rather than per seat or API call.

Open-source personal AI agent for messaging apps

OpenClaw is a free, open-source AI agent framework that turns any LLM into an autonomous personal assistant accessible through messaging apps like WhatsApp, Telegram, Discord, and Signal. Running entirely on your local machine via a Node.js gateway, it connects AI models to system tools, browsers, files, and APIs for multi-step task execution with persistent memory across sessions.

100% private document Q&A powered by local LLMs

PrivateGPT enables fully private document interaction using GPT-powered RAG without any data leaving your machine. Ingest documents (PDF, DOCX, TXT, and more) and chat with them using local LLMs via Ollama or remote providers. Built on LlamaIndex with Qdrant vector storage. 57,200+ GitHub stars, Apache 2.0 licensed. The go-to solution for air-gapped environments, regulated industries, and anyone who needs document Q&A without cloud data exposure.

MCP server for controlling Windows desktops through UIAutomation

Windows-MCP is an open-source MCP server for giving AI agents structured access to Windows desktop automation. It focuses on UIAutomation, snapshots, input control, and Windows-specific app workflows, making it different from general filesystem or shell MCP servers.

Run frontier AI models across a cluster of everyday devices

exo turns multiple local machines into a unified AI compute cluster for models that exceed a single device's memory. It automatically discovers devices, uses topology-aware auto parallelism to split work across available resources, and supports RDMA over Thunderbolt 5 for co-located clusters or standard networking for looser setups. The project exposes OpenAI Chat Completions, Claude Messages, OpenAI Responses, and Ollama-compatible APIs plus a dashboard for cluster management.

Run LLMs as a single portable executable file

Llamafile by Mozilla packages a complete LLM — model weights, inference engine, and OpenAI-compatible API server — into a single executable file that runs on Mac, Windows, Linux, FreeBSD, and OpenBSD with no installation. Built on llama.cpp and Cosmopolitan Libc for cross-platform portability, it delivers GPU-accelerated inference when available and falls back to optimized CPU execution. Supports GGUF models with a built-in web chat UI and REST API for integration.

Drag-and-drop LLM flow builder

Open-source protocol for connecting AI models to external tools and data sources, created by Anthropic. Provides a standardized way for LLMs to interact with APIs, databases, and local files through a universal client-server architecture. Rapidly adopted across the AI ecosystem as the standard interface between AI assistants and the tools they need to be useful.

Self-evolving local computer agent with a reusable skill tree

GenericAgent is a minimal, self-evolving autonomous agent from a 3.3K-line seed and ~3K core loop that gives LLMs system-level control of a local computer. It writes files, runs shell commands, browses the web, and uses keyboard/mouse/screen/mobile tools, while skill crystallization saves successful runs into a reusable skill tree that cuts token cost on repeats.

AI test generation from production traffic

Tusk is a Y Combinator W24-backed AI testing platform that converts real production traffic into unit and API tests, catching regressions in 43% of PRs. Its Drift SDK records live API traces with just 10 lines of code, then AI generates executable test cases covering thousands of edge cases from actual user behavior, auto-maintaining suites as application logic evolves without manual script writing.

Natural language interface for running code on your computer

Open Interpreter is an open-source natural-language interface for computers. It runs Python, JavaScript, shell commands, and other code locally through a ChatGPT-like terminal workflow, with user approval before execution. It can use hosted providers or local models, but its main tradeoff is safety: approved commands can access local files, apps, and system resources.

AI codeless test automation across platforms

ACCELQ Autopilot is an AI-powered codeless test automation platform with an Adaptive Relevance Engine that suggests next test steps and self-heals element locators across web, mobile, API, and desktop applications. It provides a unified testing environment that eliminates the need for scripting knowledge while maintaining the depth and flexibility required for enterprise-grade test automation.

All-in-one LLM CLI tool with shell assistant, RAG, and function calling

All-in-one LLM CLI written in Rust that gives unified access to 20+ AI providers (OpenAI, Claude, Gemini, Ollama, Groq, Mistral, DeepSeek, and more) through one terminal interface. Includes a shell assistant, interactive REPL with session management, RAG over local files and URLs, custom roles, function calling, and a built-in HTTP server exposing Chat Completions, Embeddings, and Rerank APIs plus an LLM Playground and Arena for model comparison.

Open-source desktop AI coworker for browsing and code execution.

Accomplish Coworker is an MIT-licensed open-source AI coworker that runs on the desktop, combining computer-use style browsing with code execution so agents can research, implement, run, and debug workflows in one local environment.

AI autopilot for existing test cases

AgentQ runs existing manual test cases on autopilot using AI to navigate applications and verify expected behavior without requiring test script rewriting. It bridges the gap between manual test plans and full automation by taking existing test documentation and executing it automatically, serving teams with large manual test backlogs who need immediate automation without migration effort.

Open-source toolkit for audio, music, and speech generation

Amphion is an open-source audio generation toolkit from OpenMMLab designed for reproducible research in speech synthesis, voice conversion, singing voice synthesis, and text-to-audio generation. It implements state-of-the-art models including MaskGCT, DualCodec, VITS, and VALL-E with built-in architecture visualizations for educational use. The project ships with the Emilia-Large dataset of 200,000 hours of speech data and includes multiple vocoders and evaluation metrics for benchmarking.

Visual AI testing for web and mobile applications

Applitools is a visual AI testing platform using computer vision to detect visual regressions across web and mobile apps. Its Visual AI compares screenshots with intelligent diffing that ignores irrelevant changes while catching meaningful bugs. Features Ultrafast Grid for parallel cross-browser testing, Eyes SDK integration with Cypress, Playwright, Selenium, and Storybook. Supports responsive design testing across multiple viewports and devices with a single test script.

Python and TypeScript framework for production multi-agent systems

BeeAI Framework is an Apache-2.0 toolkit for building production-ready AI agents and multi-agent systems in Python and TypeScript. Its docs cover agents, tools, RAG, memory, workflows, backend providers, serving, and A2A/MCP integration surfaces, making it a vendor-neutral option for teams comparing LangGraph, CrewAI, Mastra, and related agent runtimes.

Microsoft's framework for running 1-bit large language models on consumer CPUs

BitNet is Microsoft's official inference framework for 1-bit quantized large language models that enables running models with up to 100 billion parameters on standard consumer CPUs without requiring a GPU. By leveraging extreme quantization where weights use only 1.58 bits on average, BitNet achieves dramatic reductions in memory footprint and computational cost while maintaining competitive output quality for many practical use cases.

Open-source agentic browser that runs local AI agents in your browsing workflow.

BrowserOS is a privacy-first, open-source agentic browser for running AI assistants locally inside real browsing sessions instead of handing every task to a remote cloud browser.

Autonomous E2E test agent that explores your app

Bugster is an autonomous testing agent that explores applications independently to write and run end-to-end tests without requiring manual script creation. It identifies broken flows by navigating the UI like a real user, automatically discovering clickable elements, form inputs, and navigation paths, then generating reproducible test scripts for the issues it finds.

On-device AI inference engine for mobile and wearable applications

Cactus is a YC-backed low-latency AI engine for mobile and wearable devices that runs LLMs, transcription, embedding, and TTS models locally. It achieves 16-20 tok/sec on older devices and 70+ tok/sec on flagships with ARM SIMD kernels optimized for Snapdragon, Apple, and MediaTek processors. Supports Qwen, Gemma, Llama, DeepSeek with Flutter, React Native, and Kotlin SDKs.

Wafer-scale inference at thousands of tokens per second

Cerebras Inference serves open-weight LLMs like Llama, Qwen, and GPT-OSS on wafer-scale CS-3 chips through an OpenAI-compatible API, benchmarking between 1,800 and 2,600 output tokens per second on Llama 3.1 8B and several hundred on 70B models. A free tier offers one million tokens per day with no credit card, while paid pay-per-token pricing starts at $0.04 per million tokens for the smaller Llama models.

One desktop app for every LLM — private, cross-platform, extensible

Chatbox is a cross-platform desktop AI client supporting OpenAI, Claude, Gemini, DeepSeek, and local models via Ollama. All chat data stays on-device, making it ideal for privacy-conscious developers. Features include document analysis, code assistance with syntax highlighting, image generation, web search, and a local knowledge base for private Q&A. Available on Windows, macOS, Linux, Android, iOS, and web.

State-of-the-art open-source text-to-speech with emotion control

Chatterbox is an open-source text-to-speech model by Resemble AI that delivers state-of-the-art voice synthesis with fine-grained emotion and style control. The model supports zero-shot voice cloning from short audio samples, produces natural-sounding speech across multiple speaking styles, and runs locally without cloud dependencies. With over 24,000 GitHub stars, it has become the leading open-source alternative to commercial TTS services for developers building voice-enabled AI applications.

Desktop AI productivity studio with 300+ assistants

Cherry Studio is an open-source desktop AI application with 42K+ GitHub stars providing unified access to 50+ LLM providers through a single Electron-based interface. Features 300+ pre-built AI assistants for different tasks, multi-model conversations, knowledge base with RAG support, real-time translation, image generation, and Mermaid diagram rendering. Supports topic-based chat management, message branching, and custom assistant creation. Available on macOS, Windows, and Linux.