aicoolies logo
VibeVoice logo
VibeVoice logo

VibeVoice

Microsoft's open-source frontier voice AI for long-form multi-speaker audio

open sourceupdated Aug 16, 2026

VibeVoice is Microsoft's open-source voice AI family with both TTS and speech recognition models. The TTS model generates up to 90 minutes of expressive multi-speaker audio with 4 distinct voices. VibeVoice-ASR transcribes 60-minute recordings in a single pass with speaker identification and timestamps. Built on continuous speech tokenizers at 7.5 Hz and next-token diffusion, it compresses audio 80x more efficiently than Encodec while preserving fidelity.

Read our VibeVoice review

A detailed review by the aicoolies team — click to read

VibeVoice represents a significant architectural innovation in speech synthesis. Traditional TTS systems struggle with long-form audio due to high token rates that create computational bottlenecks, and they typically handle only one or two speakers with limited emotional range. VibeVoice solves these challenges through continuous speech tokenizers operating at an ultra-low frame rate of approximately 7.5 Hz, compressing audio representation dramatically while maintaining acoustic fidelity. The system uses a Large Language Model based on Qwen 2.5 to understand textual context and dialogue flow, combined with a diffusion head that generates high-fidelity acoustic details. The result is natural conversational audio with proper turn-taking, emotional nuance, and consistent speaker identity across long sequences.

The family includes three main components. VibeVoice-1.5B is the flagship TTS model accepted as an Oral at ICLR 2026, generating up to 90 minutes of multi-speaker conversational audio. VibeVoice-Realtime-0.5B is a lightweight variant for streaming TTS with approximately 200ms latency, designed for real-time services and LLM voice output. VibeVoice-ASR handles speech-to-text for 60-minute recordings, producing structured transcriptions with speaker identification, timestamps, and customized hotword support. The ASR component was integrated into Hugging Face Transformers in March 2026. All models support English and Chinese natively, with experimental multilingual capabilities in nine additional languages including German, French, Japanese, and Korean.

Released under MIT license with built-in safety measures including audible AI-generated disclaimers and imperceptible watermarks for provenance verification, VibeVoice is available through GitHub documentation and Hugging Face model cards. Microsoft positions the project for research and development, and the current repository notes that the TTS code was removed for responsible-use reasons even though public model cards remain available. With about 49K GitHub stars, VibeVoice is especially relevant for developers evaluating voice agents, podcast generation, and voice-enabled AI applications.

Pricing

Free and open-source (MIT license); Self-hosted only

Platforms

Python/PyTorch, Hugging Face model cards, Colab/Transformers demos; GPU recommended

Categories

Tags

Use Cases

Google Research logo

TimesFM

Google's pretrained foundation model for zero-shot time-series forecasting

TimesFM is a pretrained time-series foundation model from Google Research that performs zero-shot forecasting on diverse datasets without task-specific training. It handles univariate and multivariate time series across domains including finance, logistics, energy, and infrastructure monitoring with accuracy competitive against traditional statistical methods like ARIMA and Prophet.

Open Source
PrismML Bonsai logo

PrismML Bonsai

First commercially viable 1-bit LLMs that are 14x smaller and 8x faster

PrismML Bonsai delivers the first commercially viable 1-bit large language models with 8B, 4B, and 1.7B parameter variants. The 8B model runs in just 1GB of RAM versus 16GB for standard FP16 models, achieving 44 tokens per second on iPhone. Backed by $16.25M from Khosla Ventures and released under Apache 2.0, Bonsai makes capable LLMs practical for edge devices and resource-constrained environments.

Open Source
verl logo

verl

Production-grade reinforcement learning framework for LLM training

verl is an open-source reinforcement learning framework designed specifically for training and aligning large language models. Built for production use with support for distributed training across multiple GPUs and nodes, it implements RLHF, DPO, and other alignment algorithms that make LLMs follow instructions, avoid harmful outputs, and generate higher quality responses. Over 580 contributors and 20,000 GitHub stars signal strong adoption.

Open Source
Resemble AI logo

Chatterbox

State-of-the-art open-source text-to-speech with emotion control

Chatterbox is an open-source text-to-speech model by Resemble AI that delivers state-of-the-art voice synthesis with fine-grained emotion and style control. The model supports zero-shot voice cloning from short audio samples, produces natural-sounding speech across multiple speaking styles, and runs locally without cloud dependencies. With over 24,000 GitHub stars, it has become the leading open-source alternative to commercial TTS services for developers building voice-enabled AI applications.

Open Source
llm-d logo

llm-d

Kubernetes-native distributed LLM inference stack

llm-d is an open-source Kubernetes-native stack for distributed LLM inference with cache-aware routing and disaggregated serving. It separates prefill and decode stages across different GPU pools for optimal resource utilization, routes requests to nodes with warm KV caches, and integrates with vLLM as the serving engine. Apache-2.0 licensed with 2,900+ GitHub stars.

Open Source

Related Tools

computed discovery: shared active categories · kept separate from editor-verified Alternatives

FiftyOne logo

FiftyOne

Open-source toolkit for curating datasets and evaluating visual AI models

FiftyOne is an open-source Python toolkit from Voxel51 for building high-quality datasets and better computer-vision and multimodal AI models. It pairs a browser-based visualization App with programmatic dataset curation, embeddings, similarity search, and model-evaluation workflows.

freemiumOpen SourceTelemetry
Open Notebook logo

Open Notebook

Private, self-hosted research notebooks with flexible AI models, source chat, and podcasts

Open Notebook is an MIT-licensed, self-hosted alternative to NotebookLM for collecting sources, chatting over research, generating reusable transformations, and producing multi-speaker podcasts. Its Docker stack keeps notebook data under the user's control while supporting 18-plus model providers, including local Ollama and LM Studio workflows.

Open SourceTelemetry
Hugging Face logo

Text Embeddings Inference

Hugging Face's open-source inference server for embeddings, rerankers, and classifiers

Text Embeddings Inference is Hugging Face's Apache-2.0 server for high-throughput embedding, reranking, and sequence-classification models. TEI packages token-based dynamic batching, optimized Transformers kernels, Safetensors loading, OpenAI-compatible embedding endpoints, Prometheus metrics, and configurable OpenTelemetry tracing in deployable CPU and GPU images.

Open Source
Presidio logo

Presidio

Open-source PII detection and anonymization for AI data flows

Presidio is an MIT-licensed privacy framework for identifying and anonymizing personally identifiable information in text, images, and structured data. It can act as a de-identification layer around LLM prompts, logs, RAG corpora, and customer-data workflows.

Open Source
ElevenLabs logo

ElevenLabs

Lifelike AI voice generation, cloning, and voice agents

ElevenLabs is an AI voice platform for text-to-speech, voice cloning, and conversational AI agents, built on models like Multilingual v2 and the low-latency Flash v2.5 and Turbo v2.5. Developers call its API to generate lifelike narration, clone voices from short audio samples, dub content across 30+ languages, add sound effects, and deploy real-time voice agents for customer service, IVR, and interactive apps, with SDKs for Python, JavaScript, and more.

freemium
Deep Lake logo

Deep Lake

AI data runtime for multimodal datasets and vector search

Deep Lake is an open-source AI data runtime from Activeloop for storing, versioning, and querying multimodal data and embeddings. It fits teams building RAG, training, evaluation, or dataset-heavy agent workflows that need a bridge between vector search, structured metadata, and large image, text, audio, or video collections.

Open Source

Used in Stacks

Comparisons

VibeVoice vs Chatterbox: Open-Source Text-to-Speech Models Compared

VibeVoice and Chatterbox are both open-source text-to-speech models, but they target very different use cases. VibeVoice from Microsoft generates 90-minute multi-speaker conversations for podcast-style audio, while Chatterbox focuses on single-speaker voice cloning with emotional control. Understanding their strengths helps developers choose the right TTS model for their application.

VibeVoiceChatterbox

FAQ

What is VibeVoice?

VibeVoice is Microsoft's open-source voice AI family with both TTS and speech recognition models. The TTS model generates up to 90 minutes of expressive multi-speaker audio with 4 distinct voices. VibeVoice-ASR transcribes 60-minute recordings in a single pass with speaker identification and timestamps. Built on continuous speech tokenizers at 7.5 Hz and next-token diffusion, it compresses audio 80x more efficiently than Encodec while preserving fidelity.

Is VibeVoice free?

Yes — VibeVoice is open source and free to use. Free and open-source (MIT license); Self-hosted only

Is VibeVoice open source?

Yes — VibeVoice is open source.

What are the best VibeVoice alternatives?

The top editor-verified VibeVoice alternatives are TimesFM, PrismML Bonsai, verl, and more.

How does VibeVoice score in our review?

Our hands-on review scores VibeVoice 87/100 overall, based on speed, privacy, and developer-experience testing.