torchtune provides a PyTorch-native approach to LLM fine-tuning that prioritizes composability and transparency over abstraction. Rather than hiding training complexity behind a unified interface, torchtune exposes clean building blocks that researchers and engineers can combine, modify, and extend according to their specific requirements. The library ships with battle-tested training recipes for supervised fine-tuning, LoRA and QLoRA parameter-efficient tuning, direct preference optimization, knowledge distillation, and continued pre-training.
As Meta's official fine-tuning library within the PyTorch ecosystem, torchtune maintains first-class support for the Llama model family while extending compatibility to Mistral, Gemma, Qwen, and Phi architectures. The recipes handle distributed training across multiple GPUs using PyTorch's native FSDP and tensor parallelism, memory optimization through activation checkpointing and gradient accumulation, and quantization through integration with torchao. Unlike wrapper frameworks that add layers of abstraction, torchtune's minimal dependency design makes it straightforward to debug training issues and customize behavior at any level of the stack.
The library integrates with the broader PyTorch ecosystem including Hugging Face for model and dataset loading, Weights & Biases and TensorBoard for experiment tracking, and EleutherAI's lm-evaluation-harness for model evaluation. Configuration is handled through YAML files and a CLI that supports recipe execution with override parameters, enabling reproducible experiments without writing boilerplate code. With over 5,700 GitHub stars, torchtune has become the preferred choice for teams that want fine-tuning capabilities without sacrificing visibility into the training process.
