2 tools tagged
Showing 2 of 2 tools
Hot-swap between local LLM models via OpenAI-compatible API
llama-swap is an open-source tool that manages multiple local LLM models behind a single OpenAI-compatible API endpoint. It automatically loads and unloads models on demand, letting developers hot-swap between different models without restarting services. With 3.1K+ GitHub stars, it solves the common pain point of running multiple specialized models on limited hardware.
50x faster LLM gateway with MCP support, built in Go
Bifrost is a high-performance open-source AI gateway built from scratch in Go. Unifies access to 15+ providers and 1,000+ models through a single OpenAI-compatible API with only 11 microsecond overhead per request at 5K RPS — 50x faster than LiteLLM. Features automatic failover, load balancing, semantic caching, and functions as both MCP client and MCP server. Apache 2.0 licensed.
