LocalAI

LocalAI is an open source, MIT-licensed AI runtime that runs text, voice, vision, image, video, 3D and agent workloads locally on hardware you own, keeping data on your own machines. It exposes an OpenAI-compatible API as a drop-in replacement for upstream services, and also speaks the Anthropic, Ollama and ElevenLabs APIs, so existing clients typically need only a URL change. It supports CPU-only operation alongside GPU acceleration across x86_64, ARM64, CUDA, ROCm, SYCL, Metal and Vulkan.

The runtime uses swappable inference backends including llama.cpp, vLLM, SGLang and MLX, selected per model through configuration. Capabilities include reasoning and tool calling, realtime voice over WebRTC with transcription, diarization and speech synthesis, vision and detection, image, video and music generation, and agents with MCP, skills and RAG. A model gallery offers one-click local installation, and distributed mode adds smart routing, VRAM-aware placement, autoscaling and failover across multiple machines. APEX quantization builds reduce model size for consumer hardware.

LocalAI is aimed at developers and teams wanting to run AI on their own infrastructure, from laptops to multi-machine GPU setups. It is packaged as open source software under the MIT license, with a core binary that pulls backends in as needed.