Skip to content
Get packs

LocalAI

Open-source engine that runs AI models (text, vision, voice, image, video) locally behind an OpenAI-style API.

GitHub Download v4.12.1
Stars
50.1k
Forks
4.6k
Licence
MIT
Language
Go
Last update
4 days ago
Type
App

What it is

LocalAI is a self-hosted server that runs many kinds of AI models on your own hardware, including language, vision, voice, image and video models. It exposes an API that is compatible with OpenAI, Anthropic and ElevenLabs, so existing tools can point at it instead of a cloud service. The README says it can run without a GPU and keeps data on your own infrastructure.

  • Run a local chat model and call it from code using OpenAI-compatible requests
  • Chat with a running server from the terminal using the built-in agent
  • Load models from the model gallery, Hugging Face, Ollama or OCI registries
  • Generate speech, transcriptions or images through the same API

Great for

  • Developers who want an OpenAI-compatible endpoint running on their own machines or servers
  • Teams that need privacy and want data to stay inside their infrastructure
  • People experimenting with several model types (text, voice, image) behind one API
  • Users with mixed hardware, such as NVIDIA, AMD, Intel, Apple Silicon or CPU-only machines

Not ideal for

  • People who only want a simple desktop chat app; the README focuses on a server and API, though a macOS launcher exists
  • Users with very limited disk space, since models and backends are downloaded on demand and can be large
  • Those who need a fully managed cloud service with no setup work
Step-by-step guide: how to use LocalAIInstall, first run, examples, tips and FAQs
LocalAI alternativesLocalAI vs Ollamallama.cpp vs LocalAILlamaGPT vs LocalAI

How to get it

  1. 1

    Run with Docker

    You'll need: Docker or podman (for the container route)

    Run CPU-only container

    docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

    Run with NVIDIA GPU (CUDA 12)

    docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
  2. 2

    Run it

    • Start the container with the CPU-only or GPU command above
    • Load a model, for example: local-ai run llama-3.2-1b-instruct:q4_k_m
    • Open a second terminal and run: local-ai chat --model llama-3.2-1b-instruct:q4_k_m
    • Send requests to the server on port 8080 using the OpenAI-compatible API
  3. 3

    Get the most out of it

    • Use the model gallery (models.localai.io) to browse available models
    • Load models directly from Hugging Face, Ollama or OCI registries with the run command
    • Enable API key auth and user quotas if several people share the server, as described in the README
    • Use the built-in terminal agent to work with files and commands, with approval prompts for changes
    • Check the GPU Acceleration docs to match the container image to your hardware

Downloads

Source code (.zip)master branch · straight from GitHubRelease v4.12.18 files incl. local-ai-v4.12.1-linux-amd64Official website & docslocalai.io

Strengths

  • Broad model support across text, vision, voice, image and video, per the README
  • Works with several hardware types, including CPU-only setups
  • API compatibility with OpenAI, Anthropic and ElevenLabs reduces code changes

Trade-offs

  • Many features and backends make the project complex to configure; the README is long and the docs are external
  • Backends are downloaded on demand, so first runs may take time and bandwidth
  • The macOS DMG is not signed by Apple, requiring a manual quarantine-removal step

How it compares

ProjectStarsTypeLevelUpdatedBest for
LocalAI (this)50.1kAppintermediate4 days agoDevelopers who want an OpenAI-compatible endpoint running on their own machines or servers
ollama186kAppbeginner4 days agoDevelopers who want a local model API for their own apps
llama.cpp133kLibraryintermediate4 days agoDevelopers who want to run open-weight models locally without a cloud API
one-api37.3kAppintermediate10 months agoDevelopers who want one OpenAI-compatible endpoint for several model providers
hoppscotch80.7kAppbeginner6 days agoDevelopers who want to test REST, GraphQL, WebSocket, or other APIs without a heavy desktop client
GPT-SoVITS64.6kAppintermediate5 days agoHobbyists who want to experiment with cloning a voice they have permission to use

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.