小智 (Xiaozhi)
Self-hosted backend server that lets ESP32 voice-assistant devices talk to AI models.xinnan-tech/xiaozhi-esp32-server
- Stars
- 11.1k
- Forks
- 3.8k
- Licence
- MIT
- Language
- JavaScript
- Last update
- 4 days ago
- Type
- App
What it is
This is the server side for the open-source xiaozhi-esp32 hardware project, which runs voice assistants on ESP32 chips. It handles speech recognition, large language model replies, speech synthesis, and device control, so you can run your own instance instead of relying on someone else's. The README says it is written in Python, Java and Vue and supports MQTT+UDP, WebSocket, and MCP access points.
- Run a self-hosted voice assistant that answers questions using a chosen LLM
- Control home appliances by sending MCP commands from the management console to a device over MQTT
- Add a knowledge base through RAGFlow so the model can look up information before answering
- Use the performance tester to compare response speeds of ASR, LLM, VLLM and TTS providers
Great for
- Owners of ESP32 voice-assistant hardware who want their own backend
- Hobbyists who already connected a device to the official xiaozhi backend and want to self-host
- Developers experimenting with voice AI pipelines on low-cost hardware
- Users who want a local-only short-term memory and can run speech recognition on their own machine
Not ideal for
- Production or public-internet use: the README says the project has not passed a network security review and should not be used in production
- People without ESP32 hardware, since the server is built to serve that device project
- Anyone who wants a turnkey, no-configuration setup, since it needs API keys and component configuration
How to get it
- 1
Run with Docker
You'll need: Docker (for the Docker deployment route described in the README) · Git
Clone the repository
git clone https://github.com/xinnan-tech/xiaozhi-esp32-server - 2
Run it
- Choose the minimal install (no database, config-file storage) or the full-module install based on the README table
- Follow the Docker or source-deployment guide linked in the README (docs/Deployment.md or docs/Deployment_all.md)
- Configure API keys for the speech, LLM and TTS providers you chose
- Point your ESP32 device at the server's OTA and WebSocket addresses
- 3
Get the most out of it
- Use the streaming configuration (XunfeiStreamASR, qwen-flash, HuoshanDoubleStreamTTS) for faster responses, as the README describes
- Run the performance_tester.py script to measure which configured models respond fastest
- Enable the RAGFlow knowledge base integration so answers can draw on your own documents
- Use the MCP access point and server-side MCP to extend what the assistant can do
- Write custom plugins; new plugin files load on the next device connection, but changes to loaded plugins need a restart
Downloads
Strengths
- Broad feature list in the README: streaming speech, voiceprint recognition, memory, knowledge base and multiple tool-calling options
- Offers a free starter configuration and a streaming configuration, with a documented performance-testing method
- MIT license, with a web admin console and multi-language UI support according to the README
Trade-offs
- The README explicitly warns it is not production-ready and has not passed security testing
- Running local FunASR speech recognition is memory-heavy; full-module setup also needs a database
- Depends on third-party API services whose quality and terms are outside the project's control
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| xiaozhi-esp32-server (this) | 11.1k | App | advanced | 4 days ago | Owners of ESP32 voice-assistant hardware who want their own backend |
| xiaozhi-esp32 | 32.5k | App | intermediate | 5 days ago | Hobbyists who want a voice assistant on a supported ESP32 development board |
| n8n-mcp | 23.2k | AI model / agent | intermediate | 4 days ago | People already using n8n who want an AI assistant to draft or check workflows |
| ida-pro-mcp | 13.3k | App | intermediate | 1 months ago | Reverse engineers who already use IDA Pro 8.3 or newer and want an AI assistant to help with analysis |
| fastapi_mcp | 12k | Library | intermediate | 12 months ago | Python developers who already have a FastAPI app and want AI assistants to use it |
| RuView | 102k | Other | advanced | 4 days ago | Hobbyists who want presence detection in a room without cameras |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.