How to use AnythingLLM: install, add documents and chat

What AnythingLLM does and who it suits

AnythingLLM is described in its README as an all-in-one AI application that lets you chat with your own documents and use AI agents. The README says it includes built-in agents, multi-user support, vector databases, and document pipelines, and is available as a desktop app for Mac, Windows, and Linux.

It fits people who want a private ChatGPT-style assistant over their own PDFs and text files, developers who want to try many LLM providers from one interface, and anyone who wants to run local models on their CPU or GPU without much setup.

Install AnythingLLM on Mac, Windows or Linux

The simplest route is the desktop app. The README links to a download page at anythingllm.com/download. The project lists no prerequisites and no install commands in its README, so you do not need to set up a terminal workflow for the desktop version.

If you need several users on one instance, the README says multi-user support and permissioning are available in the Docker version only. Docker setup instructions are linked from the project docs at docs.anythingllm.com.

Choose a local or cloud language model

AnythingLLM can use a local model or a cloud model. The README says local models run on your CPU or GPU with no extra setup through a built-in engine, with NVIDIA via CUDA and AMD and Intel via Vulkan. Cloud options include OpenAI, Anthropic, Google Gemini Pro, Mistral, Groq and many others.

Output quality depends on the model you pick, so try a few. The README lists providers such as Ollama, LM Studio and LocalAI as supported, so if you already run one of those, you can point AnythingLLM at it.

• Local: built-in engine, Ollama, LM Studio, LocalAI, any llama.cpp compatible model
• Cloud: OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Google Gemini Pro, Mistral, Groq, Cohere and others

Add your documents and chat with them

AnythingLLM supports multiple document types, including PDF, TXT and DOCX. The README describes an intuitive chat UI with drag-and-drop uploads and source citations, so answers can be traced back to the files they came from.

Upload your files into the app and ask questions about them in chat. The README notes built-in optimizations for large document sets.

Use agents, memories and scheduled tasks

The README lists built-in agents, a no-code AI agent builder, custom AI agents, and agents inside a workspace that can browse the web. It also lists automatic and user-managed memories, where the model can remember important information about you or your workspace.

Scheduled Tasks runs recurring tasks or prompts on a cron schedule with full agent capabilities. Dynamic Model Routing can send chats to a provider and model based on rules you define.

Set up multiple users and the developer API

For teams, the README says multi-user instance support and permissioning are available in the Docker version only, along with custom color themes, white-labeling and the embeddable chat widget. The README also mentions a Full Developer API for custom integrations.

The project also links a hosted instance as a paid option. Check the project docs for current details on these features.