Skip to content
Get packs

Guide · Open source

How to use llama_index: install, build an index and query your docs

Updated · written with AI from the project's own documentation and checked against our catalogue data

llama_index is an open-source Python framework for connecting large language models to your own data, such as PDFs, documents, APIs and SQL databases. This guide is for developers and beginners who want a working document question-answering prototype. By the end you will know how to install the core package, add an LLM integration, and build and query a simple vector store index.

LlamaIndexrun-llama/llama_index · ★ 52.8k · MIT

Quick start

  1. 1Check Python is installed. llama_index is a Python library, so you need Python on your machine before you start.
  2. 2Install the core package. Run pip install llama-index-core to install the core framework.
  3. 3Add an LLM integration. Run pip install llama-index-llms-openai or another integration package that matches your provider.
  4. 4Set your API key. Set the OPENAI_API_KEY environment variable to your own key, as shown in the README example.
  5. 5Load your documents. Use SimpleDirectoryReader to load the files in the folder that holds your data.
  6. 6Build and query an index. Create a VectorStoreIndex from the documents and query it to get answers grounded in your data.

What llama_index does

LLMs are trained on publicly available data, so they do not know your private documents. llama_index helps you augment them with your own data. It provides data connectors to ingest existing sources and formats, ways to structure that data into indexes, and a retrieval and query interface that feeds relevant context to an LLM.

  • Data connectors for APIs, PDFs, docs and SQL
  • Indexes and graphs that structure your data for LLMs
  • Retrieval and query engines that return context-augmented answers

Install llama_index with pip

You need Python. There are two ways to start. The starter package llama-index includes core plus a selection of integrations. The customized route installs llama-index-core and then adds only the integration packages you need, which is the approach the README recommends for most projects.

Install core and then add an LLM integration, for example the OpenAI one.

pip install llama-index-core pip install llama-index-llms-openai

Understand the import pattern

Imports that include core refer to the core package. Imports without core refer to an integration package. So core classes come from llama_index.core, while a provider class such as OpenAI comes from its own integration module.

from llama_index.core.llms import LLM from llama_index.llms.openai import OpenAI

Build a simple vector store index and query it

The README shows a minimal example that loads a folder of documents with SimpleDirectoryReader and builds a VectorStoreIndex. Replace the placeholder with your own API key and data directory path. The README notes that examples live in the docs/examples folder and indices in the indices folder.

  • Set your OpenAI API key as an environment variable or in the script
  • Point SimpleDirectoryReader at the folder that holds your documents
  • Build the index from the loaded documents, then query it
import os os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY" from llama_index.core import VectorStoreIndex, SimpleDirectoryReader documents = SimpleDirectoryReader("YOUR_DATA_DIRECTORY").load_data()

Choose your providers

Because the framework is modular, you can swap LLM, embedding and vector store providers. The README lists example integrations for Ollama LLMs and Hugging Face embeddings, each installed as a separate package. There are over 300 integration packages listed on the integrations page.

  • Install only the integrations your app uses
  • Ollama and OpenAI LLM integrations are shown as examples
  • Hugging Face embeddings are shown as an example integration
pip install llama-index-llms-ollama pip install llama-index-embeddings-huggingface

Know the project's current direction

The README states that the company's main focus has shifted toward its commercial LlamaParse document platform. The open-source framework is still available as a toolkit you are welcome to use. Plan accordingly if you depend on the framework for long-term work, and check the documentation for the latest updates, since the README is updated less often than the docs.

What people use it for

  • Question answering over a folder of private PDFs or documents
  • Feeding retrieved context from your own data into an LLM prompt
  • Building agent workflows that query your own data
  • Swapping LLM or embedding providers in a modular RAG pipeline

Frequently asked questions

Is llama_index free to use?

llama_index is open source under the MIT licence, so you can use it freely in your own projects. The README also mentions a separate commercial platform, LlamaParse, which is a different product.

What do I need to run llama_index?

The source lists Python as the prerequisite and installs through pip. You will also need access to an LLM provider, such as the OpenAI integration shown in the README, which requires your own API key.

How do I install llama_index?

Run pip install llama-index-core, then add the integration packages you need, for example pip install llama-index-llms-openai. The starter llama-index package is an alternative that bundles core with a selection of integrations.

Is llama_index still maintained for open source use?

The README says the OSS framework is still available as an open toolkit, but the company's primary focus has shifted toward LlamaParse. Check the documentation for the latest updates.

What are alternatives to llama_index?

Projects often compared with it include LangChain, Haystack, Chroma and LangGraph. Choose based on whether you need a document-focused RAG toolkit, a general orchestration framework, or a vector database.

Is llama_index suitable for non-programmers?

No. It is a Python library and requires writing code. If you need a hosted service without running the framework yourself, the README points to LlamaParse instead.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

llama_index suits Python developers and beginners building retrieval-augmented question answering over private documents, especially if they want swappable providers. It is a poor fit for non-programmers or teams that want a fully managed hosted service, and a standalone parser may be simpler if you only need one-off document parsing.

Install options & downloads