Guide · Open source
How to use Agent Lightning: setup, training and tips
Updated · written with AI from the project's own documentation and checked against our catalogue data
Agent Lightning is an MIT-licensed Python framework from Microsoft for training AI agents with reinforcement learning. Its README says you can keep an agent's existing tools, context and control flow instead of rewriting it for a training system. This guide explains what it does, how to install it, and who it suits, so you can decide whether to try it.
Agent Lightningmicrosoft/agent-lightning · ★ 18.9k · MIT Install options & downloads Quick start
- 1Confirm prerequisites. Make sure you have Python, uv and a CUDA-capable GPU. The README example uses CUDA 13.0.
- 2Clone the repository. Get the agent-lightning source code and open its directory in your terminal.
- 3Install dependencies. Run uv sync from the repository directory to install the Python dependencies.
- 4Set up the verl GPU stack. Run bash scripts/setup_verl.sh 0.8.0 cu130 to install the verl GPU stack as shown in the README.
- 5Read the Quick Start. Follow the official Quick Start to do a local first run of the end-to-end flow.
- 6Try an example. Start with a small example such as Calc-X, which the README says needs only one GPU.
What Agent Lightning does
Agent Lightning trains agents with reinforcement learning (RL). According to the README, agents keep their existing tools, context and control flow while a proxy records their model interactions as training data.
The project describes three components: a Trainer that runs verl and vLLM and updates the policy, an API Gateway that proxies model requests and captures training data, and a Rollout Controller that runs agents locally or as Kubernetes Jobs.
- Trainer: runs verl and vLLM, builds training samples, updates the policy
- API Gateway: proxies model requests and captures training data
- Rollout Controller: runs agents locally or as Kubernetes Jobs
Check your requirements before installing
The installation assumes Python, uv and a CUDA-capable GPU. The README's example runs on a CUDA 13.0 machine. If you do not have GPU access, the setup will not fit your hardware.
You need enough familiarity with reinforcement learning and LLM training to work through the setup and concepts.
- Python
- uv (the package manager used in the install commands)
- CUDA-capable GPU (the example uses CUDA 13.0)
Install Agent Lightning from the repository
The project is installed by cloning the repository and building it. Run the commands below from the repository directory. The README shows these as an example installation on a CUDA 13.0 machine.
For full details, see the Installation Guide in the official documentation.
cd <this-repo>
uv sync
bash scripts/setup_verl.sh 0.8.0 cu130Run the quick start and examples
The documentation includes a Quick Start that covers a local first run and the end-to-end flow. Examples in the README range from small proof-of-concept tasks to coding agents.
The Calc-X example is described as a POC math reasoning example that requires only one GPU. The Coding Agent example is trained with repository tests and is the one tied to the SWE-bench Verified results in the README.
- Calc-X: POC math reasoning, requires only one GPU
- GSM8K: POC grade-school math reasoning
- ScienceWorld: interactive science tasks in a text-based environment
- Search-R1: multi-turn retrieval and reasoning agent
- LLM-in-Sandbox: general agent with computer and code execution tools
- Coding Agent: trained with repository tests
Know the main features and results
The README says the framework is about 3,500 lines of code and lets agents interact with the model through the proxy with zero changes to the agent. It also supports running agents as Kubernetes Jobs without external sandbox services.
The README reports that a Qwen3.5-9B coding workflow improved on SWE-bench Verified using 6K training samples, and that Qwen3.5-35B-A3B improved using 1.8K examples. These are the project's own reported results, and you should check the technical report and documentation before relying on them.
- Works with existing agent harnesses (zero changes, per the README)
- Native Kubernetes support for Jobs
- Full coding agent training example with data cleaning, reward-hacking prevention and training scripts
Decide if it fits your work
Agent Lightning is a training tool, not an end-user app. If you only want to use an AI assistant, it is not the right choice.
Comparable projects named in the source are verl, OpenRLHF and TRL. Choose based on whether you need the agent-harness approach described here and whether your team can run the documented verl and vLLM stack.
- Researchers testing RL on LLM agents
- Engineers who already have an agent and want to improve it with training data
- Teams with GPU access who can run the verl and vLLM stack
What people use it for
- Training a math reasoning agent with the Calc-X example on a single GPU
- Improving a coding agent using repository tests, as in the Coding Agent example
- Training a multi-turn retrieval agent based on the Search-R1 example
- Running agents as Kubernetes Jobs with the Rollout Controller
Frequently asked questions
Is Agent Lightning free to use?
Yes, the repository is licensed under the MIT License, as shown in the README. The license file is linked from the project page.
What do I need to install Agent Lightning?
The README lists Python, uv and a CUDA-capable GPU as prerequisites. The example installation uses a CUDA 13.0 machine.
Can I run Agent Lightning without a GPU?
The source says the examples and installation assume CUDA-capable hardware. Users without GPU resources are not the intended audience.
How do I install Agent Lightning?
Clone the repository, then run uv sync and bash scripts/setup_verl.sh 0.8.0 cu130 from the repository directory. The Installation Guide in the official documentation has more detail.
What are alternatives to Agent Lightning?
The README names verl, OpenRLHF and TRL as comparable projects. Each is a reinforcement learning or training framework you can evaluate against your needs.
Is Agent Lightning a chatbot or AI assistant?
No. The README describes it as a training framework for agents. It is not an end-user application for everyday AI assistance.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Agent Lightning suits researchers and engineers with CUDA GPU access who want to train an existing agent with reinforcement learning without rewriting it. It is a poor fit for people who only want an AI assistant, lack GPU resources, or are new to RL and LLM training. Read the documentation and technical report before committing time to the setup.
Install options & downloads