How to use the PufferLib RL Guide agent skill

What the skill makes the agent do

An agent skill is a folder of instructions and reference files that an AI agent loads when a task matches the skill's description. This one is about PufferLib. Once it is loaded, the agent reads the skill's guidance before answering questions about PufferLib and chooses between three profiles: the native current source build, the published PyPI adaptation, or a pinned historical source snapshot.

The skill's main focus is the Gymnasium-style environment contract: reset returns an observation and info, and step returns observation, reward, terminated, truncated and info. It explains how to validate that contract, how to wrap an existing Gymnasium or PettingZoo environment in the published release, how to run vectorized environments, and how to review checkpoints before loading them.

• Pick the correct PufferLib profile before writing code
• Check the step and reset contract, including the difference between terminated and truncated
• Wrap Gymnasium or PettingZoo Parallel environments in the published release
• Explain vectorization settings and how to benchmark them
• Guide training and evaluation with separate environments and seeds
• Review checkpoints and external logging with safety defaults

Install the skill on your computer

Download the skill ZIP from this page and unzip it. For a personal install, put the unzipped skill folder inside ~/.claude/skills/. For a single project, put it inside the project's .claude/skills/ folder instead, so it only applies to that project.

If you use the Claude apps rather than a local agent, open Settings, go to Capabilities, and upload the skill under Skills.

• Personal install: ~/.claude/skills/
• Project install: .claude/skills/ inside your project
• Claude apps: Settings → Capabilities → Skills, then upload the ZIP

Trigger the skill with natural requests

You do not need a special command. Name PufferLib and describe the task in ordinary language, and the agent should load the skill when the request matches its description. Mention the version or profile you care about if you know it, because the skill asks the agent to choose the version before choosing an API.

Good requests name the task and the constraint. For example, ask the agent to check whether your environment follows the reset and step contract, or to explain which PufferLib profile fits a reproduction of an existing experiment.

• Check whether my PufferLib environment follows the Gymnasium reset and step contract.
• Which PufferLib version should I use to reproduce a pinned 4.0 setup?
• Wrap this Gymnasium environment for PufferLib 3.0 and show me the wrapper.
• Review this checkpoint file and tell me whether it is safe to load.
• Explain how to set num_envs and batch size for vectorized training.

Run the bundled local checks

The skill ships small Python command-line tools. They use only the standard library, need Python 3.10 or newer, and run synthetic environments locally. They do not install PufferLib, use the network, or start a training run. Run them from the skill folder so the relative paths resolve.

The contract validator checks an environment against the Gymnasium-style rules, and the vectorization benchmark times fixed work on a serial backend. Both print strict JSON, which makes their results easy for the agent to read and compare.

• env_contract_validator.py: checks spaces, step/reset behaviour, seeding and cleanup on a synthetic environment
• benchmark_vectorization.py: benchmarks vectorization on a serial backend
• env_template.py, train_template.py and repro_plan.py: templates and dry-run planning helpers

Review checkpoints and external logging safely

Treat checkpoints, pickles, ROMs, maps and environment packages as untrusted until you have verified them. The skill says metadata inspection is not proof of safety. Hash the checkpoint bytes first, and only load them in a sandbox if you trust the source.

If you enable external logging to Weights & Biases or Neptune, the skill's rules say credentials must never be passed on the command line, in config files, in tags or run names, or printed. Keep logging local unless you have deliberately opted in.

• Hash checkpoint bytes before any trusted, sandboxed loading
• Never import an environment by an arbitrary dotted path; the bundled tools accept only allowlisted built-ins and slug identifiers
• Cap steps, environments, workers, memory, disk and wall time
• Keep training and evaluation environments and seeds separate

Install PufferLib itself only after reviewing the source

The skill can guide installation, but it does not replace reviewing the source. The published PyPI release is a source archive, so it needs a native build with a C or CUDA toolchain and specific NumPy and Gymnasium limits. The skill's commands are illustrative and were not executed during review, so run them in a sandbox without credentials or sensitive mounts, and verify the archive digest and resolved dependencies first.

• Use uv with a committed pyproject.toml and uv.lock
• Verify the archive digest before building
• Do not run a remote installer directly from a pipe