Skip to content
Get packs

TransformerLens Interpretability

Lets an agent inspect and edit transformer model internals for mechanistic interpretability research.

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
4 files

What it does

Teaches the agent how to use the TransformerLens Python library to open up language models like GPT-2 and look inside them. The agent can cache internal activations, study attention patterns, run activation patching experiments to find which parts of a model cause a behaviour, and do direct logit attribution. The skill gives code patterns, a list of supported models, and a checklist for these experiments, so the agent writes correct research code instead of guessing at the API.

Great for

  • Researchers studying how transformer models compute specific behaviours such as indirect object identification
  • Students learning mechanistic interpretability who want guided, runnable examples
  • ML engineers who need to cache and inspect intermediate activations in GPT-style models

Skip it if

  • Non-transformer architectures, which the skill itself says should use nnsight or pyvene instead, because TransformerLens targets transformer models
  • Production safety decisions or auditing a deployed model without expert review, since the skill only supplies research techniques and does not validate findings
  • Users without a Python environment and GPU access, since model loading and patching on larger models need substantial compute

Try asking

  • “Run activation patching on GPT-2 small to find which layers and positions change the prediction from Rome to Paris”
  • “Cache the residual stream and attention patterns for a prompt and plot a layer-by-position heatmap”
  • “Reproduce the indirect object identification circuit analysis on gpt2-small with direct logit attribution”
  • “Explain what the HookedTransformer activation cache keys like resid_post and pattern mean”
Step-by-step guide: how to use TransformerLens InterpretabilityInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 4 original files, the MIT licence and an attribution note.

    transformer-lens.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip transformer-lens.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip transformer-lens.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Start with gpt2-small, which is small enough to run on a laptop or free GPU, before trying larger models
  • Use names_filter in run_with_cache to cache only the activations you need, which saves a lot of memory
  • Keep your Hugging Face token out of shared code; the example sets it as a plain string, so use an environment variable or secret store instead

What's inside

  • references/api.md
  • references/README.md
  • references/tutorials.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.