Guide · Agent skill
How to use the Constitutional AI skill: setup, workflows and tips
Updated · written with AI from the project's own documentation and checked against our catalogue data
The constitutional-ai skill teaches an AI agent Anthropic's method for making models safer without human-labelled harmful examples. It covers a supervised phase where the model critiques and revises its own answers against a written set of principles, and an RL phase where AI-judged preferences train a reward model. After reading this guide you will know how to install the skill, how to ask an agent for help with it, and what you need before running any of the training code.
Constitutional AI TrainingOrchestra-Research/AI-Research-SKILLs · MIT Download the skill Quick start
- 1Download the ZIP. Download the constitutional-ai skill ZIP from this page.
- 2Unzip into a skills folder. Unzip it into ~/.claude/skills/ for personal use, or into .claude/skills/ inside your project.
- 3Or upload it in the Claude apps. In the Claude apps, go to Settings → Capabilities → Skills and upload the skill.
- 4Describe your safety-training goal. Ask the agent in plain language for help with self-critique, revision, or RLAIF, naming your base model and framework.
- 5Set up the environment. Install Python, transformers, torch and trl, and prepare a base model and GPU compute.
- 6Replace placeholders and adapt the code. Swap 'base-model' for your own model and write the helper functions the examples leave out.
- 7Check the labels. Review a sample of AI-generated critiques and preferences before using them to train.
What the constitutional-ai skill makes the agent do
Constitutional AI (CAI) trains models to be harmless through self-critique and AI feedback, without requiring human labels for harmful outputs. The key idea is that a model critiques and revises its own responses using a 'constitution', a set of principles written in plain language.
The skill is a guide for building safety-training pipelines. It is not a tool that makes a chatbot safer in normal conversation, and it does not change how the agent behaves in an ordinary chat.
- Phase 1 (supervised learning): generate responses, have the model critique them against the constitution, revise them, then fine-tune on the revised answers.
- Phase 2 (RLAIF, RL from AI Feedback): sample pairs of responses, have an AI judge which follows the constitution better, train a reward model on those preferences, then run RL training.
- Chain-of-thought critique: prompts that make the model reason step by step about helpfulness, honesty and harmlessness before suggesting a revision.
Install the skill from the ZIP or the Claude apps
Download the ZIP for this skill from this page. The skill's files are a single SKILL.md file.
Option A, a personal install: unzip the download into ~/.claude/skills/ so the skill sits in that folder.
Option B, a project install: unzip the download into the .claude/skills/ folder inside your project.
Option C, the Claude apps: upload the skill under Settings → Capabilities → Skills.
Trigger the skill with natural requests
You do not need a special command. Describe the task in plain language that matches the skill's purpose: safety alignment, reducing harmful outputs, or training without human labels.
Good requests name the stage you need, for example writing a critique prompt, drafting a constitution, or outlining an RLAIF training loop. Mention your base model and framework so the agent uses the right library calls.
- Write a constitution with four principles for a safety fine-tune of my open model, and explain how to test it.
- Help me set up the self-critique and revision loop for prompts that ask for harmful content.
- Show me how to build RLAIF preference pairs and train a reward model with trl.
- Make a chain-of-thought critique prompt that checks helpfulness, honesty and harmlessness.
Understand the example code and its requirements
The code in the skill is a template. The examples use a placeholder model name, 'base-model', so you must replace it with a real base language model you are allowed to train. Several helper functions shown in the examples, such as create_dataset and parse_preferences, are not defined in the skill and you will need to write them.
The skill lists these requirements: Python, the transformers library, torch (PyTorch), the trl library, a base language model, and GPU compute for training. The hardware notes suggest an NVIDIA A100 or H100 for the 7B model, with 1× A100 40GB for the supervised phase and 2× A100 40GB for the RL phase.
- Python with transformers, torch and trl installed
- A base language model to train (you supply it)
- GPU compute; the skill recommends BF16 mixed precision
- Additional inference time for generating critiques and preferences
Safety notes before you run anything
The skill does not run scripts and does not use the network. The code it gives is example code that you copy and adapt, so review it before running it on your own machine or cluster.
The skill is not a runtime safety filter. For filtering live traffic, the skill itself points to NeMo Guardrails or LlamaGuard. AI-judged labels can be noisy or biased, so check a sample of them yourself and consider using several evaluators with a majority vote.
- Replace 'base-model' with a model you have the rights to use.
- Check AI-generated preference labels by hand on a sample before training on them.
- Use a separate guardrail tool for production filtering.
Tips for better results
The skill's common-issue notes give practical fixes. If the model refuses too much, add a principle that asks it to engage thoughtfully and explain concerns. If self-critiques are weak, ask the critique to examine the response for any issues, however minor. If revisions do not improve, run several critique and revision rounds; the skill's example uses three.
If AI preferences are noisy, get judgements from several evaluators and take a majority vote. Choose constitution principles carefully, since they drive every label.
- Refusals too frequent: add a principle favouring thoughtful engagement over refusal.
- Weak critiques: use a stricter critique prompt.
- Revisions not improving: iterate three rounds of critique and revision.
- Noisy preferences: majority vote across multiple evaluators.
for _ in range(3): # 3 rounds of critique/revision
critique = generate_critique(response)
response = generate_revision(response, critique)What people use it for
- Drafting a constitution of principles for a safety fine-tune of an open model.
- Building a supervised self-critique and revision dataset from prompts that elicit harmful answers.
- Generating AI-judged preference pairs and training a reward model for an RLAIF experiment.
- Writing chain-of-thought critique prompts that make a model's safety reasoning explicit.
Frequently asked questions
How do I install the constitutional-ai skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into .claude/skills/ inside a project. You can also upload it in the Claude apps under Settings → Capabilities → Skills.
What do I need to use the constitutional AI skill?
You need Python, the transformers library, torch (PyTorch), the trl library, a base language model you can train, and GPU compute for training. The skill's notes suggest NVIDIA A100 or H100 GPUs.
Is the constitutional-ai skill free?
The skill is published under the MIT licence. This page does not list any price for it, so check the licence text in the repository for the terms.
Will this skill make my chatbot safer in normal chats?
No. The skill guides you in building a safety-training pipeline. It does not change how the agent behaves in a regular conversation.
What is the difference between constitutional AI and RLHF?
Constitutional AI uses AI-generated preferences based on written principles, which the skill describes as scalable with no human labels. RLHF uses human preferences, which the skill describes as more accurate but more expensive.
What should I use instead for filtering harmful content in production?
The skill points to NeMo Guardrails for runtime content filtering and LlamaGuard for a pre-trained moderation model. It is designed for training, not live filtering.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are an ML engineer or researcher building a safety-training pipeline with your own open model and want a clear starting point for self-critique and RLAIF. Pick something else if you only want a safer chatbot answer or need production content filtering, since this skill is a training guide and needs data, compute and a base model.
Download the skill