Guide · Agent skill
How to use the Caveman Optimize Evaluator agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The caveman-optimize skill is an agent skill for AI coding agents that helps you review one report-only observation from the Caveman CLI, a tool that tracks how an AI setup uses tokens. It walks the agent through picking an observation, finding a specific place in your code that could cause it, proposing one small change, and testing that change against a baseline on identical inputs. This guide explains what an agent skill is, how to install this one, how to trigger it, and what it needs before you start.
Caveman Optimize EvaluatorJuliusBrussee/caveman · Apache-2.0 Download the skill Quick start
- 1Confirm prerequisites. Make sure the Caveman CLI is installed and you are logged in, and that your project is a git repository with fixed test fixtures and a quality check.
- 2Download and unzip the skill. Download the ZIP from this page and unzip it so SKILL.md sits in a caveman-optimize folder.
- 3Place the skill folder. Move the folder into ~/.claude/skills/ for personal use or into the project's .claude/skills/ folder for one repository.
- 4Ask for an evaluation in plain language. Request an inspection of your Caveman optimization report and say you want to approve any change first.
- 5Choose one observation. Review the supported observations the agent lists and pick one explicitly. The agent will not rank them for you.
- 6Approve the candidate and eval design. Review the single proposed change and the paired baseline and candidate eval, then approve or adjust it.
- 7Read the report and decide. Check the quality result and the reported decision of keep, reject, or inconclusive before adopting anything.
What is an agent skill, and what does caveman-optimize do?
An agent skill is a folder containing a SKILL.md file that tells an AI agent how to handle a specific kind of task. When the task matches the skill's description, the agent follows the documented steps instead of improvising.
The caveman-optimize skill handles one job: turning a Caveman optimization observation into a candidate change with a paired baseline evaluation. It treats observations as diagnostic input only. They describe recorded aggregate patterns, not guaranteed savings or safe code changes.
- Lists the supported observations from the Caveman CLI and preserves their titles and text verbatim.
- Asks you to choose one observation. It does not rank them for you.
- Looks for a specific code mechanism that could produce the observed pattern and cites the callsite.
- Proposes one minimal change and an eval design, then waits for your approval.
- Reverts the change if quality drops or the result is inconclusive.
What you need before you start
The skill stops at its first step unless you have a logged-in Caveman CLI session. It also needs a git repository with fixed test fixtures and a quality check, because a paired evaluation cannot be fair without them.
- Caveman CLI installed and logged in. The skill runs caveman opportunities list to read observations.
- A git repository with fixed test fixtures.
- A quality check you can run to confirm task outcomes stay acceptable.
- Network access: the skill uses the network, and it runs scripts.
caveman opportunities listInstall the caveman-optimize skill
Download the ZIP for this skill from this page and unzip it. The skill is a single file, SKILL.md, so you can place the folder in one of two locations depending on how you want to use it.
For a personal setup available across projects, put the unzipped folder in ~/.claude/skills/. For a single project, put it in that project's .claude/skills/ folder so it applies only there.
In the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.
- Personal: ~/.claude/skills/ (available in every project)
- Project only: .claude/skills/ inside your repository
Trigger the skill with a natural request
You do not need a special command. The skill's description says it applies when you ask to inspect or evaluate a Caveman optimization report, so ask in plain language that names the report or observation.
Expect the agent to read the observations first and then stop to ask you which one to pursue. Nothing in the code changes until you approve a candidate and its eval design.
- "Inspect my Caveman optimization report and show me the supported observations."
- "Evaluate one of the Caveman observations against my repo, but ask me before changing anything."
- "Read the current Caveman observations and help me pick one to test."
Follow the approval steps
The workflow has clear checkpoints. The agent presents each observation with its id, exact title, exact observation text and last_seen_at. You then choose one. Only after that does the agent search the code for a mechanism and propose a candidate.
Before editing, the agent proposes a paired eval that runs baseline and candidate on identical fixed inputs. It records the quality check, the measurement method, the fixture and command used, and any confounder. You approve or change that design before any edit is made.
After the paired run, the agent reports the result with a decision of keep, reject, or inconclusive. A rejected or inconclusive candidate is reverted.
Understand what the skill will not do
The skill is deliberately conservative. It does not claim dollar savings, does not invent figures, and does not create Caveman experiments or proposals. It also does not mark opportunities implemented or change their lifecycle.
Some observation ids are retired and are never selected. If the only actionable-looking item is unlabeled-traffic, the skill hands off to caveman-discover, because labeling is not a profile optimization.
- Never converts token or byte reduction into dollars without provider-complete accounting.
- Treats a local paired result as evidence only for the stated candidate on the stated fixture.
- Stops rather than guesses if the CLI is unavailable or authentication fails.
What people use it for
- Testing whether a context-window observation points to a real mechanism in your prompt setup, using a fixed fixture for both arms.
- Checking a tool-output size observation against a repository's focused code checks before changing how tool results are returned.
- Documenting an evidence-first decision for a team that wants a written record before changing agent prompts or tool configuration.
- Deciding whether an exploration-load observation is worth a small, reversible change, with the result reported as observation-only rather than savings.
Frequently asked questions
Is the caveman-optimize skill free?
This page does not state pricing or licence terms beyond the licence listed for the skill, which is Apache-2.0. Check the repository's licence file for the full terms before using or redistributing it.
What do I need to install caveman-optimize?
Download the ZIP from this page, unzip it, and place the folder in ~/.claude/skills/ or a project's .claude/skills/ folder. You can also upload it under Settings → Capabilities → Skills in the Claude apps. The skill itself needs a logged-in Caveman CLI and a git repository with fixed fixtures and a quality check.
Why does the skill stop without making changes?
It stops when the Caveman CLI is unavailable, authentication fails, the observations are missing, there is no fixed fixture or quality check, or you have not chosen an observation. These stops are intentional so that no code is changed without a fair comparison and your approval.
Does this skill tell me how much money I will save?
No. The skill never claims dollar savings. Token or byte reductions are reported as local, paired results on a stated fixture, and converting them to dollars requires provider-complete, same-request accounting.
How is this different from caveman-discover?
caveman-optimize works on report-only profile observations. If the only actionable item is unlabeled-traffic, the skill hands off to caveman-discover, because labeling traffic is not a profile optimization.
Can the skill apply changes automatically?
No. It requires explicit operator approval for the observation, the candidate change and the eval design. It reverts the candidate if quality regresses or results are inconclusive.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use caveman-optimize if you already run the Caveman CLI and want a strict, documented process for testing one token-efficiency idea at a time, with approval at each stage. Pick a different approach if you want quick automatic cost cuts, or if your repository has no fixed fixtures or quality checks, since the skill will stop rather than guess.
Download the skill