Skip to content
Get packs

Guide · Agent skill

How to use the PluginEval evaluation-methodology agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

The evaluation-methodology skill is a reference guide that teaches your AI agent how PluginEval measures the quality of agent skills and plugins. It covers the scoring layers, what each dimension means, how composite scores and badges are calculated, and how to read a low score. This guide shows you how to install it, how to trigger it with normal requests, and what it can and cannot tell you. You do not need prior experience with agent skills to follow it.

PluginEval Evaluation Methodologywshobson/agents · MIT

Quick start

  1. 1Download the ZIP. Download the evaluation-methodology skill ZIP from this page.
  2. 2Unzip into a skills folder. Unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project.
  3. 3Or upload in the Claude apps. In the Claude apps, go to Settings → Capabilities → Skills and upload the skill.
  4. 4Install the PluginEval CLI. Install the plugin-eval CLI, which the skill needs to run scoring commands. The skill itself only provides explanation.
  5. 5Ask a natural question. Ask about a score, dimension, badge or threshold in plain language, such as why a skill scored low on triggering accuracy.
  6. 6Run a quick static score. Run plugin-eval score on your skill with --depth quick to get a deterministic static result to compare against.

What the skill makes the agent do

The skill gives the agent background knowledge about PluginEval, a tool that scores a skill or plugin from 0 to 100. With this skill loaded, the agent can explain what a score is made of, what a dimension such as triggering_accuracy means, and why a particular score came out low.

The skill only explains. It does not run scoring itself. To produce real scores you need the PluginEval CLI (plugin-eval) installed separately.

  • Explains the three scoring layers: static, LLM judge and Monte Carlo.
  • Explains each dimension and its weight in the composite score.
  • Explains how badges (Bronze, Silver, Gold, Platinum) and letter grades are assigned.
  • Suggests fixes for common anti-patterns, using the bundled rubric and fix references.

Install the skill from a ZIP file or the Claude apps

Download the ZIP for this skill from this page and unzip it. To make it available for all of your projects, unzip it into ~/.claude/skills/. To make it available for one project only, unzip it into that project's .claude/skills/ folder.

If you use the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.

  • Personal install: ~/.claude/skills/
  • Project install: .claude/skills/ inside your project folder
  • Apps: Settings → Capabilities → Skills

Trigger the skill with natural requests

You do not need a special command. Ask the agent in plain language about the topic. The skill's description lists the situations it is meant for, so requests that match those situations are the ones most likely to load it.

Good requests name the score, dimension or badge you care about, for example asking why a skill scored poorly on triggering accuracy or what a Gold badge requires.

  • "Why did my skill get a low triggering_accuracy score?"
  • "How is the PluginEval composite score calculated?"
  • "What minimum score should I set as a threshold for my marketplace?"
  • "How should I explain PluginEval badges to a partner like Neon?"

Use it to improve a skill's score

When a score is low, start with the static layer. It is the only layer that gives the same result on every run. Check the anti-pattern flags in the JSON output, since each one cuts the composite by 5%, down to a floor of 50%.

For triggering accuracy at quick depth, a common fix is to add a trigger phrase such as "Use this skill when" followed by several specific contexts in the description.

  • Look at layers[0].anti_patterns in the JSON output when a score drops after you add content.
  • Keep hard MUST, ALWAYS and NEVER rules for safety and correctness, and explain the reason for the rest.
  • Use the static layer for comparisons you want to repeat, because judge scores change between runs.
plugin-eval score ./path/to/skill --depth quick --output json --threshold 70

Know the limits before you rely on the scores

The LLM judge and Monte Carlo layers are experimental and have not been checked against human labels. Treat their numbers as rough signals rather than proof of quality. The skill's own files say the same thing.

The skill is for evaluating existing skills. It is not a guide to writing a new skill from scratch.

  • Good fit: plugin authors investigating a low score, marketplace maintainers setting thresholds, and developers improving triggering or orchestration fit.
  • Not a fit: people who want to author a new skill, or anyone who needs judge or Monte Carlo scores as validated proof.

What people use it for

  • Finding out why a plugin author's skill scored below a marketplace threshold.
  • Deciding whether a low triggering_accuracy score calls for a clearer description.
  • Setting a minimum score for a marketplace and documenting what each badge means.
  • Preparing a plain-language explanation of quality badges for an external partner.

Frequently asked questions

How do I install the evaluation-methodology skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project. In the Claude apps, you can upload it under Settings → Capabilities → Skills.

Does the skill run PluginEval scores for me?

No. The skill only provides explanation. To run scoring commands you need the PluginEval CLI (plugin-eval) installed separately.

What does the skill require?

It requires the PluginEval CLI to run scoring commands. The skill does not run scripts and does not use the network.

Is PluginEval free to use?

The source for this skill is published under the MIT licence. This guide does not state pricing for the PluginEval CLI, so check its repository for its terms.

Can I use this skill instead of writing my own evaluation rules?

It is a reference for how PluginEval measures quality, not a replacement for your own standards. Use it to understand scores and thresholds, and set your marketplace's rules yourself.

Why do my judge scores change every time I run them?

The judge writes new test prompts and tasks on each run, so its scores vary. Use the static layer for comparisons you want to repeat, since it gives the same result every time.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use this skill if you are a plugin author trying to understand a low PluginEval score, or a marketplace maintainer setting thresholds and explaining badges. Skip it if you want to write a new skill from scratch, or if you need judge or Monte Carlo scores as validated proof of quality, since those layers are experimental.

Download the skill