Skip to content
Get packs

Guide · Agent skill

How to use the AI Security Assessment agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

The ai-security skill teaches an AI agent to assess AI and LLM-based systems for weaknesses such as prompt injection, jailbreak attempts, training-data leakage (model inversion), data poisoning and agent tool abuse. It bundles a Python scanner, scripts/ai_threat_scanner.py, that checks test prompts against regex signatures and labels findings with MITRE ATLAS technique IDs. This guide explains what the skill does, how to install it, how to trigger it, and what its limits are.

AI Security Assessmentalirezarezvani/claude-skills · MIT

Quick start

  1. 1Download and unzip. Download the ZIP from the directory page and unzip the ai-security folder.
  2. 2Place the skill. Put the folder in ~/.claude/skills/ for personal use, or in a project's .claude/skills/ folder. You can also upload it under Settings → Capabilities → Skills in the Claude apps.
  3. 3Confirm you have Python 3. The scanner needs Python 3 to run scripts/ai_threat_scanner.py.
  4. 4Prepare a test file. Optionally write your test prompts as a JSON array of strings or objects with a prompt key.
  5. 5Ask for an assessment. Describe the AI system, access level and risks you care about in a plain request.
  6. 6Run the scanner. Run the scanner with your target type and access level, adding --authorized for gray-box or white-box runs.
  7. 7Check the exit code and act. Use exit codes 0, 1 and 2 to triage, then add guardrails and retest.

What agent skills are and what this one makes the agent do

An agent skill is a folder with a SKILL.md file that tells an AI agent how to handle a particular kind of task. The ai-security skill is methodology plus tooling: it tells the agent how to structure an AI security assessment and how to recommend guardrails.

The skill covers prompt injection, jailbreak assessment, model inversion risk, data poisoning risk and agent tool abuse. It supports LLMs, classifiers and embedding models. It is about AI/ML systems specifically, not general application security.

  • Scans prompts for injection signatures and reports matches
  • Maps findings to MITRE ATLAS technique IDs
  • Scores model-level risks and recommends guardrail controls

Install the ai-security skill

Download the ZIP for the skill from the directory page and unzip it. For a personal install, place the unzipped folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.

You can also upload the skill in the Claude apps. Go to Settings, then Capabilities, then Skills, and upload it there.

  • Personal: ~/.claude/skills/
  • Project only: .claude/skills/ inside your project
  • Claude apps: Settings → Capabilities → Skills

Trigger the skill with natural requests

The skill's description says to use it when assessing AI/ML systems for prompt injection, jailbreak vulnerabilities, model inversion risk, data poisoning exposure or agent tool abuse. Ask the agent for that kind of assessment in plain language and it should pick up the skill.

Name the target and the kind of risk you care about. Say whether it is a black-box, gray-box or white-box review, and whether you have authorization, because the skill requires written authorization for gray-box and white-box testing.

  • "Assess my customer support chatbot for prompt injection before release."
  • "Review this RAG pipeline for indirect injection from retrieved web pages."
  • "Check our agent's tool permissions for tool abuse via injection."

Run the scanner on your prompts

The scanner is a Python 3 script. Run it from the skill folder with the commands below. Add --test-file to scan your own prompts. The test file is a JSON array of strings or objects with a "prompt" key. Gray-box and white-box runs need the --authorized flag.

Exit codes tell you the result: 0 means low risk with no critical findings, 1 means medium or high risk findings, and 2 means critical findings or missing authorization for invasive access levels.

  • Test file entries can be plain strings or objects like {"prompt": "..."}
  • --list-patterns shows every injection signature name with its ATLAS ID
python3 scripts/ai_threat_scanner.py \ --target-type llm \ --access-level gray-box \ --test-file prompts.json \ --authorized --json

Understand what the scanner can and cannot tell you

The scanner uses static signature matching on text. It checks inputs before they reach the model and does not test a live model. The injection score (0.0 to 1.0) is the proportion of in-scope signatures matched across the tested prompts. The skill says a score above 0.5 warrants immediate guardrail deployment.

A clean result is not evidence of security. The regex signatures miss many attacks, and the scanner reports only signature matches.

  • Signatures include direct_role_override, indirect_injection, jailbreak_persona, system_prompt_extraction, tool_abuse and data_poisoning_marker
  • Treat any external content (web pages, documents, email, API responses) as untrusted input

Use the skill for common assessment tasks

For a pre-release check, feed known jailbreak templates through the scanner. The skill says any template scoring critical needs guardrail remediation before the model is exposed to untrusted users.

For model inversion, the skill rates risk by access level: white-box critical (0.9), gray-box high (0.6), black-box low (0.3). It recommends monitoring inference logs for high-volume querying from one identity.

What people use it for

  • Engineers adding an LLM feature running a first-pass check of prompts and guardrails before release.
  • Security teams running a documented, authorized review of an AI assistant or agent.
  • Teams building RAG or web-browsing agents assessing indirect injection risk from retrieved content.
  • Checking a classifier for adversarial robustness with a white-box run.

Frequently asked questions

Is the ai-security skill free and what is its licence?

The skill is published under the MIT licence, as listed on the directory page. Check the licence file in the repository for the exact terms before you redistribute it.

What do I need to run the ai-security skill?

You need Python 3 to run scripts/ai_threat_scanner.py. An optional JSON test file of prompts helps, and gray-box or white-box testing requires written authorization.

Does the scanner test my live model?

No. The scanner uses static signature matching on text and assesses inputs before they reach the model. It does not require live model access.

Does a clean scan mean my AI system is secure?

No. The regex scanner misses many attacks and reports only signature matches, so a clean result is not evidence of security.

Why won't the scanner run gray-box or white-box tests?

The skill requires written authorization for those access levels. Add the --authorized flag only when you have that permission, and do not test systems you do not own or have permission to test.

Which skill should I use for general web or infrastructure security?

The skill says general application security is handled by security-pen-testing, behavioral anomaly detection in infrastructure by threat-detection, and adversary simulation by red-team.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use this skill if you are adding an LLM feature or running an authorized review of an AI assistant or agent and want a structured, first-pass check of prompts and guardrails. Pick something else if you need to certify a system as safe, since the regex scanner only reports signature matches, or if your work is general application or infrastructure security.

Download the skill