Skip to content
Get packs

Arize Prompt Optimization

Improves LLM prompts using production traces, evals and annotations via the ax CLI

Source Download skill
Repository
github/awesome-copilot
Stars
40,703
Licence
MIT
Files
3 files

What it does

Once installed, the agent can pull real LLM call records (spans) from an Arize project, extract the prompts that were actually sent to a model, and look at performance signals such as human grades and automated eval explanations. It then runs a data-driven loop to suggest and test prompt improvements instead of guessing. It relies on the ax command-line tool to talk to Arize.

Great for

  • Engineers maintaining an LLM app who already log traces to Arize
  • Prompt engineers who want evidence from production failures rather than trial and error
  • Teams with human annotations or LLM-as-judge evals they want to act on

Skip it if

  • People without an Arize account or existing trace data, since the skill has nothing to analyse (requires setup first)
  • Quick one-off wording tweaks to a prompt with no production data, where a plain chat is simpler
  • Anyone expecting it to work without the ax CLI and API credentials configured

Try asking

  • “Optimize the system prompt for my support bot in the my-workspace space using the failing traces from last week”
  • “Extract the current prompt from the LLM spans in our summarizer project and reconstruct it as a messages array”
  • “Find LLM spans with low eval scores and explain why the prompt is underperforming”
  • “Improve the prompt for our ChatCompletion call based on human annotation labels”
Step-by-step guide: how to use Arize Prompt OptimizationInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    arize-prompt-optimization.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip arize-prompt-optimization.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip arize-prompt-optimization.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Run ax spaces list and ax projects list first so the agent has the exact space and project names to target
  • Provide annotated or eval-scored traces; the eval explanation fields are the most useful signal for improvements
  • Review every proposed prompt change before deploying it to production, since the loop is data-driven but not validated by you

What's inside

  • references/ax-profiles.md
  • references/ax-setup.md
  • SKILL.md

Redistributed unmodified from github/awesome-copilot under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.