Skip to content
Get packs

Arize Experiment Runner

Create, run, export and compare AI model evaluation experiments on Arize using the ax CLI.

Source Download skill
Repository
github/awesome-copilot
Stars
40,703
Licence
MIT
Files
3 files

What it does

Once installed, the agent knows how to use the Arize ax command-line tool to manage experiments: listing and inspecting them, exporting every run with its model outputs and scores to a JSON file, and helping you compare how different models perform on the same dataset. It is written as step-by-step guidance for the agent, including how to fix common setup errors and a rule to never invent model outputs or scores.

Great for

  • ML engineers who evaluate or benchmark LLM or model versions on Arize
  • Teams that want to export experiment results and compare runs
  • People already using Arize who want the agent to drive the ax CLI

Skip it if

  • People without an Arize account or the ax CLI, since the skill cannot work without them
  • Quick one-off model comparisons that do not need a tracked evaluation platform
  • Anyone who expects the skill to hold credentials or call models by itself without their own API keys and setup

Try asking

  • “Export all runs from my experiment 'summarizer-v2' on dataset 'support-tickets' to a JSON file”
  • “List the 20 most recent experiments for the dataset 'qa-benchmark'”
  • “Compare correctness scores between my two latest experiments on the same dataset”
  • “Create an experiment from these run results and link it to my dataset”
Step-by-step guide: how to use Arize Experiment RunnerInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    arize-experiment.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip arize-experiment.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip arize-experiment.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Give the dataset and experiment names rather than IDs; names are easier to remember and the skill prefers them.
  • Ask the agent to show the list of runs or a summary before comparing, so you can check the numbers come from real exports.
  • Keep your API key in ax profiles rather than pasting it into chat; the skill itself forbids reading .env files or searching for credentials.

What's inside

  • references/ax-profiles.md
  • references/ax-setup.md
  • SKILL.md

Redistributed unmodified from github/awesome-copilot under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.