Skip to content
Get packs

Phoenix AI Observability

Trace, evaluate and monitor LLM applications with the open-source Phoenix platform

Source Download skill
Repository
Orchestra-Research/AI-Research-SKILLs
Stars
14,195
Licence
MIT
Files
3 files

What it does

Teaches the agent how to set up Phoenix, an open-source tool that records what a language-model application does step by step (its calls, inputs, outputs and timing) so developers can debug it. It also covers running quality evaluations on saved data, comparing prompts or models in experiments, and monitoring systems in production. Once installed, the agent can write the code to instrument an app and explain or set up Phoenix for you.

Great for

  • Developers building LLM apps who need to see why a response went wrong
  • Teams that want a self-hosted observability tool instead of a managed vendor
  • ML or data engineers running systematic evaluations and prompt comparisons

Skip it if

  • Users who only want a general ML experiment tracker; Weights & Biases or MLflow may fit better, as the skill itself notes
  • Anyone sending sensitive prompts or user data to a hosted service without checking where traces are stored, since traces can contain full prompt and response text
  • Non-technical users, because setup requires Python and running a server

Try asking

  • “Set up Phoenix tracing for my OpenAI chatbot and show me how to view the traces”
  • “Write an evaluation that checks my RAG app's answers for hallucinations using Phoenix”
  • “Help me run a prompt comparison experiment on a dataset in Phoenix”
  • “Configure a self-hosted Phoenix server backed by PostgreSQL”
Step-by-step guide: how to use Phoenix AI ObservabilityInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 3 original files, the MIT licence and an attribution note.

    phoenix.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip phoenix.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip phoenix.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Review traces and evaluation prompts before enabling them on production data, since they may capture sensitive content
  • Keep API keys in environment variables rather than pasting them into code or chat
  • Ask the agent to start with a local SQLite setup for testing before configuring PostgreSQL

What's inside

  • references/advanced-usage.md
  • references/troubleshooting.md
  • SKILL.md

Redistributed unmodified from Orchestra-Research/AI-Research-SKILLs under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.