How to use the Axiom SRE Incident Investigator agent skill

What the axiom-sre skill makes the agent do

Once installed, the agent follows a structured incident process instead of guessing. During a P1 outage it first looks for the cause that can be stopped quickly, for example a recent deploy that should be rolled back, a feature flag that was toggled, or a traffic spike that needs rate limiting. Only after the bleeding is contained does it debug.

For the investigation itself, the skill tells the agent to state hypotheses, design queries that try to disprove them, and label any unverified claim. It is instructed to discover available datasets and names before querying, and to recover by re-running discovery when a query returns a not-found error. The skill also includes a postmortem template for writing up the incident afterwards.

• Stabilize first, then debug, with rollback as a recommended first move for deploy-related outages
• Hypothesis-driven analysis, with conclusions tied to query results
• Discovery before querying for Axiom, Grafana, Pyroscope, Kubernetes and Slack
• A persistent memory system for patterns, queries, facts and incidents
• A postmortem template in reference/postmortem-template.md

How to install the axiom-sre skill

Download the skill ZIP from the page where you found this guide. Then choose one of the two install routes below. If you want the skill for every project on your machine, unzip it into your personal skills folder. If you only want it for one repository, unzip it into that project's .claude/skills/ folder.

If you use the Claude apps instead of a local setup, upload the skill under Settings → Capabilities → Skills. The skill is named axiom-sre and its entry point is SKILL.md, so after unzipping, the folder should contain SKILL.md, a README.md and a reference/ directory.

• Personal install: unzip into ~/.claude/skills/
• Project install: unzip into the project's .claude/skills/ folder
• Claude apps: Settings → Capabilities → Skills, then upload the skill

Run the init script and add your tools

When the skill activates, the agent runs scripts/init first. The first time, init creates the configuration file at ~/.config/axiom-sre/config.toml and creates the memory directories. The init script does not query anything over the network.

If no deployments are configured yet, init prints setup guidance and exits. Add at least one tool (Axiom, Grafana, Pyroscope, Sentry or Slack) to the config file, then run init again. Note that the paths in the skill are relative to the folder containing SKILL.md, so the agent resolves that absolute path before using scripts/init.

How to trigger the skill with natural requests

You do not need special syntax. The skill's description covers incident response, root cause analysis, production debugging and log investigation, so plain requests are enough. Give the agent the symptom, the service and the time window if you know them. The more specific you are, the faster the agent can design useful queries.

Examples of requests that should activate the skill:

• We're seeing a spike in 5xx errors on checkout since 14:30. Help me triage it.
• Find the root cause of the latency increase on orders-api in the last hour.
• Investigate these logs and tell me which hypothesis the data supports.
• Production is slow after the last deploy. What should we roll back or check first?
• Write a postmortem for yesterday's database outage using our template.

What a typical investigation looks like

The agent works in a loop. It discovers what datasets and names exist in each configured tool, checks the schema of any dataset it plans to query, and then checks the values of fields it plans to filter on. Queries are run with the matching tool script, such as axiom-query for Axiom. Results are reported with timestamps, IDs and counts where possible.

The skill sets a strict rule that the agent must not use field names or dataset names from memory. For an Axiom dataset, the first query is a schema check with getschema, shown below. You can ask the agent to show you the discovery output and queries so you can review them.

• Discover the tool's datasets or sources before any query
• Run getschema on each dataset, including _time
• Enumerate low-cardinality fields such as service names, status codes and log levels
• Confirm findings with the user and mark anything unverified

Requirements, network use and safety notes

The skill needs a Bash-capable agent environment because it runs bundled scripts, including init, the discover scripts, axiom-query, grafana-query, pyroscope-diff, kubectl, slack and curl-auth. The README lists jq and curl as tools you need. You also need access to the observability or cluster accounts you want the agent to query, and API credentials for each tool. The skill runs scripts and uses the network.

Credentials are meant to stay out of visible output. The skill instructs the agent to make authenticated requests through scripts/curl-auth, which passes tokens through environment variables, and to never print secrets, write them to files or send them anywhere. Because the skill runs shell scripts and queries live systems, review any change it proposes before a human applies it.

• Requires Bash, jq and curl
• Needs accounts and API credentials only for the tools you enable
• Runs scripts and makes network calls to the systems you configure
• Authenticated calls go through curl-auth so secrets are not shown
• Review proposed rollbacks, config changes and kubectl actions before applying them

Tips for better results

Give the agent a time window and a service name at the start. Ask it to show the discovery output when a name looks wrong. If a query fails with a not-found or unknown dataset error, the skill tells the agent to re-run discovery and retry with the correct name, so you do not need to fix names yourself.

Use the memory system and postmortem template after each incident. The skill is designed to save useful patterns and facts as it learns them, so the next investigation starts with more context.

• Start with the symptom, the service and a time window
• Enable only the tools you actually need, since discovery runs per tool
• Ask the agent to label any claim it has not verified
• Use the postmortem template to close the incident