Skip to content
Get packs

GKE TPU Metrics Monitoring

Monitor and troubleshoot GKE TPU nodes, node pools and workloads using Cloud Monitoring metrics and PromQL

Source Download skill
Repository
google/skills
Stars
21,782
Licence
Apache-2.0
Files
4 files

What it does

When installed, the agent gets a structured guide for checking the health of Google Kubernetes Engine (GKE) clusters that run TPUs, Google's AI accelerator chips. It tells the agent which Cloud Monitoring metrics to look at (TPU busy time, memory use, node readiness, node pool availability, maintenance or preemption interruptions) and which PromQL queries to run, helping determine whether a slowdown or interruption is caused by the underlying infrastructure. It can also calculate recovery and interruption averages (MTTR and MTBI) over the last 7 days.

Great for

  • Engineers running large AI training or inference jobs on GKE TPU node pools who need to find out whether outages come from hardware maintenance or preemption
  • Site reliability or platform teams tracking TPU node availability and recovery times
  • Anyone who already has GKE system metrics enabled and wants a guided way to query them

Skip it if

  • General GKE workloads without TPUs: the skill is scoped to TPU metrics and says not to use it for non-TPU monitoring
  • Debugging application logic or code errors that are not visible in metrics, since the skill focuses on metric queries rather than logs
  • Users without access to the relevant Google Cloud project, because the queries need live cluster data and valid credentials

Try asking

  • “Check whether node pool tpu-pool-1 in my cluster prod-ai is Running and whether all its hosts are available”
  • “Show me TPU duty cycle and memory usage for the containers in my training job over the last hour”
  • “Which GKE nodes in cluster ml-east are not Ready, and what conditions are they reporting?”
  • “Calculate MTTR and MTBI for my TPU node pools over the last 7 days”
Step-by-step guide: how to use GKE TPU Metrics MonitoringInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 4 original files, the Apache-2.0 licence and an attribution note.

    gke-ai-troubleshooting-tpu-metrics-monitoring.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip gke-ai-troubleshooting-tpu-metrics-monitoring.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip gke-ai-troubleshooting-tpu-metrics-monitoring.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Give the agent your project ID, cluster name, location and node pool name up front so it does not have to guess them.
  • Ask it to confirm the prerequisites in Step 1 (port 8431, JAX and GKE versions, system metrics enabled) before reading metric results, since missing metrics often mean a setup gap rather than a fault.
  • Review any gcloud commands the agent proposes before approving them, because the skill directs it to run gcloud commands in the terminal that query your live cloud project.

What's inside

  • references/attribution.md
  • references/failure_signatures.md
  • scripts/validate_queries.sh
  • SKILL.md

Redistributed unmodified from google/skills under the Apache-2.0 licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.