Skip to content
Get packs

Guide · Agent skill

How to diagnose a stuck TPU slice MIG with an agent skill

Updated · written with AI from the project's own documentation and checked against our catalogue data

This agent skill teaches an AI agent how to investigate a Google Kubernetes Engine (GKE) TPU slice that fails to create, or a JobSet that stays pending, when the Compute Engine managed instance group (MIG) behind the node pool is still updating or unstable. It is for platform and ML engineers who see SliceCreationFailed errors. After following this guide you will be able to install the skill, ask your agent to run the investigation, and know what the agent will and will not do.

Diagnose stuck TPU slice MIGsgoogle/skills · Apache-2.0

Quick start

  1. 1Confirm prerequisites. Check that gcloud and kubectl are installed and authenticated, the project has active billing, and the three APIs are enabled.
  2. 2Check IAM roles. Confirm you hold Logs Viewer, Compute Viewer and Kubernetes Engine Viewer on the project.
  3. 3Download and unzip the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder.
  4. 4Or upload in the Claude apps. In the Claude apps, open Settings → Capabilities → Skills and upload the skill.
  5. 5Describe the symptom. Ask the agent in plain language, naming the project, cluster, location, node pool and issue time.
  6. 6Review the findings. Read the likely causes and linked documentation, then apply any fix yourself.

What the skill makes the agent do

Agent skills are folders of instructions that an AI agent loads when a request matches the skill's description. This one covers GKE TPU Dynamic Slice creation and JobSet scheduling failures caused by backing MIGs that are not stable (status.isStable: false).

The agent checks the Slice custom resource status first for a SliceCreationFailed reason, then checks whether the underlying MIGs are stable, and reports likely causes. The skill tells the agent to run only read-only commands and to give fixes to you rather than apply them.

  • Reads Slice status and identifies the affected partitions, nodes and MIG.
  • Checks MIG stability and instance progress.
  • Inspects Cloud Audit Logs for MIG errors.
  • Reports likely causes and links the relevant documentation section for any recommended fix.

Install the skill from the ZIP download

Download the ZIP for this skill from this page and unzip it. For a personal setup, place the unzipped folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.

If you use the Claude apps instead, upload the skill under Settings → Capabilities → Skills.

  • Personal: ~/.claude/skills/gke-ai-troubleshooting-tpu-slice-mig-stuck-updating/
  • Project: .claude/skills/gke-ai-troubleshooting-tpu-slice-mig-stuck-updating/
  • Claude apps: Settings → Capabilities → Skills, then upload

Set up what the skill requires

The skill depends on the gcloud CLI and kubectl, both installed and authenticated to a Google Cloud project. The project needs an active Cloud Billing account and the container.googleapis.com, compute.googleapis.com and logging.googleapis.com APIs enabled.

Your account needs three IAM roles: Logs Viewer (roles/logging.viewer), Compute Viewer (roles/compute.viewer) and Kubernetes Engine Viewer (roles/container.viewer). The skill's documentation links point to the official setup guides for each item.

  • gcloud CLI and kubectl, authenticated to the project
  • Active Cloud Billing account
  • APIs: container.googleapis.com, compute.googleapis.com, logging.googleapis.com
  • IAM: Logs Viewer, Compute Viewer, Kubernetes Engine Viewer

Trigger the skill with a natural request

You do not need to type the skill name. Describe the symptom in plain language and include the details the agent will ask for: project, cluster, location, node pool name and the time the problem started. The skill uses a 60-minute window by default, from 30 minutes before to 30 minutes after the issue time.

The skill applies when a slice creation fails with SliceCreationFailed, when JobSets stay pending, or when MIG status shows isStable=False or versionTarget.isReached=False during TPU slice formation.

  • "My TPU slice on cluster tpu-prod in us-central1 shows SliceCreationFailed. Check whether the node pool's MIG is still updating."
  • "Our JobSet training-run is stuck pending on project ml-platform since 14:00. Investigate the TPU node pool."
  • "Why is my MIG unstable during TPU slice formation? Read-only checks only, please."

Read the agent's report

The agent reports likely causes and the doc section for each recommended fix. Because the skill is read-only, it will not drain, delete or recreate nodes, and it will not change the cluster. Any change, such as resuming or rolling back a node pool upgrade, is yours to run after you review the findings.

If the failure is unrelated to MIG stability, the skill points you to the separate gke-ai-troubleshooting-tpu-dynamic-slices-monitoring skill.

  • Expect a summary of slice status, MIG stability and log findings.
  • Expect links to documentation sections, not automatic fixes.
  • Check that the agent used the attribution prefix on gcloud commands, as the skill requires.

What people use it for

  • Finding out why a TPU slice on a GKE node pool shows SliceCreationFailed after a node pool upgrade.
  • Checking whether a MIG with isStable=False is holding back a JobSet that has been pending.
  • Pulling Cloud Audit Logs for MIG errors during a specific 60-minute window around an incident.
  • Getting a read-only diagnosis to share with a colleague before anyone changes the cluster.

Frequently asked questions

How do I install the stuck TPU slice MIG skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.

What do I need before using this skill?

You need gcloud CLI and kubectl installed and authenticated to a Google Cloud project with an active Cloud Billing account. The container, compute and logging APIs must be enabled, and you need the Logs Viewer, Compute Viewer and Kubernetes Engine Viewer roles.

Does the skill fix my TPU slice automatically?

No. The skill is explicitly read-only. It runs diagnostic commands only, will not drain, delete or recreate nodes, and gives you any fix to apply yourself.

Is this skill free to use?

The source does not state pricing for the skill itself. It is licensed under Apache-2.0. Google Cloud resources you query may incur costs under your own billing account.

What if my SliceCreationFailed error is not caused by the MIG?

Use the separate gke-ai-troubleshooting-tpu-dynamic-slices-monitoring skill instead. This skill is only for slice failures tied to MIG stability, and the skill itself points you to that one.

Does the skill run scripts or use the network?

The source says the skill runs no scripts but does use the network, because it queries Google Cloud APIs through gcloud and kubectl. It also requires that every gcloud command carry a usage attribution prefix.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

This skill suits platform or ML engineers running TPU workloads on GKE who see SliceCreationFailed errors or JobSets stuck pending during node pool updates. It is not the right choice if your slice failures are unrelated to MIG stability, or if you expect the agent to repair the cluster automatically.

Download the skill