Guide · Agent skill
How to use the Audit noindex Pages agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The Audit noindex Pages skill (skill name all-noindex-pages) gives an AI agent a checklist for reviewing a website's noindex directives, the meta tags or headers that tell search engines not to list a page. It is written for front-end and SEO developers preparing a launch or redesign, marketers investigating pages that dropped out of search results, and teams running periodic indexability reviews. After this guide you will know what the skill does, how to install it, how to ask for an audit in plain language, and what to check before acting on its findings.
Audit noindex Pagesthedaviddias/Front-End-Checklist · MIT Download the skill Quick start
- 1Download the ZIP. Get the all-noindex-pages skill ZIP from this page.
- 2Unzip into the skills folder. Place it in ~/.claude/skills/ for personal use, or in your project's .claude/skills/ folder.
- 3Or upload in the Claude apps. Go to Settings, then Capabilities, then Skills, and upload the skill.
- 4Provide real page output. Share the URLs, rendered HTML, or templates you want checked, since the skill needs actual page output.
- 5Ask for an audit. Describe the goal in plain words, such as finding important pages wrongly set to noindex.
- 6Review the findings. Check each flagged route against the rendered HTML and HTTP headers before making any change.
What the skill makes the agent do
The skill tells the agent to identify every page that uses the noindex directive in meta tags or HTTP headers, and to check whether important pages such as products or articles are being blocked by mistake. It also asks the agent to confirm that noindex is used correctly on admin, utility, or thin-content pages.
The skill insists on verifying the final rendered HTML and HTTP response rather than trusting source files alone. The agent is asked to flag the exact routes or templates that cause problems and to describe how to verify the final page output.
- Find pages with noindex in meta tags or headers
- Check that critical pages are not accidentally blocked
- Confirm noindex is applied to thin, admin, or utility pages
- Flag exact routes or templates that cause problems
Install the skill
Download the ZIP file for the skill from this page. The skill package contains SKILL.md and references/rule.md.
For a personal install, unzip it into ~/.claude/skills/. For a single project, unzip it into the project's .claude/skills/ folder. Alternatively, in the Claude apps, open Settings, go to Capabilities, and upload the skill under Skills.
- Personal: unzip into ~/.claude/skills/
- Project only: unzip into .claude/skills/ inside your project
- Claude apps: Settings → Capabilities → Skills, then upload
Trigger the skill with natural requests
You do not need to type the skill name. The skill's description covers auditing metadata, crawlability, structured data, and indexability, so ask for the outcome you want in plain words. Give the agent the URLs or code it needs, because the skill depends on real page output.
- Audit my site for noindex pages that should be indexed.
- Why have my product pages dropped out of search results? Check the rendered HTML and headers.
- Review our templates for noindex used on admin or thank-you pages.
- Run a periodic indexability review of these URLs and list the routes causing problems.
Read the findings before you change anything
The skill is built to report, and its guidance is to remove noindex only from pages that should be indexed and ranking. The skill also explains the difference between noindex and disallow in robots.txt, which matters because the two work differently. Ask the agent to explain its reasoning for each flagged route.
Do not apply fixes to production SEO settings without a human reviewing the findings. A wrong noindex change can affect search visibility.
- Confirm each flagged page is one you expect to appear in search
- Compare the rendered HTML with the HTTP headers for the same URL
- Review template changes before deploying them
Example of a noindex directive
The reference file shows two common patterns. The first, noindex with follow, suits utility pages such as Thank You pages or internal search results. The second, noindex with nofollow, is for pages that should be completely ignored.
<meta name="robots" content="noindex, follow">What people use it for
- Checking a site before launch to make sure product or article pages are not set to noindex
- Investigating why pages dropped out of search results after a redesign
- Confirming admin, thank-you, or internal search pages are correctly marked noindex
- Running a periodic metadata review and listing the templates that output noindex
Frequently asked questions
Is the Audit noindex Pages skill free?
The skill is published under the MIT licence, as stated on this page. Check the licence file in the source repository for the full terms.
What do I need to use this skill?
The skill lists no requirements. It does not run scripts and does not use the network, but it needs real page output, such as rendered HTML, HTTP headers, or template code, to give reliable results.
How do I install the all-noindex-pages skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder. You can also upload it in the Claude apps under Settings, then Capabilities, then Skills.
What is the difference between noindex and disallow in robots.txt?
The skill explains that noindex is a page-level directive in meta tags or headers that keeps a page out of search results, while disallow in robots.txt controls crawling. Choose based on whether you want the page left out of search or simply not crawled.
Why won't the skill just read my source files?
The skill asks the agent to verify the final rendered HTML and HTTP response, because source files may not match what search engines receive. Give the agent real output for the most accurate results.
Can the agent change my noindex settings for me?
The skill is designed to report findings and suggest fixes. Its guidance warns against live changes to production SEO settings without a human reviewing the findings.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are a front-end or SEO developer preparing a launch, a marketer trying to explain lost search traffic, or a team doing regular indexability reviews, and you can share real URLs or page output. Pick something else if you cannot share the URLs or code with the agent, or if you need an agent to make live SEO changes unsupervised.
Download the skill