Skip to content
Get packs

Firecrawl Site Crawler for SEO

Crawl, map, and scrape whole websites via Firecrawl for SEO audits

Source Download skill
Repository
AgriciDaniel/claude-seo
Stars
25,798
Licence
MIT
Files
2 files

What it does

This skill teaches an AI agent to inspect a whole website using the Firecrawl service instead of only looking at one page. Once installed, the agent can list every URL on a site, crawl many pages to pull their text and links, scrape JavaScript-heavy pages that plain fetching misses, and search within a site. It is an add-on to the Claude SEO toolkit and depends on a Firecrawl connection being set up first.

Great for

  • SEO specialists who want site-wide audits such as broken links, orphan pages, and URL structure checks
  • Developers or agencies auditing large or JavaScript-rendered sites (React, Vue, Angular)
  • Anyone who needs a fast inventory of a site's pages and titles

Skip it if

  • Simple one-page checks, since the built-in fetch tool is free and sufficient (the skill itself says so) and Firecrawl uses paid credits
  • Sites you do not have permission to crawl, since large crawls can burden a server and may breach terms of service
  • Users without a Firecrawl account or API key, because the skill will not work without the Firecrawl MCP server installed

Try asking

  • “Crawl example.com's /blog/ section and list any broken internal links”
  • “Map my site and compare the discovered URLs against my XML sitemap to find orphan pages”
  • “Scrape this React single-page app and extract the main content for an SEO review”
  • “Search my competitor's site for pages about 'running shoes' to check content coverage”
Step-by-step guide: how to use Firecrawl Site Crawler for SEOInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 2 original files, the MIT licence and an attribution note.

    seo-firecrawl.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip seo-firecrawl.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip seo-firecrawl.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Ask for a map first to estimate page count, then request a limited crawl so credit use stays predictable
  • Use includePaths or excludePaths to target sections like /blog/* and skip /admin/* or /api/*
  • Check your Firecrawl credit balance before large crawls, since each crawled page costs one credit

What's inside

  • LICENSE.txt
  • SKILL.md

Redistributed unmodified from AgriciDaniel/claude-seo under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.