Skip to content
Get packs

BrightData Page Fetcher

Fetches web pages as clean markdown, escalating from free methods to a paid proxy only when needed.

Source Download skill
Repository
danielmiessler/LifeOS
Stars
19,682
Licence
MIT
Files
6 files

What it does

Once installed, the agent can pull the content of a web page, or crawl a whole site, and return it as markdown text. It tries free methods first (a plain request, then a request with browser-like headers, then a real Chrome browser) and only uses Bright Data's paid managed fetch service when the cheaper methods fail or the user asks for it. It reports which method worked for each page and refuses to bypass sites that block automated access.

Great for

  • Developers who need page content in markdown for documentation, research or LLM input
  • Analysts collecting public pages from a known list of URLs or a documentation site
  • People who already have a Bright Data account and want cost-aware fallback to it

Skip it if

  • Simple public pages where the built-in WebFetch tool is enough, since this adds setup and steps
  • Social media or platform data via named data-provider actors, which the skill explicitly excludes
  • Logged-in or private browser sessions, which belong to the Interceptor skill; scraping private data may also breach site terms

Try asking

  • “Fetch example.com/pricing as markdown”
  • “Crawl the docs under docs.example.com/api/ and list any pages that failed”
  • “Use Bright Data to fetch this region-specific listing”
  • “Map this website and give me a site map with page stats”
Step-by-step guide: how to use BrightData Page FetcherInstall, first run, examples, tips and FAQs

Install in 3 steps

  1. 1

    Download the skill

    The ZIP contains the 6 original files, the MIT licence and an attribution note.

    BrightData.zip
  2. 2

    Add it to your agent

    Claude Code (all projects)

    mkdir -p ~/.claude/skills && unzip BrightData.zip -d ~/.claude/skills/

    Claude Code / Agent SDK (this project only)

    mkdir -p .claude/skills && unzip BrightData.zip -d .claude/skills/

    Claude apps (web/desktop): Settings → Capabilities → Skills → Upload skill, and choose the ZIP.

  3. 3

    Ask for the task

    No special command needed — the agent reads the skill's description and loads it when your request matches. Try one of the prompts above.

Get better results

  • Set up the free tiers first and confirm a test page works before adding Bright Data credentials
  • Ask for a small crawl limit first; crawls over 20 pages require confirmation and can cost credits
  • Check the stats in the reply to see which pages needed the paid tier

What's inside

  • Intents/Crawl.md
  • Intents/FourTierScrape.md
  • Setup.md
  • SKILL.md
  • Tools/MarkdownPreCheck.help.md
  • Tools/MarkdownPreCheck.ts

Redistributed unmodified from danielmiessler/LifeOS under the MIT licence. The assessment was drafted with AI from the skill's files on 2026-10-29. Skills can instruct your agent to run code — read SKILL.md and any scripts before installing.