Guide · Agent skill
How to use the Data Scraper Agent agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The Data Scraper Agent is an agent skill: a SKILL.md instruction file that teaches an AI agent how to build a small automated data-collection bot. This guide is for developers and analysts who want a recurring watcher for public pages or APIs, and it explains how to install the skill, how to ask for it, and what you need before the agent starts writing code.
Data Scraper Agentaffaan-m/ECC · MIT Download the skill Quick start
- 1Download the skill. Download the ZIP from this page and unzip it so you have a folder containing SKILL.md.
- 2Install the skill. Place the folder in ~/.claude/skills/ for personal use or in .claude/skills/ for a project, or upload the ZIP under Settings → Capabilities → Skills in the Claude apps.
- 3Gather credentials. Get a Gemini Flash API key, a GitHub account, and credentials for Notion, Google Sheets or Supabase.
- 4Describe the source. Tell the agent the public website or API, the fields to extract and how often it should run.
- 5Review the generated code. Read the scraper and LLM scripts the agent writes, and confirm the source's terms allow automated access.
- 6Set up the schedule. Ask the agent to configure GitHub Actions to run the scraper on your chosen schedule.
- 7Add the feedback loop. Once collection works, ask the agent to add a feedback loop so the bot learns from your choices.
What is an agent skill and what does this one make the agent do?
An agent skill is a folder containing instructions that an AI agent loads when a task matches the skill's description. This skill is contained in a single file, SKILL.md, and it guides the agent through building a production-style data collection agent for any public source, such as job boards, prices, news, GitHub repositories or sports scores.
The agent builds the bot in three layers: COLLECT (a Playwright or BeautifulSoup scraper that runs on a schedule), ENRICH (a Gemini Flash language model that scores, summarises or classifies the text) and STORE (Notion, Google Sheets or Supabase). The skill also describes a feedback loop that learns from the user's decisions over time.
- COLLECT: Playwright or BeautifulSoup scrapers
- ENRICH: Gemini Flash to score, summarise and classify
- STORE: Supabase, Google Sheets or Notion
- SCHEDULE: GitHub Actions cron jobs
- LEARN: a feedback loop based on user choices
How do you install the data-scraper-agent skill?
Download the skill ZIP from this page and unzip it. For a personal setup, place the skill folder in ~/.claude/skills/. For a single project, place it in the project's .claude/skills/ folder.
If you use the Claude apps instead, open Settings, go to Capabilities, and upload the skill there.
- Personal install: ~/.claude/skills/
- Project install: .claude/skills/ inside your project
- Claude apps: Settings → Capabilities → Skills, then upload the ZIP
How do you trigger the skill with a natural request?
You do not need to name the skill. The skill activates when your request matches its purpose. The source lists these example phrasings as triggers: asking to scrape or monitor a public website or API, saying things like build a bot that checks or monitor X, asking to track jobs, prices, news, repos, sports scores, events or listings, and asking how to automate data collection without paying for hosting.
Be specific about the source, the fields you want and how often it should run. The more concrete the request, the easier it is for the agent to plan the scraper.
- Build a bot that checks a public job board every day and saves new postings to a Google Sheet.
- Monitor a product price page each week and store the price history in Supabase.
- Track new releases in a public GitHub repo and summarise them with Gemini Flash into Notion.
What do you need before you start?
The skill lists its requirements plainly. You need Python, Playwright or BeautifulSoup, a Gemini Flash API key, a GitHub account for GitHub Actions, and an account with Notion, Google Sheets or Supabase along with its credentials.
The skill is described as running scripts and using the network, so the agent will create code that makes web requests and calls external services.
- Python
- Playwright or BeautifulSoup
- Gemini Flash API key
- GitHub account (for GitHub Actions scheduling)
- Notion, Google Sheets or Supabase account and credentials
What safety points should you check?
The skill does not address legal or ethical checks. Before scraping a site, check its terms and robots.txt yourself, and do not use the skill on sites that prohibit automated access.
Do not use the skill for sensitive or private data. Results are passed to a third-party LLM and stored in external services. Keep API keys and service credentials out of shared code and review any script the agent writes before you run it on a schedule.
- Check the site's terms and robots.txt before scraping.
- Use only public data; the skill is not designed for private or sensitive information.
- Remember that text is sent to a third-party LLM and stored in external services.
- Store API keys and credentials securely, not in public repositories.
- Review scripts the agent generates, since the skill runs scripts and uses the network.
What tips help the agent build a reliable watcher?
Define the source and the fields to extract in the first request, because step one of the skill's workflow is defining where to scrape and what to extract. Then choose one storage target to start with, since Notion, Sheets and Supabase are all options and one is enough for a first version.
Start with a daily or weekly schedule. The skill's stack uses free tiers, so a modest schedule keeps usage low, but the source does not give specific free-tier limits, so check each provider's current terms yourself.
- Name one source, a few fields and one storage target for version one.
- Ask for the feedback loop only after the basic collection works.
- Verify the schedule runs in GitHub Actions before relying on the output.
What people use it for
- Monitoring a public job board for new postings and saving them to a spreadsheet.
- Tracking product prices on public pages on a recurring schedule.
- Watching news sites or GitHub repositories and summarising new items with a free LLM.
- Collecting public sports scores or event listings into Supabase for later review.
Frequently asked questions
How do I install the data-scraper-agent skill?
Download the ZIP from this page and unzip it. Put the folder in ~/.claude/skills/ for personal use or in a project's .claude/skills/ folder, or upload the ZIP in the Claude apps under Settings → Capabilities → Skills.
Is the data-scraper-agent skill free?
The skill is published under the MIT licence. The skill's stack is described as free to run, using Gemini Flash, GitHub Actions, and free tiers of Supabase or Sheets, though each provider sets its own terms and limits that you should check.
What do I need to run it?
You need Python, Playwright or BeautifulSoup, a Gemini Flash API key, a GitHub account for GitHub Actions, and a Notion, Google Sheets or Supabase account with credentials.
Can I scrape any website with this skill?
No. The skill does not address legal or ethical checks, so you should not scrape sites that prohibit automated access in their terms or robots.txt. Check the rules for each source before you build a scraper for it.
What are alternatives to this skill?
The source does not compare this skill with other tools. You could build a similar scraper by hand with Playwright or BeautifulSoup and a scheduler, or use a hosted scraping service, but this guide does not assess those options.
Why won't the skill work for my private data?
The skill is not designed for sensitive or private data, because results are passed to a third-party LLM and stored in external services. Use it only for public data.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use this skill if you are a developer or analyst who wants a free, scheduled watcher for public pages or APIs, and you are comfortable reviewing generated Python code. Pick something else if you need a finished app, if the target site forbids automated access, or if your data is sensitive.
Download the skill