Guide · Agent skill
How to use the Blog & Feed Ingestion agent skill (blog-ingest)
Updated · written with AI from the project's own documentation and checked against our catalogue data
The blog-ingest skill teaches an AI agent how to take a whole blog, newsletter or RSS/Atom feed and save every public post as a source page in your gbrain knowledge store. It is aimed at researchers, students and anyone backfilling years of posts from a publication they follow. After this guide you will know what the skill does, how to install it, how to trigger it with a natural request, and what to watch for before running a bulk import.
Quick start
- 1Download the ZIP. Download the blog-ingest ZIP from this page.
- 2Unzip into a skills folder. Unzip it into ~/.claude/skills/ for personal use or into .claude/skills/ in a project.
- 3Install related skills. Make sure brain-ingest-gate, idea-ingest and the referenced conventions files are also installed.
- 4Ask for a publication ingest. Give the agent a blog, newsletter or feed URL and say ingest this whole blog or a similar trigger phrase.
- 5Test on a few posts. Let the agent write 3 to 5 posts first and check that the output is clean.
- 6Review the run. Check the run summary for skipped gated posts and any flagged text.
What the blog-ingest skill makes the agent do
Given a blog, newsletter or feed URL, the skill tells the agent to find the site's RSS or Atom feed (or its sitemap), list every public post, and save each one as a clean source page in your brain. It skips posts that are already saved, so re-running an import does not create duplicates. It also paces its requests to reduce the chance of being blocked.
Saving the page is only the first step. The skill hands each article to a separate enrichment skill (brain-ingest-gate) for linking and entity extraction. The text of every post is treated as untrusted data, not as instructions for the agent.
- Feed discovery: checks the homepage for an advertised feed, then common paths such as /feed and /rss.xml, then /sitemap.xml.
- Pagination: follows next links, WordPress paged feeds, or sitemap indexes to reach the full archive.
- Dedup: checks canonical URLs before every write, so re-runs are cheap.
- Pacing: default 1.5 seconds between fetches, with backoff on HTTP 429 responses (capped at 30 seconds, honouring Retry-After).
Install the blog-ingest skill
Download the ZIP for this skill from the page you are reading. Unzip it into one of these locations, depending on how you want to use it:
You can install it for yourself across all projects by unzipping into ~/.claude/skills/. To limit it to one project, unzip it into that project's .claude/skills/ folder.
If you use the Claude apps instead, open Settings, go to Capabilities, and upload the skill under Skills.
- Personal install: ~/.claude/skills/
- Project install: .claude/skills/ inside the project folder
- Claude apps: Settings → Capabilities → Skills
Check the requirements before you start
The skill is a set of procedures, not a standalone program. It needs a gbrain brain or knowledge store that the agent can write pages into, plus the agent's ordinary fetch, read and write tools.
The procedure also refers to related skills that must be installed alongside it: brain-ingest-gate, idea-ingest, and the conventions files it cites. Without them the enrichment handoff and routing will not work as described.
- A gbrain brain the agent can write to
- Ordinary agent fetch, read and write tools
- brain-ingest-gate, idea-ingest and the referenced conventions files installed
Trigger the skill with a natural request
You do not need a special command. Give the agent a publication URL and ask in plain language. The skill's trigger phrases include ingest this publication, ingest this whole blog, ingest this feed, ingest this newsletter archive, save this whole substack, backfill this blog, walk this RSS feed and ingest every post from.
For a single article, do not use this skill. The skill routes single article, essay or tweet URLs to idea-ingest instead.
- "Ingest this whole blog: https://example.com"
- "Backfill this blog and save every post from it"
- "Walk this RSS feed and save the archive"
What a run looks like and how to test it safely
The skill enumerates the full list of candidate post URLs first, removes duplicates, and reports the count before fetching bodies. Every whole-publication run is treated as a bulk run, so the agent tests on 3 to 5 posts, checks that the output exists and is clean, then ramps up progressively.
Source pages are filed under sources/articles/ with the publication slug, and run manifests go under projects/. The manifest records which discovery mechanism worked (substack, rss or html).
- Step 1: the agent reports how many posts it found.
- Step 2: a small test batch of 3 to 5 posts is written and checked.
- Step 3: the run ramps up in stages, then finishes the rest.
- Step 4: review the flagged posts and the run summary.
Safety notes: scripts, network and gated content
The skill runs no scripts. It does use the network, because the agent fetches the site's pages and feeds. It only ingests public posts. Gated or paywalled posts are detected, skipped and logged with a reason. The skill does not use cookies, logins or other workarounds to get them.
Posts can contain text written to instruct an AI. The skill keeps such text as quoted content, marks it in the page, and does not carry it out as a task.
- Runs scripts: no
- Uses network: yes
- Paywalled or members-only posts are skipped, never worked around
- Imperative text inside posts is quoted and flagged, not obeyed
- Check a site's terms before bulk copying; the skill can crawl many pages quickly even with pacing
What people use it for
- Archiving a researcher's favourite newsletter into a searchable personal knowledge base.
- Backfilling years of posts from a blog you follow.
- Re-running an import later to pick up new posts without duplicates.
- Building a source library on one topic from several publications, one at a time.
Frequently asked questions
How do I install the blog-ingest skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder. In the Claude apps, upload it under Settings → Capabilities → Skills.
What do I need to run blog-ingest?
You need a gbrain brain or knowledge store the agent can write pages into, plus ordinary fetch, read and write tools. The related skills brain-ingest-gate, idea-ingest and the referenced conventions files must also be installed.
Is blog-ingest free?
The skill is licensed under MIT. The source does not state pricing for any related service, so check the terms of your own agent and gbrain setup.
Can blog-ingest import paywalled or members-only posts?
No. The skill detects gated content, skips it with a logged reason, and does not use cookies, logins or endpoint workarounds to get it.
Should I use blog-ingest for a single article?
No. For one article URL, the skill routes you to idea-ingest. blog-ingest is for enumerating a whole publication or feed.
Does blog-ingest avoid duplicate pages when I run it again?
Yes. It checks canonical URLs before each write and skips posts that already exist, so re-runs are safe and do not duplicate pages.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use blog-ingest if you want to archive an entire public blog, newsletter or feed into a gbrain knowledge store and re-run the import later without duplicates. Pick something else if you only need one article, if the content is paywalled, or if you do not have gbrain and its related skills set up.
Download the skill