Guide · Agent skill

How to use the Archive Crawler agent skill

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

Archive Crawler is an agent skill that explores a folder of old personal files, such as a Dropbox archive, backups, email exports or a hard-drive dump, and picks out the content that looks most valuable to you: your own writing, ideas, letters and relationship material. This guide explains what the skill does, how to install it, how to set the required allow-list, how to trigger it with plain requests, and what to watch for on safety. By the end you will be able to run a supervised review of a permitted folder.

Archive Crawlergarrytan/gbrain · MIT Download the skill

Quick start

1Download the ZIP. Get the archive-crawler ZIP from this page. It contains SKILL.md.
2Unzip the skill. Extract it into ~/.claude/skills/ for all projects, or into a project's .claude/skills/ folder for one project. In the Claude apps, upload it under Settings → Capabilities → Skills instead.
3Create the allow-list. Add archive-crawler.scan_paths to gbrain.yml with each folder the agent may read. Optionally add deny_paths for sensitive subfolders.
4Make a request. Ask naturally, for example "crawl my archive" or "find gold in my archive", and name a topic if you have one.
5Review the map and priority queue. The agent inventories the allowed folders and proposes an order. Adjust it before the crawl starts.
6React to each candidate. The agent shows one item at a time. Give your reaction in your own words so it is recorded exactly.
7Check the filed pages. Review the pages filed into originals/, personal/ and ideas/, and the manifest at projects/<archive-slug>/STATUS.md.

What an agent skill is and what archive-crawler makes the agent do

An agent skill is a folder containing a SKILL.md file that tells an AI agent how to perform a specific task. The archive-crawler skill is for reviewing personal file archives. Once installed, the agent maps the folders you allow, ranks them by likely value, and shows you candidate items one at a time.

The skill follows three phases. First it builds an inventory of the tree and writes a progress log (a manifest) at projects/<archive-slug>/STATUS.md. Then it crawls in priority order, reading each candidate and applying a filter that keeps personal writing, conversations with substance, ideas, relationship letters, creative work and similar material while skipping system files, binary blobs, receipts, tax documents and newsletters. Finally it files keepers into your brain under originals/, personal/ or ideas/, depending on their primary subject.

Each item is tracked as unseen, reviewed, ingested, skipped or high-signal, so the agent never re-shows the same item.
Your reactions are recorded in your own words, not paraphrased.
The skill lists the permitted scan paths before it starts and lets you override the priority order.

Install the archive-crawler skill

Download the ZIP for archive-crawler from this page. The package contains SKILL.md. Unzip it into one of two locations: ~/.claude/skills/ to make the skill available to you everywhere, or a project's .claude/skills/ folder to limit it to that project.

If you use the Claude apps instead, upload the skill under Settings → Capabilities → Skills.

Personal install: unzip into ~/.claude/skills/.
Project install: unzip into the project's .claude/skills/ folder.
Claude apps: upload the skill under Settings → Capabilities → Skills.

Set the required allow-list in gbrain.yml

The skill refuses to run unless gbrain.yml defines archive-crawler.scan_paths. This is a deliberate safety fence: the agent will not guess which folders are safe to read. List every folder the agent may scan. You can also add an optional deny_paths list inside the allow-list to exclude sensitive subfolders such as finances or medical records.

If scan_paths is empty or missing, the skill exits with a refusal message telling you to add explicit paths and run again.

# gbrain.yml — the allow-list is mandatory
archive-crawler:
  scan_paths:
    - ~/Documents/writing/
    - ~/Dropbox/Archive/
    - /mnt/backup/old-letters/
  # Optional deny-list inside the allow-list:
  # deny_paths:
  #   - ~/Documents/finances/
  #   - ~/Documents/medical/

Trigger the skill with a natural request

You do not need a special command. The skill responds to phrases such as the ones below. Name the folder or topic you care about so the agent starts in the right place.

Example requests the skill is written to respond to: crawl my archive, find gold in my archive, archive crawler, scan my dropbox for, and mine my old files for. A request like "mine my old files for letters to my brother" gives the agent a topic to look for within the allowed paths.

"Crawl my archive" to start a full inventory and proposed review order.
"Find gold in my archive" to start the crawl phase on the permitted folders.
"Scan my dropbox for essays about my first company" to focus the review on a topic within the allow-list.

Supported archive types and what the agent reads

The skill is written for several source types: local folders, Dropbox, Backblaze B2, Gmail takeout in .mbox format, and Outlook .pst archives. For plain text, HTML and Markdown it reads files directly. For .mbox email files it uses Python's standard mailbox library to walk messages and pull out plain-text bodies. For .docx files it reads the text from the document's XML. For .zip, .tar and .tar.gz archives it extracts to a temporary folder and walks the result.

For .pst files the skill first checks the file header. If the header is not valid it reports the file as corrupt or null and does not try to read it. Valid files are converted with readpst.

Requirements: Python 3 with the standard mailbox library for .mbox files.
Legacy .doc files need antiword or catdoc, as the skill's instructions state.

Safety notes: scripts, network and private content

Archive Crawler runs scripts and uses the network. Its instructions include Python and shell commands that read files, and it can work with cloud sources such as Dropbox and Backblaze B2. Before you grant access, check which folders are on the allow-list and whether any deny-list entries are needed.

The skill reads file contents to judge their value, so anything inside an allowed folder can be read and may be ingested. Content the agent reads is sent to whichever model the agent uses. If you are not comfortable with that, do not grant access to large personal archives.

Start with one small folder to see how the review works.
Put tax, finance, medical and credential folders on deny_paths or leave them out of scan_paths.
Review the proposed priority queue before approving a crawl.
Check what was filed into originals/, personal/ and ideas/ after each session.

Tips for a useful review session

Work in small batches. The skill is a slow, interactive review, so a session with a focused folder and a clear topic usually produces more useful results than a sweep of everything. Use your own words when you react to a candidate, because the manifest keeps those exact words. Override the priority order when you know where your best writing lives.

Give the agent a topic (letters, essays, ideas) to focus the filter.
Reread the session log in the manifest to see what was already shown.
Keep a backup of your archive before a large crawl.

What people use it for

Rediscovering years of personal journal entries stored in a Dropbox folder.
Finding letters you wrote to family members inside an email takeout export.
Collecting early product ideas and theses from old document folders into an ideas/ store.
Pulling readable text out of old .docx writing files in a backup mount.

Frequently asked questions

How do I install the archive-crawler skill?

Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder. In the Claude apps, upload it under Settings → Capabilities → Skills.

Why won't archive-crawler run?

The skill refuses to run unless gbrain.yml contains an archive-crawler.scan_paths allow-list. Add the folders the agent may scan; if the list is empty or missing, the skill exits with a refusal message.

What does the skill need to work?

It needs the gbrain personal knowledge system with a gbrain.yml file, access to the archive (local mount, Dropbox, Backblaze B2 or an mbox export), and Python 3 with the standard mailbox library for .mbox files.

Is archive-crawler free to use?

The skill is published under the MIT licence. This page does not state pricing for gbrain or the models the agent uses, so check those separately.

Can it scan my tax or medical folders?

You should not, unless you deny them. The skill reads file contents to judge value and may ingest them, so keep those folders out of scan_paths or list them in deny_paths.

How is this different from a normal file search?

A normal search finds files matching a query. Archive Crawler reviews candidates one at a time, filters for personal writing and ideas, and records your reactions, so it is a slower, interactive process.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Archive Crawler suits people with years of personal writing, letters or notes who want to rediscover it, and who already use gbrain or are willing to set an explicit folder allow-list. Pick a different tool if you only need a quick search of files, or if you are not comfortable letting an agent read private archives through a model.

Download the skill