Guide · Agent skill

How to use the Peekaboo macOS UI Automation agent skill

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

The Peekaboo skill lets an AI agent work on a Mac the way a person would: take screenshots, read what is on screen, and then click, type, scroll and manage apps. It is for developers, testers and anyone who wants an agent to handle repetitive desktop steps in native macOS apps. By the end of this guide you will know what the skill needs, how to install it, how to ask for tasks in plain language, and which risks to keep in mind.

Peekaboo macOS UI Automationspinabot/brigade · MIT Download the skill

Quick start

1Confirm you are on macOS. The skill only runs on macOS (darwin). Check that your machine is a Mac before you start.
2Install the Peekaboo CLI. Install the tool through Homebrew using the formula steipete/tap/peekaboo.
3Grant the required permissions. In macOS, allow Screen Recording and Accessibility for the app that runs the agent. Then run peekaboo permissions to confirm.
4Download and place the skill. Download the ZIP from this page and unzip it into ~/.claude/skills/ or a project's .claude/skills/ folder, or upload it under Settings → Capabilities → Skills in the Claude apps.
5Ask for a read-only task first. Request a screenshot or summary of a specific window to confirm the agent can see the app before asking for any clicks or typing.
6Supervise the action steps. Once read-only work succeeds, allow clicking and typing tasks while you watch the screen and stop the agent if it targets the wrong element.

What are agent skills and what does this one do?

An agent skill is a folder containing a SKILL.md file that tells an AI agent when and how to use a set of tools. The agent reads the skill when a request matches it, then follows the instructions and commands inside.

The Peekaboo skill teaches the agent to capture and inspect macOS screens, target UI elements, and drive input. It can take screenshots, read the on-screen layout of apps and windows, click buttons, type text, press keys, scroll, drag, open or quit apps, and use menus and the Dock. The agent can chain these actions to complete multi-step tasks on the desktop.

The skill does not do this work alone. It relies on a separate command-line tool called Peekaboo, which must be installed on the Mac first.

Capture screens, windows and menu bar regions as images
Read app and window layouts and annotate UI elements
Click, type, press keys, scroll, drag and swipe
Launch, quit, hide and switch apps; manage windows and Spaces
Open menus, menu bar items and Dock items

What do you need before installing?

The skill runs only on macOS (darwin). It cannot work on Windows or Linux machines.

You need the Peekaboo CLI, which the skill installs through Homebrew using the formula steipete/tap/peekaboo. You also need to grant two macOS permissions: Screen Recording and Accessibility.

macOS (darwin only)
Peekaboo CLI, installable via the Homebrew formula steipete/tap/peekaboo
macOS Screen Recording permission
macOS Accessibility permission

How do you install the skill?

Download the skill ZIP from this page and unzip it. The folder contains SKILL.md. Then place the folder in one of these locations:

For a personal setup, unzip it into ~/.claude/skills/. To limit it to one project, unzip it into that project's .claude/skills/ folder. If you use the Claude apps, you can instead upload the skill under Settings → Capabilities → Skills.

Personal: ~/.claude/skills/
Single project: .claude/skills/ inside the project folder
Claude apps: Settings → Capabilities → Skills

How do you install the Peekaboo CLI and grant permissions?

The skill's metadata lists a Homebrew install for the Peekaboo CLI using the formula steipete/tap/peekaboo. After installing, check that the tool is ready and that permissions are granted.

Run the permissions check. If Screen Recording or Accessibility is missing, grant it in macOS System Settings for the app that runs the agent, then check again.

peekaboo permissions

How do you trigger the skill with a natural request?

You do not need to type commands yourself. Describe the task in plain language and mention the Mac app or window involved. The agent will read the skill when the request matches its description, which is about capturing and automating macOS UI with the Peekaboo CLI.

Good requests name the app, say what to look at or click, and state the end result. Vague requests such as 'fix my Mac' give the agent too little to act on.

Take a screenshot of the Safari window titled Dashboard and summarise the figures.
Open TextEdit, go to Format > Font > Show Fonts, and tell me what options appear.
Launch Safari at example.com and resize its window to 1200 by 800.
Find the Login window in Safari, fill in the username field and tell me when it is ready for me to check.

What commands does the agent use under the hood?

The skill's quickstart shows the basic happy path: check permissions, list apps, capture an annotated view, click an element by ID, and type text. The annotated capture lets the agent identify targets before it clicks.

Commands share a snapshot cache and accept --json or -j for scripting. Run peekaboo or peekaboo <cmd> --help to see flags.

Use peekaboo see --annotate before clicking so targets are identified first
Most interaction commands accept --app, --window-title and --window-id to target the right window
Use --json for machine-readable output

peekaboo see --annotate --path /tmp/peekaboo-see.png
peekaboo click --on B1
peekaboo type "Hello" --return

What can the skill automate in practice?

The source gives examples for several kinds of task. A typical flow is to capture the window, click a field by its ID, type, then press Tab to move on. Menus and app management follow the same pattern with menu click and app commands.

Screenshots can also be sent for analysis. The skill's examples show capturing a screen or window and asking for a summary with --analyze.

Filling in a form in a native app: see, click the field, type, press tab
Opening a menu item such as File or Format options in TextEdit
Launching an app at a URL and arranging its window size and position
Capturing a dashboard window and asking for a summary of its figures
Watching a region of the screen over time with live capture

peekaboo see --app Safari --window-title "Login" --annotate --path /tmp/see.png
peekaboo click --on B3 --app Safari
peekaboo type "user@example.com" --app Safari

What safety steps should you follow?

The skill acts on the live screen with your macOS permissions. A mis-targeted click or keystroke can change or delete data in whatever app is in focus. Keep an eye on the agent while it works, and do not leave it running unattended.

Avoid sensitive tasks such as entering passwords or banking details. The skill's own example types a password, but that does not make it a safe choice for real credentials.

The skill runs scripts (yes) and does not use the network (no), according to its metadata. Review the commands the agent plans to run if you are unsure what it will do.

Do not use it for passwords, banking or other sensitive data entry
Do not run it unattended; check the screen during the task
Save work in open apps before starting
Start with read-only tasks such as capturing and summarising before any clicking or typing

What people use it for

Developers automating repetitive clicking or form-filling in native macOS apps
Testers exploring how a Mac app's interface behaves without writing custom scripts
Users who want an agent to take and analyse screenshots of specific windows
Opening menus, launching apps and arranging windows as part of a multi-step desktop task

Frequently asked questions

How do I install the Peekaboo agent skill?

Download the skill ZIP from this page and unzip it into ~/.claude/skills/ for personal use, or into a project's .claude/skills/ folder to limit it to that project. In the Claude apps, you can instead upload it under Settings → Capabilities → Skills.

What do I need to run Peekaboo on a Mac?

You need macOS, since the skill runs only on darwin. You also need the Peekaboo CLI installed through the Homebrew formula steipete/tap/peekaboo, plus Screen Recording and Accessibility permissions.

Does Peekaboo work on Windows or Linux?

No. The skill is limited to macOS (darwin) and cannot work on other operating systems.

Is the Peekaboo skill free?

The source provided for this page does not state pricing or licence terms beyond the MIT licence shown for the skill. Check the licence file in the source repository for the details that apply to you.

Why does the agent need Screen Recording and Accessibility permissions?

Screen Recording lets the tool capture screenshots and read what is on screen. Accessibility lets it click, type and press keys. Run peekaboo permissions to check both are granted.

Can I use the skill to enter passwords or banking details?

It is not recommended. The agent acts on the live screen with your permissions, so mistakes or exposure are possible. Keep sensitive data entry for yourself.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Use this skill if you are on a Mac and want an agent to capture, read and automate native apps for repetitive clicking, form-filling or interface testing, and you can supervise it. Pick something else if you work on Windows or Linux, need unattended runs, or need the agent to handle passwords or banking details.

Download the skill