Guide · Open source
How to use SpiderFoot: install, run a scan and read results
Updated · written with AI from the project's own documentation and checked against our catalogue data
SpiderFoot is an open-source intelligence (OSINT) automation tool. It queries many data sources and runs modules that feed each other to collect information about a target such as a domain, IP address or email address. This guide shows you how to install it from the source code, start the web interface, and understand what a scan is built to do.
SpiderFootsmicallef/spiderfoot · ★ 24.2k · MIT Install options & downloads Quick start
- 1Check prerequisites. Confirm you have Python 3.7 or newer and pip3 installed.
- 2Clone the repository. Run git clone on the SpiderFoot repository and enter the spiderfoot folder.
- 3Install libraries. Run pip3 install -r requirements.txt to install the Python dependencies.
- 4Start the server. Run python3 ./sf.py -l 127.0.0.1:5001 to start the embedded web server.
- 5Open the interface. Visit http://127.0.0.1:5001 in your browser.
- 6Run a scan. Create a scan, enter an authorised target and choose modules, starting with those that need no API key.
- 7Review and export. Verify findings manually, then export results as CSV, JSON or GEXF if needed.
What SpiderFoot does and what you can target
SpiderFoot connects to many data sources and runs modules that pass data to one another in a publisher/subscriber model. The project says this is meant to maximise data extraction from a single starting point.
You can target an IP address, domain or sub-domain, hostname, network subnet (CIDR), ASN, email address, phone number, username, a person's name, or a Bitcoin address.
- Host, sub-domain and TLD enumeration
- Email, phone number and human name extraction
- Threat intelligence and blacklist queries
- S3, Azure and DigitalOcean bucket enumeration
- Port scanning and banner grabbing
- Data breach searches
Install SpiderFoot from source
You need at least Python 3.7 and pip3. The project recommends a packaged release over the master branch, because master can contain untested features and modules. The development build is installed by cloning the git repository and installing the Python libraries from requirements.txt.
Run these commands in a terminal:
- Requires Python 3.7+ and pip3
- Clone the repository, then change into the spiderfoot folder
- Install the Python libraries listed in requirements.txt
git clone https://github.com/smicallef/spiderfoot.git
cd spiderfoot
pip3 install -r requirements.txtStart the web server and open the interface
SpiderFoot has an embedded web server that provides a web-based interface. The command below binds it to 127.0.0.1 on port 5001, so it is only reachable from your own machine. Open the address in your browser once the server is running.
SpiderFoot can also be used entirely from the command line, so the web interface is optional.
- Address to open after starting: http://127.0.0.1:5001
python3 ./sf.py -l 127.0.0.1:5001Run your first scan
Create a new scan in the web interface and enter a target from the list of supported entity types, such as a domain name or IP address. Choose which modules to run. Most modules do not require API keys, so you can start without paying for any data source.
Only scan targets you are authorised to investigate. The tool gathers data on people and systems, so authorisation matters before you start.
- Pick a target type that matches your question, such as a domain for attack surface checks
- Start with modules that need no API key
- Add API keys later for modules that use paid or tiered sources
Read and export the results
SpiderFoot's modules feed each other, so one finding can lead to more lookups. Results need interpretation: a hit is a lead to verify, not a confirmed fact. The project also documents a YAML-configurable correlation engine with 37 pre-defined rules that flag patterns across results.
Results can be exported as CSV, JSON or GEXF, and the data is stored in an SQLite back-end that you can query yourself.
- Export formats: CSV, JSON and GEXF
- Data is kept in an SQLite back-end for custom querying
- Visualisations are available for exploring relationships
Manage API keys and optional integrations
Most of the 200+ modules do not need API keys. Many modules that do need one have a free tier. The source also notes that API keys can be exported and imported.
SpiderFoot can call other tools such as DNSTwist, Whatweb, Nmap and CMSeeK, and it has TOR integration for dark web searching. Check the project documentation for setup details, since the README does not describe those steps.
- Over 200 modules, most without API keys
- Modules can be Free API, Tiered API or Internal
- API key export and import is supported
What people use it for
- Reviewing what your organisation has exposed on the internet, such as subdomains and cloud storage buckets
- Supporting reconnaissance during penetration tests or red team exercises
- Querying many threat intelligence and blacklist sources for one IP address or domain in a single place
- Looking for associated social media accounts or leaked data tied to a username or email address
Frequently asked questions
Is SpiderFoot free?
SpiderFoot is MIT-licensed open source software, and most of its modules do not require API keys. Some modules use tiered APIs, and the project offers a cloud-hosted paid product called SpiderFoot HX.
What do I need to run SpiderFoot?
You need at least Python 3.7 and pip3 to install the required Python libraries. The source also recommends a packaged release over the master branch for stability.
How do I start SpiderFoot?
Clone the repository, run pip3 install -r requirements.txt, then run python3 ./sf.py -l 127.0.0.1:5001. Open http://127.0.0.1:5001 in your browser to use the web interface.
Can I use SpiderFoot without the web interface?
Yes. The README states that SpiderFoot can be used completely via the command line, as well as through its embedded web-based interface.
What is SpiderFoot HX and how is it different?
SpiderFoot HX is the commercial version that is cloud-based and managed for you. It adds features such as multi-user collaboration, multiple targets per scan, and scheduled attack surface monitoring with notifications.
How does SpiderFoot compare to Maltego, theHarvester, Recon-ng and Amass?
All four are reconnaissance and OSINT tools, but they differ in focus and interface. This guide does not benchmark them, so compare them against your own targets and workflow before choosing.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
SpiderFoot suits security professionals, threat intelligence researchers and organisations checking their own internet exposure, especially if they are comfortable with Python and command-line setup. It is a poor fit for beginners with no networking or security background, for anyone without authorisation to investigate their targets, or for people who want a fully managed cloud service, for which the project points to SpiderFoot HX.
Install options & downloads