Guide · Open source

How to use proxy_pool: install, run and call the API

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

proxy_pool is an open-source Python service that regularly collects free proxy addresses from public websites, checks that they still work, and stores the working ones in Redis. This guide shows you how to set it up, run its scheduler and API, and call it from a web scraper. By the end you will know the basic setup steps and how to request a proxy from your own code.

ProxyPooljhao104/proxy_pool · ★ 23.8k · MIT Install options & downloads

Quick start

1Check prerequisites. Confirm you have Python 3.8 to 3.11 and a Redis database available.
2Clone the repository. Run git clone on https://github.com/jhao104/proxy_pool.git and enter the folder.
3Install dependencies. Run pip install -r requirements.txt inside the project folder.
4Edit setting.py. Set HOST, PORT and DB_CONN so the service can reach your Redis instance.
5Start the schedule process. Run python proxyPool.py schedule to begin collecting and testing proxies.
6Start the API server. Run python proxyPool.py server in a second terminal and call the /get endpoint.

What you need before you install proxy_pool

The README lists Python 3.8 to 3.11 as supported versions, and proxy_pool needs a running Redis database to store proxies. Setup is done by cloning the repository and editing a configuration file, so this project is aimed at developers rather than non-technical users.

Python 3.8, 3.9, 3.10 or 3.11
A Redis database
Git, to clone the repository

Install proxy_pool from the source code

Download the code with git, then install the Python dependencies listed in requirements.txt. The README also offers a releases page with zip files if you do not want to use git.

git clone https://github.com/jhao104/proxy_pool.git
cd proxy_pool
pip install -r requirements.txt

Configure the database and API in setting.py

Open setting.py in the project folder. HOST and PORT set the API listening address, and DB_CONN sets the Redis connection string. The optional PROXY_FETCHER_EXCLUDE list lets you disable specific proxy sources by their name.

HOST: the API listening IP, default 0.0.0.0
PORT: the API listening port, set to 5000 in the example
DB_CONN: the Redis connection string, for example redis://:pwd@127.0.0.1:8888/0
PROXY_FETCHER_EXCLUDE: a list of source names to skip

HOST = "0.0.0.0"
PORT = 5000
DB_CONN = 'redis://:pwd@127.0.0.1:8888/0'
# PROXY_FETCHER_EXCLUDE = ["freevpnnode"]

Run the scheduler and the API server

proxy_pool runs as two processes. The schedule process collects and tests proxies on a timer, and the server process provides the HTTP API. Start each one in its own terminal.

schedule: the collection and validation process
server: the Web API service

python proxyPool.py schedule
python proxyPool.py server

Get a proxy from the API and use it in a scraper

Once the server is running, the README describes endpoints such as /get for a random proxy, /all for every proxy, /count for the number of proxies, and /delete to remove one. The /get and /pop endpoints accept ?type=https to filter for HTTPS-capable proxies. The README suggests wrapping these calls in functions. If a request fails, the example retries and then deletes the proxy from the pool.

/get: return a random proxy
/pop: return and remove a proxy
/all: return all proxies
/count: show how many proxies are stored
/delete: remove a proxy with ?proxy=host:port

import requests

def get_proxy():
    return requests.get("http://127.0.0.1:5010/get/").json()

def delete_proxy(proxy):
    requests.get("http://127.0.0.1:5010/delete/?proxy={}".format(proxy))

Add your own proxy source

The project ships with several free proxy sources, but the README warns that free sources have limited quality. To add a source, create a .py file in fetcher/sources/ that inherits BaseFetcher, sets name, url and enabled, and yields host:port strings from fetch(). The schedule process picks up new sources on its next run. Running python proxyPool.py fetcher lists the enabled sources.

Create a file in fetcher/sources/
Inherit BaseFetcher and set name, url and enabled
Implement fetch() as a generator yielding host:port

from fetcher.baseFetcher import BaseFetcher
from util.webRequest import WebRequest

class MyProxyFetcher(BaseFetcher):
    name = "myproxy"
    url = "https://www.example.com/"
    enabled = True

    def fetch(self):
        r = WebRequest().get("https://www.example.com/api/proxies")
        for item in r.json:
            yield item["ip"] + ":" + item["port"]

What people use it for

Rotating IP addresses in a web scraper you are developing locally
Learning how scheduled crawling and proxy validation fit together
Running a self-hosted pool that you extend with your own proxy sources

Frequently asked questions

Is proxy_pool free to use?

proxy_pool is released under the MIT licence, as shown on its repository page. The proxies it collects come from free public sites, and the README says their quality is limited.

What do I need to install proxy_pool?

You need Python 3.8 to 3.11 and a Redis database. You clone the repository, run pip install -r requirements.txt, and edit setting.py for your Redis connection.

Can I run proxy_pool with Docker?

The README includes a Docker image (jhao104/proxy_pool) and a docker-compose option run from the project folder. You pass the DB_CONN environment variable to point at your Redis instance.

Are the free proxies reliable enough for production?

The README says free proxies have limited quality and recommends paid providers for production scraping. proxy_pool is better suited to experiments and learning.

Does proxy_pool keep my connection anonymous?

The README does not discuss anonymity or security. Free proxies may be untrustworthy, so do not send sensitive data through them.

What are the alternatives to proxy_pool?

The comparable projects listed for this item are scylla, proxy-pool-rs and ProxyBroker. Compare them on language, storage and how they gather proxies before choosing.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

proxy_pool suits Python developers who want a self-hosted, extensible pool for experiments, learning, or scrapers that can tolerate unreliable proxies. If you need dependable proxies for production scraping, or you are not comfortable with Python, Redis and config editing, consider a paid provider or a different tool.

Install options & downloads