Guide · Open source
How to use Portkey AI Gateway: setup, routing and guardrails
Updated · written with AI from the project's own documentation and checked against our catalogue data
Portkey's AI Gateway is an open-source layer that sits between your app and large language model providers. Your app sends requests to one OpenAI-style API, and the gateway forwards them to the provider you choose. This guide shows you how to start it locally, send a first request, add retries and guardrails, and read the request logs.
Portkey AI GatewayPortkey-AI/gateway · ★ 13.3k · MIT Install options & downloads Quick start
- 1Install Node.js and npm. The gateway needs Node.js and npm. Make sure both are available in your terminal before you continue.
- 2Start the gateway. Run npx @portkey-ai/gateway. The API runs at localhost port 8787 under /v1.
- 3Install the Python client. If you use Python, run pip install -qU portkey-ai.
- 4Send a request. Create a Portkey client with your provider and API key, then call chat.completions.create with a model name.
- 5Add a config. Attach a config with retry attempts or output guardrails using with_options, then test the reply.
- 6Review logs. Open http://localhost:8787/public/ to see your local request logs in the Gateway Console.
What the AI Gateway does for your app
Without a gateway, each model provider has its own client and error handling. The Portkey gateway gives your app one interface and handles routing in the middle. The README describes features such as automatic retries, fallbacks, load balancing and guardrails that check inputs or outputs.
It is written in TypeScript and released under the MIT licence. It is aimed at developers who call several providers and want one interface, or teams that need retries or fallbacks when a provider fails.
Start the gateway locally with npx
You need Node.js and npm installed. Run the command below in a terminal to start the gateway on your own machine.
When it starts, the README says the gateway runs at http://localhost:8787/v1 and the Gateway Console runs at http://localhost:8787/public/.
- Gateway API: http://localhost:8787/v1
- Gateway Console: http://localhost:8787/public/
npx @portkey-ai/gatewaySend your first request from Python
The README's quickstart uses the Python client. Install it first with pip, then point the client at a provider and pass your provider API key. The request is sent through the gateway to the model you name.
Python is only needed if you use this client. The gateway itself runs on Node.js.
pip install -qU portkey-ai
from portkey_ai import Portkey
client = Portkey(
provider="openai",
Authorization="sk-***"
)
client.chat.completions.create(
messages=[{"role": "user", "content": "What's the weather like?"}],
model="gpt-4o-mini"
)Add retries and a guardrail with a config
Configs let you set routing rules, reliability settings and guardrails. The README example sets up to 5 retry attempts and an output guardrail that denies any reply containing the word Apple.
According to the README, with this config the reply would always be Bat, because replies containing Apple are denied and the retry setting tries again until a valid reply is returned or attempts run out.
- retry: attempts set to 5
- output_guardrails: a default.contains check that denies replies with the word Apple
config = {
"retry": {"attempts": 5},
"output_guardrails": [{
"default.contains": {"operator": "none", "words": ["Apple"]},
"deny": True
}]
}
client = client.with_options(config=config)Check requests in the Gateway Console
The README says the Gateway Console shows all your local logs in one place. Open http://localhost:8787/public/ while the gateway is running to view them.
This is useful when you want to see which requests were sent and what came back while you test routing or guardrail settings.
Know when to use the open-source gateway or the enterprise version
The README lists deployment guides for Docker, Node.js server, Cloudflare Workers, Replit and others, which are linked from the repository docs. It also mentions a Portkey Cloud option and a separate Enterprise version for private deployments.
The README places org management and similar governance features in the Enterprise version. If you only need the open-source gateway for routing, retries and guardrails, you do not need that version.
- Docker, Node.js server, Cloudflare Workers and Replit deployment guides are linked in the README
- Enterprise features such as org management are described as a separate paid version
What people use it for
- Calling several LLM providers through one OpenAI-style interface
- Retrying a request automatically when a model provider fails
- Denying model replies that contain words you do not want, using an output guardrail
- Viewing local request logs while testing routing rules in the Gateway Console
Frequently asked questions
Is Portkey AI Gateway free?
The repository is listed under the MIT licence, which is an open-source licence. The README separately describes an Enterprise version for private deployments with org management and other features, which is a paid offering.
What do I need to install Portkey AI Gateway?
The README lists Node.js and npm as prerequisites for running the gateway with npx. Python is only needed if you use the Python client example, which is installed with pip install -qU portkey-ai.
Do I need to know how to code to use it?
Yes. The setup assumes familiarity with code and calling APIs. You start the gateway from a terminal and send requests from your own code.
What is the difference between Portkey AI Gateway and LiteLLM?
Both sit between your app and LLM providers. This guide does not compare features in detail, so check each project's documentation against your needs. LiteLLM is listed as a comparable project in this directory.
What is the Gateway Console?
The README says it is a console at http://localhost:8787/public/ that shows all of your local request logs in one place when the gateway is running locally.
Does Portkey AI Gateway work if I only use one model provider?
The README describes the gateway as adding a layer to manage. If you only call one model from one provider and do not need routing, retries or fallbacks, a direct API client may be simpler.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Portkey AI Gateway suits developers who call several LLM providers, want one OpenAI-style interface, or need retries, fallbacks and output guardrails. It is a poor fit if you only call one model from one provider or if you are not comfortable running Node.js and calling APIs; in those cases a direct provider client or a simpler tool like OpenRouter may be easier to manage.
Install options & downloads