Installation

Deploy Portkey AI Gateway with your preferred method

Deployment Options

The AI Gateway can be deployed in multiple ways to suit your infrastructure needs:

NPM/Bun

Quick local development

Docker

Containerized deployment

Node.js Server

Self-hosted production server

Cloudflare Workers

Edge deployment

Kubernetes

Scalable orchestration

Cloud Providers

AWS, Azure, GCP

For a fully managed solution without infrastructure concerns, consider Portkey’s hosted gateway which processes billions of tokens daily in production.

---

Quick Start (npx)

The fastest way to get started:

1. Run with npx

   No installation required - just run:

   npx @portkey-ai/gateway
   

   Or with Bun:

   bunx @portkey-ai/gateway
   
2. Verify it's running

   The gateway starts on port 8787:

   • API: http://localhost:8787/v1
   • Console: http://localhost:8787/public/

   Test it:

   curl http://localhost:8787
   

   You should see: AI Gateway says hey!

3. Make your first request

   Now follow the quickstart guide to make your first API call.

This method is perfect for local development and testing. For production, use Docker or a Node.js server.

---

Docker

Deploy using Docker for containerized production environments.

1. Run with Docker

   Pull and run the latest image from Docker Hub:

   docker run --rm -p 8787:8787 portkeyai/gateway:latest
   

   The gateway is now running at http://localhost:8787

2. Run with environment variables

   Pass configuration via environment variables:

   docker run --rm -p 8787:8787 \
     -e REDIS_CONNECTION_STRING="redis://localhost:6379" \
     portkeyai/gateway:latest
   
3. Build from source (optional)

   Clone the repository and build your own image:

   git clone https://github.com/portkey-ai/gateway
   cd gateway
   docker build -t portkey-gateway .
   docker run --rm -p 8787:8787 portkey-gateway
   

Docker Compose

For multi-container setups with Redis caching:

1. Download docker-compose.yaml

   wget "https://raw.githubusercontent.com/Portkey-AI/gateway/main/docker-compose.yaml"
   
2. Start the services

   docker compose up -d
   

   This starts:

   • AI Gateway on port 8787
   • Redis for caching (if configured)
3. View logs

   docker compose logs -f gateway
   
4. Stop the services

   docker compose down
   

The Docker image is built from the Dockerfile which uses a multi-stage build for optimal size.

---

Node.js Server

Run the gateway as a standalone Node.js application.

1. Clone the repository

   git clone https://github.com/portkey-ai/gateway
   cd gateway
   
2. Install dependencies

   npm install
   

   Or with Bun:

   bun install
   
3. Build the project

   npm run build
   

   This compiles the TypeScript code and prepares the production bundle.

4. Start the server

   node build/start-server.js
   

   Or use npm script:

   npm run start:node
   
5. Configure (optional)

   Create a conf.json file to customize settings. See conf_sample.json for available options.

   You can also use environment variables:

   export REDIS_CONNECTION_STRING="redis://localhost:6379"
   node build/start-server.js
   

For development, use npm run dev:node which includes hot-reload.

Running as a Service

Create a systemd service file for production:

/etc/systemd/system/portkey-gateway.service

[Unit]
Description=Portkey AI Gateway
After=network.target

[Service]
Type=simple
User=portkey
WorkingDirectory=/opt/portkey-gateway
ExecStart=/usr/bin/node /opt/portkey-gateway/build/start-server.js
Restart=always
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl enable portkey-gateway
sudo systemctl start portkey-gateway
sudo systemctl status portkey-gateway

---

Cloudflare Workers

Deploy to Cloudflare’s edge network for low-latency global distribution.

1. Clone and setup

   git clone https://github.com/portkey-ai/gateway
   cd gateway
   npm install
   
2. Configure Wrangler

   Make sure you have the Wrangler CLI installed and authenticated:

   npm install -g wrangler
   wrangler login
   
3. Deploy to Cloudflare

   npm run deploy
   

   This builds and deploys the gateway to Cloudflare Workers.

4. Get your worker URL

   Wrangler will output your worker URL:

   https://your-worker.your-subdomain.workers.dev
   

Cloudflare Workers have request limits on the free tier. Consider upgrading for production use.

---

Kubernetes

Deploy to Kubernetes for production-grade orchestration.

1. Create deployment manifest

   Create deployment.yaml:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: portkey-gateway
   spec:
     replicas: 3
     selector:
       matchLabels:
         app: portkey-gateway
     template:
       metadata:
         labels:
           app: portkey-gateway
       spec:
         containers:
         - name: gateway
           image: portkeyai/gateway:latest
           ports:
           - containerPort: 8787
           env:
           - name: NODE_ENV
             value: "production"
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: portkey-gateway
   spec:
     type: LoadBalancer
     ports:
     - port: 80
       targetPort: 8787
     selector:
       app: portkey-gateway
   
2. Apply to cluster

   kubectl apply -f deployment.yaml
   
3. Verify deployment

   kubectl get deployments
   kubectl get services
   kubectl get pods
   
4. Access the gateway

   Get the external IP:

   kubectl get service portkey-gateway
   

For production, consider adding:

• Horizontal Pod Autoscaling (HPA)
• Resource limits and requests
• Ingress controller for HTTPS
• ConfigMaps for configuration
• Secrets for API keys

---

AWS EC2

Quick deployment to AWS EC2 using CloudFormation.

1. Use CloudFormation template

   The repository includes a CloudFormation template for one-click deployment:

   
2. Configure parameters

   Set:

   • VPC ID
   • Subnet ID
   • Instance Type (t2.micro for testing, t3.small for production)
3. Launch stack

   CloudFormation will:

   • Launch an EC2 instance
   • Install Docker
   • Run the gateway container
   • Configure security groups (port 8787)
4. Access your gateway

   Get the public DNS from CloudFormation outputs:

   http://<instance-public-dns>:8787
   

---

Other Platforms

Replit

Deploy with one click:

Zeabur

Use the template:

Azure, GCP, OpenShift

For enterprise deployments on:

• Azure
• Google Cloud Platform
• Red Hat OpenShift
• Other cloud providers

See the enterprise deployment guide or contact the team.

---

Configuration

Environment Variables

Common configuration options:

Variable	Description	Default
PORT	Server port	8787
REDIS_CONNECTION_STRING	Redis URL for caching	-
NODE_ENV	Environment mode	development

Configuration File

Create conf.json for advanced settings:

{
  "port": 8787,
  "cache": {
    "type": "redis",
    "connection": "redis://localhost:6379"
  }
}

See conf_sample.json in the repository for all available options.

---

Verification

After installation, verify your deployment:

1. Health check

   curl http://localhost:8787
   

   Should return: AI Gateway says hey!

2. Test API endpoint

   curl http://localhost:8787/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "x-portkey-provider: openai" \
     -H "Authorization: Bearer sk-***" \
     -d '{
       "model": "gpt-4o-mini",
       "messages": [{"role": "user", "content": "test"}]
     }'
   
3. Check the console

   Open http://localhost:8787/public/ to view the web console.

---

Next Steps

Make Your First Request

Follow the quickstart to make your first API call

Configure Routing

Learn about routing, fallbacks, and load balancing

Add Guardrails

Protect your AI apps with input/output validation

Production Deployment

Best practices for production deployments

Need help? Join our Discord community or check the GitHub repository.