Cloudflare Workers Deployment

Deploy Portkey AI Gateway to Cloudflare’s edge network

Deploy the Portkey AI Gateway to Cloudflare Workers for global distribution with sub-millisecond latency from 300+ data centers worldwide.

Overview

Cloudflare Workers provides:

• Global Edge Network - Deployed to 300+ cities worldwide
• Zero Cold Starts - Instant response times
• Auto-scaling - Handles any traffic volume
• Built-in Security - DDoS protection and WAF
• Generous Free Tier - 100,000 requests/day

Prerequisites

• Cloudflare account
• Wrangler CLI installed
• Node.js 18+ installed

Quick Deployment

1. Clone the repository

   Clone the Portkey AI Gateway repository:

   git clone https://github.com/portkey-ai/gateway
   cd gateway
   
2. Install dependencies

   Install the required npm packages:

   npm install
   
3. Authenticate with Cloudflare

   Login to your Cloudflare account via Wrangler:

   npx wrangler login
   

   This opens a browser window for authentication.

4. Deploy to Cloudflare

   Deploy the gateway:

   npm run deploy
   

   Or directly with Wrangler:

   wrangler deploy --minify src/index.ts
   

After deployment, you’ll receive a URL like https://rubeus.YOUR_SUBDOMAIN.workers.dev.

Configuration

Wrangler Configuration

The gateway includes a wrangler.toml configuration file:

wrangler.toml

name = "rubeus"
compatibility_date = "2024-12-05"
main = "src/index.ts"
compatibility_flags = [ "nodejs_compat" ]

[vars]
ENVIRONMENT = 'dev'
CUSTOM_HEADERS_TO_IGNORE = []

#
# Configuration for STAGING environment
#
[env.staging]
name = "rubeus-dev"

[env.staging.vars]
ENVIRONMENT = 'staging'
CUSTOM_HEADERS_TO_IGNORE = []

#
# Configuration for PRODUCTION environment
#
[env.production]
name = "rubeus"
logpush = true

[env.production.vars]
ENVIRONMENT = 'production'
CUSTOM_HEADERS_TO_IGNORE = []

Environment Variables

Set environment variables using Wrangler:

# Set a variable
wrangler secret put MY_SECRET

# List secrets
wrangler secret list

# Delete a secret
wrangler secret delete MY_SECRET

Deployment Environments

Deploy to different environments:

Development

npm run dev

Custom Domain

Add a custom domain to your Worker:

1. Add domain in Cloudflare Dashboard

   1. Navigate to Workers & Pages
   2. Select your Worker
   3. Go to Settings → Triggers
   4. Click “Add Custom Domain”
2. Configure DNS

   Cloudflare automatically configures DNS for domains in your account.

3. Verify deployment

   Access your gateway at your custom domain:

   curl https://gateway.yourdomain.com/health
   

Using Wrangler

Add domains via Wrangler:

wrangler domains add gateway.yourdomain.com

Development

Local Development

Run the gateway locally with Wrangler:

npm run dev

Or directly:

wrangler dev src/index.ts

The local server runs at http://localhost:8787.

Development with Node.js

For faster iteration during development:

npm run dev:node

This uses Node.js instead of the Cloudflare Workers runtime.

Bindings

KV Namespace

Add KV storage for caching:

wrangler.toml

[[kv_namespaces]]
binding = "CACHE"
id = "your_kv_namespace_id"
preview_id = "your_preview_kv_namespace_id"

Create a KV namespace:

wrangler kv:namespace create "CACHE"
wrangler kv:namespace create "CACHE" --preview

R2 Storage

Add R2 buckets for object storage:

wrangler.toml

[[r2_buckets]]
binding = "BUCKET"
bucket_name = "portkey-gateway-data"

D1 Database

Add D1 for SQL storage:

wrangler.toml

[[d1_databases]]
binding = "DB"
database_name = "portkey-gateway-db"
database_id = "your_database_id"

Monitoring and Logs

View Logs

Stream real-time logs:

wrangler tail

With filters:

# Filter by status
wrangler tail --status error

# Filter by sampling rate
wrangler tail --sampling-rate 0.5

Analytics

View analytics in the Cloudflare dashboard:

1. Navigate to Workers & Pages
2. Select your Worker
3. View the Analytics tab

Enable Logpush

For production environments, enable Logpush in wrangler.toml:

[env.production]
logpush = true

Configure destinations in the Cloudflare dashboard.

Resource Limits

Free Tier

• 100,000 requests/day
• 10ms CPU time per request
• 128 MB memory

Paid Plans

• Unlimited requests ($0.50 per million)
• 50ms CPU time per request
• 128 MB memory

Best Practices

• Use KV for caching to reduce compute time
• Minimize external API calls
• Use streaming responses for large payloads
• Implement proper error handling

Deployment Strategies

Blue-Green Deployment

Deploy to a staging environment first:

# Deploy to staging
wrangler deploy --env staging

# Test staging
curl https://rubeus-dev.YOUR_SUBDOMAIN.workers.dev/health

# Deploy to production
wrangler deploy --env production

Gradual Rollout

Use Cloudflare Workers’ percentage-based routing for gradual rollouts:

1. Deploy new version with a different name
2. Configure traffic splitting in the dashboard
3. Gradually increase traffic to the new version

CI/CD Integration

GitHub Actions

Create .github/workflows/deploy.yml:

deploy.yml

name: Deploy to Cloudflare Workers

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Setup Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '20'
      
      - name: Install dependencies
        run: npm install
      
      - name: Deploy to Cloudflare Workers
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          environment: 'production'

GitLab CI

Create .gitlab-ci.yml:

.gitlab-ci.yml

deploy:
  image: node:20
  script:
    - npm install
    - npm install -g wrangler
    - wrangler deploy --env production
  only:
    - main
  variables:
    CLOUDFLARE_API_TOKEN: $CLOUDFLARE_API_TOKEN

Troubleshooting

Build Errors

If you encounter build errors:

# Clear node_modules and reinstall
rm -rf node_modules package-lock.json
npm install

# Try building locally first
npm run build

Authentication Issues

Re-authenticate with Wrangler:

wrangler logout
wrangler login

CPU Time Exceeded

Optimize your code or upgrade to a paid plan for longer CPU time limits.

Next Steps

Cloudflare Workers Docs

Learn more about Cloudflare Workers

Configuration

Configure the gateway for your needs