Node.js Deployment

Deploy Portkey AI Gateway as a standalone Node.js server

The Portkey AI Gateway can be deployed as a standalone Node.js server, giving you full control over your hosting environment.

Quick Start

The fastest way to run the gateway locally:

npm

npx @portkey-ai/gateway

The gateway will start on http://localhost:8787.

From Source Deployment

For production deployments, build and run from source:

1. Clone the repository

   Clone the Portkey AI Gateway repository:

   git clone https://github.com/portkey-ai/gateway
   cd gateway
   
2. Install dependencies

   Install the required npm packages:

   npm install
   

   Or with bun for faster installation:

   bun install
   
3. Build the project

   Build the production bundle:

   npm run build
   

   This creates optimized files in the build/ directory.

4. Start the server

   Run the production server:

   node build/start-server.js
   

   Or use the npm script:

   npm run start:node
   

Development Mode

For development and testing, use the development server:

npm run dev:node

This starts the server with hot-reloading using tsx.

Configuration

Environment Variables

Configure the gateway using environment variables:

LOG_LEVEL=info \
NODE_ENV=production \
PORT=8787 \
node build/start-server.js

Configuration File

Create a conf.json file for advanced configuration:

conf.json

{
  "port": 8787,
  "logLevel": "info",
  "cors": {
    "enabled": true,
    "origins": ["*"]
  }
}

Place this file in the project root directory.

Process Management

Using PM2

PM2 is a production process manager for Node.js applications.

1. Install PM2

   npm install -g pm2
   
2. Start with PM2

   pm2 start build/start-server.js --name portkey-gateway
   
3. Configure auto-restart

   Save the process list and configure startup:

   pm2 save
   pm2 startup
   

PM2 Ecosystem File

Create an ecosystem.config.js file for advanced PM2 configuration:

ecosystem.config.js

module.exports = {
  apps: [{
    name: 'portkey-gateway',
    script: './build/start-server.js',
    instances: 'max',
    exec_mode: 'cluster',
    env: {
      NODE_ENV: 'production',
      PORT: 8787
    },
    error_file: './logs/error.log',
    out_file: './logs/output.log',
    log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
    merge_logs: true,
    max_memory_restart: '1G'
  }]
}

Start with the ecosystem file:

pm2 start ecosystem.config.js

PM2 Management Commands

# View status
pm2 status

# View logs
pm2 logs portkey-gateway

# Restart
pm2 restart portkey-gateway

# Stop
pm2 stop portkey-gateway

# Delete
pm2 delete portkey-gateway

# Monitor
pm2 monit

Systemd Service

For Linux systems, create a systemd service:

Create Service File

Create /etc/systemd/system/portkey-gateway.service:

portkey-gateway.service

[Unit]
Description=Portkey AI Gateway
After=network.target

[Service]
Type=simple
User=nodejs
WorkingDirectory=/opt/portkey-gateway
ExecStart=/usr/bin/node build/start-server.js
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=portkey-gateway
Environment=NODE_ENV=production
Environment=PORT=8787

[Install]
WantedBy=multi-user.target

Manage the Service

1. Reload systemd

   sudo systemctl daemon-reload
   
2. Enable on boot

   sudo systemctl enable portkey-gateway
   
3. Start the service

   sudo systemctl start portkey-gateway
   
4. Check status

   sudo systemctl status portkey-gateway
   

Reverse Proxy Setup

Nginx

Configure Nginx as a reverse proxy:

/etc/nginx/sites-available/portkey-gateway

server {
    listen 80;
    server_name gateway.yourdomain.com;

    location / {
        proxy_pass http://localhost:8787;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
    }
}

Enable the site:

sudo ln -s /etc/nginx/sites-available/portkey-gateway /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Add SSL with Certbot

sudo certbot --nginx -d gateway.yourdomain.com

Performance Tuning

Node.js Options

Optimize Node.js for production:

node --max-old-space-size=4096 \
     --max-http-header-size=16384 \
     build/start-server.js

Clustering

Utilize multiple CPU cores by running multiple instances behind a load balancer, or use PM2’s cluster mode as shown above.

Monitoring

View Logs

# With PM2
pm2 logs portkey-gateway

# With systemd
sudo journalctl -u portkey-gateway -f

# Direct logs
tail -f logs/*.log

Health Check

Check if the gateway is running:

curl http://localhost:8787/health

Troubleshooting

Port Already in Use

If port 8787 is already in use, change it:

PORT=8788 node build/start-server.js

Memory Issues

Increase Node.js heap size:

node --max-old-space-size=8192 build/start-server.js

Permission Errors

Ensure the user has proper permissions:

sudo chown -R nodejs:nodejs /opt/portkey-gateway

Next Steps

Docker Deployment

Containerize your Node.js deployment

Configuration

Configure the gateway for production