Create Batch

Create a large batch of API requests for asynchronous processing

POST /v1/batches

Create a batch of API requests to be processed asynchronously. Batches are useful for processing large volumes of requests that don’t require immediate responses.

Authentication

Requires provider authentication headers:

x-portkey-provider: openai
Authorization: Bearer YOUR_OPENAI_API_KEY

Request

Headers

x-portkey-provider · string · required

The provider to route the request to (e.g., openai)

Authorization · string · required

Bearer token for the provider API

Body Parameters

input_file_id · string · required

The ID of an uploaded file that contains requests for the batch. The file must be a JSONL file with each line containing a request object.

endpoint · string · required

The endpoint to be used for all requests in the batch. Currently supports:

• /v1/chat/completions
• /v1/completions
• /v1/embeddings

completion_window · string · required

The time frame within which the batch should be processed. Currently only 24h is supported.

metadata · object

Optional metadata to attach to the batch

Response

id · string

The batch identifier

object · string

The object type, always “batch”

endpoint · string

The endpoint used for the batch

errors · object

Error information if any requests failed

input_file_id · string

The ID of the input file

completion_window · string

The completion time frame

status · string

The status of the batch: validating, in_progress, finalizing, completed, failed, or cancelled

output_file_id · string

The ID of the file containing the outputs (available when status is completed)

error_file_id · string

The ID of the file containing errors (if any)

created_at · integer

Unix timestamp of when the batch was created

in_progress_at · integer

Unix timestamp of when the batch started processing

completed_at · integer

Unix timestamp of when the batch completed

metadata · object

Custom metadata attached to the batch

Example

Input File Format

First, create a JSONL file with your requests:

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "What is 2+2?"}]}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "What is the capital of France?"}]}}

cURL

# First, upload the batch file
curl https://localhost:8787/v1/files \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F purpose="batch" \
  -F file="@batch_requests.jsonl"

# Then create the batch
curl https://localhost:8787/v1/batches \
  -H "x-portkey-provider: openai" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Response Example

{
  "id": "batch_abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-abc123",
  "completion_window": "24h",
  "status": "validating",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1713894800,
  "in_progress_at": null,
  "completed_at": null,
  "metadata": {
    "description": "Daily batch job"
  }
}

Best Practices

Batch Size Recommendations

• Keep batch sizes reasonable (1,000 - 50,000 requests)
• Monitor processing times and adjust batch sizes accordingly
• Split very large jobs into multiple batches

Error Handling

• Always check the error_file_id after batch completion
• Implement retry logic for failed requests
• Use the custom_id field to track individual requests

Cost Optimization

• Batch API typically offers 50% cost savings compared to real-time API
• Use batches for non-time-sensitive workloads
• Combine similar requests to maximize efficiency

Related Endpoints

Retrieve Batch

Check batch status and results

Cancel Batch

Cancel a running batch

List Batches

View all batches