OpenAI Batch API 50% Discount Guide: Implementation & Economics

Asynchronous Workload Arbitrage: Leveraging the 50% Batch API Discount

For non-interactive, asynchronous AI operations—such as offline document summarization, nightly embedding backfills, customer support sentiment classification, and synthetic dataset generation—sending requests through standard synchronous HTTP endpoints is an unnecessary budget drain. The OpenAI Batch API offers a flat 50% discount on all prompt and completion token rates in exchange for a 24-hour completion SLA.

In practice, over 95% of batch jobs complete within 2 hours because OpenAI routes batch jobs to idle datacenter capacity during off-peak demand valleys.

Cost Comparison: Synchronous vs 50% Batch API Rates (per 1 Million Tokens)

Complete Python Batch File Submission Script

from openai import OpenAI
import json

client = OpenAI()

# 1. Prepare JSONL tasks
tasks = [
    {"custom_id": f"task-{i}", "method": "POST", "url": "/v1/chat/completions",
     "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Extract entities: ..."}]}}
    for i in range(1000)
]
with open("batch.jsonl", "w") as f:
    for t in tasks:
        f.write(json.dumps(t) + "
")

# 2. Upload file & create batch
batch_file = client.files.create(file=open("batch.jsonl", "rb"), purpose="batch")
batch_job = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h"
)
print(f"Batch dispatched: {batch_job.id} (Status: {batch_job.status})")

Automated Python Batch Polling & Result Ingestion Script

from openai import OpenAI
import time

client = OpenAI()

def poll_batch_job(batch_id: str, poll_interval_sec: int = 60):
    while True:
        job = client.batches.retrieve(batch_id)
        print(f"Batch {batch_id} status: {job.status} ({job.request_counts.completed}/{job.request_counts.total})")
        if job.status == "completed":
            content = client.files.content(job.output_file_id)
            with open("results.jsonl", "wb") as f:
                f.write(content.read())
            print("Batch results saved to results.jsonl (50% token savings realized)")
            break
        elif job.status in ["failed", "expired", "cancelled"]:
            raise RuntimeError(f"Batch failed with status: {job.status}")
        time.sleep(poll_interval_sec)

Production Batch API SLA & Operations Checklist

1. Format Validation Pre-Flight: Validate every JSON line in your `batch.jsonl` file locally using `json.loads()` before upload. A single malformed line causes OpenAI to fail the entire file upload.
2. Embed Custom Task IDs: Always populate `custom_id` with internal database record primary keys (e.g. `doc_user123_chunk45`) to enable O(1) database reconciliations upon output download.
3. Separate Interactive from Offline Queues: Route user-facing interactive chat through standard `/v1/chat/completions` and route nightly document parsing, tagging, and embedding backfills through `/v1/batches`.
4. Handle Partial Failures: Check `job.error_file_id` upon batch completion; retry failed tasks individually using standard endpoints if necessary.

Batch Submission Retry Protocol: Implement exponential backoff when handling HTTP 429 rate limit errors during initial batch file uploads. Queue batch jobs in a local SQLite or Redis task table to guarantee idempotent re-submission if network disconnections interrupt large file transfers.