Classify thousands of items with the Batch API

Approach

Build one request per item with a custom_id, submit them with client.messages.batches.create, poll until processing ends, then stream the results and match them back by custom_id. Results arrive in any order.

Solution

import time

import anthropic

client = anthropic.Anthropic()
SYSTEM = "Classify the support ticket as billing, technical, account or other. Reply with one word."

tickets = load_tickets()                          # [{"id": "T-1001", "text": "..."}, ...]
batch = client.messages.batches.create(requests=[
    {"custom_id": t["id"],
     "params": {"model": "claude-haiku-5-5", "max_tokens": 5, "system": SYSTEM,
                "messages": [{"role": "user", "content": t["text"]}]}}
    for t in tickets
])

while client.messages.batches.retrieve(batch.id).processing_status != "ended":
    time.sleep(60)

labels, failed = {}, []
for result in client.messages.batches.results(batch.id):
    if result.result.type == "succeeded":
        labels[result.custom_id] = result.result.message.content[0].text.strip().lower()
    else:
        failed.append(result.custom_id)              # errored / expired / canceled: resubmit these
print(len(labels), "labelled,", len(failed), "to retry")

Pitfalls

• Assuming results come back in submission order; always use custom_id.
• Using batches for interactive features; processing can take up to 24 hours.
• Not resubmitting errored or expired requests.

Variations

• Combine batches with prompt caching for a shared long system prompt.
• Validate labels against the allowed set and send invalid ones to a second pass.

Further reading: Batch processing