GuidesUnderstanding AI
Batch processing: thousands of AI calls, for less
Real time or batch
Real time
Batch
A batch with the Claude API
const batch = await client.messages.batches.create({
requests: products.map((p) => ({
custom_id: p.id,
params: { model: process.env.CLAUDE_MODEL!, max_tokens: 512, messages: [{ role: 'user', content: instruction(p) }] },
})),
});
// later
const status = await client.messages.batches.retrieve(batch.id);
if (status.processing_status === 'ended') {
for await (const r of await client.messages.batches.results(batch.id)) {
if (r.result.type === 'succeeded') save(r.custom_id, r.result.message);
}
}
Good use cases
| Case | Why batch |
|---|---|
| Rewrite 5,000 product pages | Nobody is waiting for the answer |
| Classify ticket archives | Large volume, one time |
| Extract data from the month’s invoices | Overnight processing |
| Replay a test set | Hundreds of questions |
The custom_id links each result to its request: results do not necessarily come back in order.
Cost: cap your AI API bill and prompt caching. The basics: call the Claude API from Node.js.
Frequently asked questions
- How long do I have to wait?
- At Anthropic, most batches finish in under an hour, with a maximum of 24 hours. Plan your processing accordingly.
- How much cheaper is it?
- Providers apply a significant discount to batches compared with standard calls. Check the current rate on their pricing page.
- Can batch be combined with prompt caching?
- Yes. When every request shares the same long beginning (instructions, examples), the cache lowers the cost further.