AI & LLM APIs Cookbook
“How would you do X?” — practical, production-ready recipes with the approach, full code, pitfalls and variations.
AI & LLM APIs
- Answer with citations that point at the source textShow users exactly which sentence in which document supports each part of the answer.
- Build hybrid RAG search on PostgreSQL with pgvectorAnswer questions from your documents using the database you already run, with good results for identifiers too.
- Cache a long system prompt and documentsStop paying full price for the same long instructions and reference documents on every request.
- Cache answers to frequent questions safelyServe the most common questions instantly and cheaply without serving stale or personal answers.
- Classify thousands of items with the Batch APIRun large, non-urgent jobs (tagging tickets, summarising documents) cheaper and without hitting rate limits.
- Compare two prompt versions with an unbiased model judgeDecide whether a new prompt is better without position bias deciding for you.
- Expose an internal API as an MCP serverMake an internal system (tickets, inventory) available to every MCP-capable AI app once, instead of per app.
- Extract structured data with a schemaTurn free-text emails or documents into validated objects your code can store.
- Fit conversation history into a token budgetLong chats eventually exceed the context window or cost too much; trim history before sending.
- Handle rate limits, overload and timeoutsKeep a feature working when the API returns 429 or 529 errors or responds slowly.
- Handle untrusted documents and emails safelySummarise or answer from content written by others without letting it instruct your model.
- Redact personal data before prompts and logsKeep card numbers, emails and keys out of model requests, logs and traces, and restore them where needed.
- Route easy requests to a smaller modelCut cost and latency by sending simple requests to a small model and hard ones to a large model.
- Run an eval set with pytestCheck a prompt change against real cases automatically before it ships.
- Store and trim chat history in a databaseKeep multi-turn conversations across requests and servers without sending ever-growing histories.
- Stream model answers to the browser with SSEShow the answer as it is generated instead of making users wait several seconds for the full reply.
- Trace LLM calls with OpenTelemetrySee every model and tool call with tokens, latency and errors in your existing tracing system.
- Track token usage and cost per feature and customerKnow which features and customers drive the AI bill, and catch runaway usage early.
- Use MCP server tools from your own appConnect to an MCP server, offer its tools to the model, and run the calls the model makes.
- Write a bounded tool-calling loopLet the model call your functions to answer, without infinite loops, crashes or invented results.