AI Engineering Tutorial · LLM Basics · Lesson 6 of 25
Streaming Responses
Show answers as they are generated, so users see progress in under a second instead of waiting for the whole reply.
Why stream
Generating a long answer can take several seconds. With streaming, the API sends the answer in small pieces as they are produced; your app shows them immediately. Total time is the same, but time to first token is much shorter, which is what users notice.
with client.messages.stream(model="claude-sonnet-5-5", max_tokens=500,
messages=[{"role": "user", "content": question}]) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
def fake_stream(text, chunk=6, ms_per_chunk=40):
elapsed = 0
for i in range(0, len(text), chunk):
elapsed += ms_per_chunk
yield elapsed, text[i:i + chunk]
answer = "Caching keeps copies of responses so repeated requests are fast."
first = None
shown = ""
for elapsed, piece in fake_stream(answer):
first = first or elapsed
shown += piece
print("time to first token:", first, "ms")
print("time to full answer:", elapsed, "ms")
print(shown)
Output
time to first token: 40 ms time to full answer: 440 ms Caching keeps copies of responses so repeated requests are fast.
Things to handle
- Partial output: validate structured output only when the stream has finished.
- Errors mid-stream: show what arrived and offer a retry.
- Forwarding to browsers: server-sent events (SSE) are the usual way to pass the stream on.
See Streaming messages.
Test yourself
What does streaming improve?
Total tokens usedTime until the user sees the first wordsModel accuracyThe price per token
Show answer
Answer: Time until the user sees the first words
Streaming doesn't change cost or quality; it shows output as soon as it's generated.