Ollama HTTP API：stream=true 真的比較快嗎？M1 Pro 實測拆給你看Ollama HTTP API: Is stream=true Actually Faster? M1 Pro Numbers

Setup

Two prompts:

• Short: "Answer in one sentence: what is Traditional Chinese?"
• Long: "In 400-500 characters across at least three paragraphs, explain how Apple Silicon's unified memory affects local LLM inference."

Two modes:

• stream: true — push tokens as they're generated
• stream: false — wait for full generation, return one JSON

Script: tests/scripts/run_streaming_test.py. Raw data: tests/outputs/results.jsonl.

The numbers (median of 5)

Two things up front:

1. tokens/s is nearly identical across modes (short 44-45, long 50-51) — streaming doesn't make the model go faster.
2. Total time is nearly identical — streaming doesn't reduce total work.

The only thing streaming changes is when the first token arrives. On the long prompt, blocking makes the client stare at nothing for 19.5 seconds; stream starts producing output at 9.7 seconds — the user sees half the wait.

On the short prompt the gap disappears: ~7.7 seconds either way. The response is short enough that streaming's head-start has nothing to amortize against.

Decision rule

"How do I know the length in advance" is a fair question:

• Summaries, tags, short translations → mostly < 200 tokens → blocking is enough
• Long-form rewrites, code generation, explanations → mostly 500+ tokens → stream
• When uncertain, default to stream — extra streaming code on the client is cheaper than debugging blocking-mode timeouts

A side finding: the model ignores "one sentence"

Side data: my prompt asked "Answer in one sentence: what is Traditional Chinese?" — intended ~50-token response. gemma4:e2b returned 324-349 tokens (a paragraph) in both modes.

Not this article's main point, but worth flagging: length constraints in prompts are unreliable on small models. If your client needs short output, either truncate post-hoc, or tighten the prompt format ("output exactly N characters", "no explanation"). From a streaming perspective: even "short" prompts can quietly turn into 7-8 second medium responses — which is already in the "default to stream when uncertain" zone.

Code

Stream (Python)

import json, urllib.request

body = json.dumps({
    "model": "gemma4:e2b",
    "prompt": "...",
    "stream": True,
}).encode()
req = urllib.request.Request(
    "http://localhost:11434/api/generate",
    data=body,
    headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(req) as resp:
    for line in resp:
        chunk = json.loads(line)
        print(chunk.get("response", ""), end="", flush=True)
        if chunk.get("done"):
            break

Blocking (Python)

import json, urllib.request

body = json.dumps({
    "model": "gemma4:e2b",
    "prompt": "...",
    "stream": False,
}).encode()
req = urllib.request.Request(
    "http://localhost:11434/api/generate",
    data=body,
    headers={"Content-Type": "application/json"},
)
with urllib.request.urlopen(req) as resp:
    payload = json.loads(resp.read())
print(payload["response"])

Both use stdlib — no extra deps. Streaming adds NDJSON line-by-line parsing and a done: true exit check — roughly 5 extra lines.

FAQ

Q: Doesn't the streaming client code slow things down? A: NDJSON parsing and the extra prints/flushes are negligible on Apple Silicon. Measured stream totals are only ~0.5s (short) or ~0.6s (long) above blocking. That gap lives in the post-final-chunk assembly on the client, not in inference, and it's marginal next to model time.

Q: Will 12B results look the same? A: Same principle (stream = identical total time, earlier first token), but 12B runs slower on M1 Pro (~12-15 tok/s vs e2b's ~50), so the crossover shifts down — even a 200-token response can land in "should stream" territory.

Q: Does /api/chat behave the same as /api/generate? A: Should match. The underlying inference engine and streaming mechanism are the same. /api/chat adds message-history structure but streaming behaves identically.

Verdict

Waveform: STEADY_PULSE

Rules:

• Expected response > 5s → stream
• Expected response < 3s → blocking
• In between → either, pick whichever client code is cleaner
• When uncertain → default to stream

Streaming is a UX optimization, not a performance one. It turns "waiting" into "watching" — a meaningful distinction for local LLMs, which run 2-5x slower than cloud APIs and where users tolerate "watching" far better than "waiting."

Ollama HTTP API docs →