Some providers send one very large piece of a streamed answer in a single go: a long reasoning trace, a code execution result, a turn with many tool calls, or a response echo carrying a big tool list. Anything past 128 KB in one line killed the chat mid-answer with a misleading `400, message: Got more than 131072 bytes when reading`. Nothing was rejected upstream, that is our own reader giving up on an oversized line.
Open WebUI already had code that assembles lines itself with no such limit, but it only ran when CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE was set. Unset is the default, and in that case the raw capped reader was used instead, so a default install always broke. That path now always assembles lines, and the setting goes back to being what its name says: an optional cap, off by default. It applies to the Ollama stream as well, since both now share the same reader.
The assembly loop only splits once a line actually completes, because the old one re-concatenated and re-split the whole buffer on every network chunk. Without that, allowing long lines would have traded an error for multi-second event loop stalls.
| | 20 MB in one line | 200k small lines |
| --- | --- | --- |
| before | 4249 ms | 27.3 ms |
| after | 37 ms | 25.2 ms |
stream_wrapper without a content handler iterates aiohttp's response.content, which reads line by line: every line costs a buffer scan, a slice, a bytes concat, a generator resume and its own ASGI response message. A typical SSE event is two lines (the data line and the blank separator), so every upstream token event became two yields and two transport writes even on routes where the body is never inspected.
stream_wrapper now takes passthrough=True, which iterates response.content.iter_any(): the exact same bytes, one yield per network read, no line scanning. It is applied only to routes no internal consumer parses line-by-line: the ollama pull/push/create/generate proxies and its v1 completions, chat completions, messages and responses endpoints, plus the openai embeddings, responses and catch-all proxies. The two internally consumed chat routes keep line iteration, which the streaming middleware and the Ollama-to-OpenAI converter require; the ollama send_request signature documents that constraint.
Benchmark (local aiohttp SSE server, 500 events, consumed through stream_wrapper):
| metric | before (readline) | after (iter_any) |
| --- | --- | --- |
| stream consumption time | 1.46 ms | 0.62 ms |
| generator yields + response writes per stream | 1000 | 1 |
The single yield is a loopback artifact (the whole body arrives in one buffered read); over a real network it becomes one yield per TCP read instead of two per SSE event.
Functionally verified: line mode and passthrough mode produce byte-identical output for the same stream, and passthrough always yields fewer, larger chunks.
Classic298andGitHub
Timothy Jaeryang Baek