Commit Graph
18569 Commits
Author SHA1 Message Date
Classic298 9ee810ad09 feat: add Staan as a native web search provider (#30138)
European deployments that need EU data residency have no hosted web search option out of the box: every plug-and-play provider shipped today is US-based, and the only sovereign alternative is self-hosting SearXNG, which means running and maintaining that infrastructure yourself. Staan (staan.ai) is a European search API with EU data residency, so adding it gives those deployments a drop-in choice.

It is configured like any other provider, through the admin UI or STAAN_API_KEY, STAAN_MARKET and STAAN_MAX_SNIPPETS. Market sets the region and language of the results and defaults to en-us. Max snippets asks Staan to fetch each result page and return semantically scored chunks of it, which get merged into that result's snippet so retrieval has more to work with; leaving it at 0 uses the plain search endpoint.

Wired the same way as Tavily and Exa, with domain filtering going through the shared get_filtered_results.

Requested in #26006.
2026-09-18 10:38:36 -05:00
Classic298 655337f7bf fix: vendor openpyxl so Excel I/O works in the Pyodide code interpreter (#30140)
openpyxl has been listed as a Pyodide package since March, but it is not part of the Pyodide distribution, so the build never wrote a wheel for it and never warned about it. In the browser code interpreter every Excel operation failed with ModuleNotFoundError: import openpyxl, pd.read_excel() and DataFrame.to_excel() alike.

It now takes the PyPI wheel path together with its dependency et-xmlfile, and the code interpreter installs it when a snippet imports openpyxl or calls pandas' Excel helpers, which need it without importing it by name.

Injected lock entries are now keyed by the canonical dashed package name, the only spelling pyodide resolves them by. As a side effect black's mypy-extensions now resolves from the bundled wheel too, so formatting Python in the editor no longer reaches out to PyPI at runtime.

Verified offline against a built static/pyodide with every network call blocked: a to_excel then read_excel roundtrip loads openpyxl and et-xmlfile from the local directory and returns the frame.

Fixes #30130
2026-09-18 10:38:02 -05:00
Classic298 95406fd28d fix: build the pgvector ivfflat index once there are rows to cluster on (#30143)
ivfflat places its centroids by clustering the rows it can see when the index is built, so an index built on an empty table gets centroids that mean nothing, and everything inserted afterwards is filed against them. On a fresh install the index is created immediately after the table, before a single chunk exists, and it is never rebuilt, so that install keeps a permanently untrained index and quietly retrieves the wrong chunks. An install that upgraded into the version introducing the index is unaffected, its table already had rows.

The index is now created once the table holds 50 rows per list, the sample size ivfflat itself aims for. Below that, and until the next start, searches fall back to an exact scan, which is correct and costs about a millisecond at that size. hnsw is untouched, it builds its graph as rows are inserted and has nothing to train on. One trade-off: the build moves from the first start to that later one, so an instance that has grown large in between pays a one-time index build during startup.

Measured on PostgreSQL 17 and pgvector 0.8 with the default lists=100 and probes=1, 384 dimensions, recall@10 against an exact scan, varying only how many rows existed when the index was built:

| rows at build | 200 | 1000 | 2500 | 5000 | 20000 |
|---|---|---|---|---|---|
| recall@10 | 0.180 | 0.563 | 0.967 | 1.000 | 1.000 |

Through PgvectorClient.search on a 20000-row table, an index built as it is today scores 0.480 against 1.000 built after the rows arrive, at the same 5 ms. An existing install can repair its index with REINDEX INDEX idx_document_chunk_vector, measured to take it from 0.480 back to 1.000.

Fixes #30134
2026-09-18 10:37:25 -05:00
Timothy Jaeryang Baek 1ddba7e2c6 refac 2026-09-18 11:31:59 -04:00
Timothy Jaeryang Baek ca1eefe293 refac 2026-09-18 11:30:15 -04:00
Timothy Jaeryang Baek 9d98ffcddf refac 2026-09-18 10:55:55 -04:00
Timothy Jaeryang Baek 7cbaabe02f refac 2026-09-18 10:51:00 -04:00
Timothy Jaeryang Baek ad9da98168 refac 2026-09-17 20:11:05 -04:00
Timothy Jaeryang Baek dbb17a5725 refac 2026-09-17 19:49:35 -04:00
Classic298 9923c53c10 fix: treat SVG uploads as documents instead of vision images (#30102)
Uploading an .svg to a chat attached it as a vision image input, so the model received a data URI it could not decode. PIL-backed servers answered "cannot identify image file" and OpenAI answered "The image data you provided does not represent a valid image". No setting made it work.

SVG now takes the ordinary file upload path, so its XML source is extracted and indexed and the model can answer questions about it. Rasterizing was the alternative and it would have discarded the part of an SVG a model reads best, the source itself. Raster formats are untouched and still go up as image inputs.

A shared helper replaces the ad hoc image/ prefix checks at the points that decide image input versus document, on both ends. It normalises the content type first, because a stored "image/SVG+xml" or a trailing charset parameter slipped past a plain comparison.

One behaviour change worth knowing: an SVG now needs the model to have the file upload capability, where before it rode in as an image.

Fixes #30100
2026-09-17 17:40:07 -04:00
Classic298 d08b69025d issue-template: require reproducing on latest AND dev right before submitting (#30104)
Reports for bugs already fixed on dev, sometimes weeks earlier, are common
because reporters check an old version once and never retest. Merges the
two soft checkboxes into one strict, time-bound requirement and warns in
the template body that unreproduced reports get closed without discussion.
2026-09-17 17:39:54 -04:00
Classic298 460f2e7634 fix: surface initial chat title generation failures in the log (#30106)
When title generation for a brand new chat fails, the chat silently keeps the provisional title (the full first user message) and nothing is written to the log at the default level, so there is no way to tell that the feature is broken rather than disabled. The failure was caught by a broad `except Exception` and reported with `log.debug`, which is invisible unless GLOBAL_LOG_LEVEL is set to DEBUG. Issue #29533 describes an outage of this path that survived four releases for exactly that reason.

This logs it with `log.exception` instead, matching how the rest of main.py reports background task failures, so an admin sees one ERROR line plus the traceback naming the real cause.

Nothing else changes: the success path is untouched, the exception is still swallowed so the detached task cannot take anything down with it, and `background_tasks_handler` does not log this exception itself, so there is no duplicate traceback.

Fixes #29533
2026-09-17 17:39:43 -04:00
Classic298 0837f310be fix: reject Docling conversions that failed inside an HTTP 200 response (#30107)
Uploading a file that Docling declines or fails to convert either dies with `TypeError: argument of type 'NoneType' is not iterable`, or silently succeeds and stores the literal string `<No text content found>` as the document's text, which then gets indexed and handed to the model as if it were the file. Docling returns the conversion outcome inside the HTTP 200 body, so checking only the HTTP status made a refused conversion look identical to a successful one, and the `errors` array that says why in plain words was never read.

Failed and skipped conversions are now rejected with the messages Docling returned, so an unsupported format surfaces as "File format not allowed: example.dxf" and the traceback is gone. The markdown field is also read as nullable, because Docling returns JSON `null` for every content format it was not asked to produce, which any Docling Parameters setting `to_formats` without `md` will hit, and that null was what raised the TypeError.

Successful conversions with empty markdown keep the existing `<No text content found>` placeholder, matching what TikaLoader and the Mistral loader already do in the same package.

Fixes #29808
2026-09-17 17:39:35 -04:00
Timothy Jaeryang Baek 58b36765a7 refac 2026-09-16 22:47:37 -04:00
Classic298 fbc4897269 refac: resolve knowledge file access from the file association (#29937)
`has_access_to_file` now derives knowledge-base access purely from the knowledge-file association, instead of also consulting the `collection_name` value stored on the file record.
2026-09-16 15:59:25 -04:00
Classic298 ff7f35a30d fix: stop logging a traceback twice per failed vector search (#29981)
A vector DB outage renders a full traceback twice for every collection and query pair. query_doc and query_doc_with_hybrid_search each log and then re-raise, and the fan-out handler that catches them logs the same exception again, so twenty knowledge bases and five expanded queries turn one outage into hundreds of identical stack traces per message on every replica.

Both helpers lose the try/except that only logged before re-raising. Each fan-out keeps reporting failures through its return value and now carries the collection name back, so one record after the gather names every collection that failed and attaches a single traceback. The hybrid path logs before its existing raise, so a total failure is still reported before the caller falls back to vector search.

Measured with every collection failing, the vector fan-out drops from 1000 traceback records for fifty collections across ten queries to one, and the hybrid fan-out from 100 for ten collections across five queries to one. Results are byte-identical across eighteen and twenty-six scenarios covering healthy, failing, partial and degenerate inputs.

Three limits worth stating. Nine of the fifteen vector clients swallow the error inside search() and return None, chroma and pgvector among them, so those backends see neither the old tracebacks nor the new record. get_sources_from_items queries one attached item at a time, so a chat with twenty knowledge bases still logs twenty records. And with hybrid search on a total outage logs once from each fan-out as the caller falls back, down from two per pair plus one.

The old "All collection queries failed" warning goes with this. The replacement cannot express that case, because a collection can fail one query and succeed another, so naming every failed collection no longer implies an empty payload.
2026-09-16 15:59:05 -04:00
Classic298 556e48e058 fix: honor false defaults on checkbox prompt variables (#30037)
A prompt template with `{{flag | checkbox:default=false}}` opened the input variables form with the checkbox already ticked, and the same happened for `False`, `"false"`, `0` and `"0"`. Template defaults arrive as strings and the checkbox was bound straight to that value, so any non-empty string counted as checked and there was no way to define a checkbox that starts unchecked once a default was given.

The checkbox now reads its ticked state from the value explicitly: it is checked only for `true` (boolean or string) and stays unchecked for anything else, while clicking it still writes a boolean like before. The seeded value itself is left alone, so what gets substituted into the prompt for an untouched form is unchanged, and typing `true` or `false` into the text field next to the checkbox now moves the tick accordingly.

Fixes #30036
2026-09-16 15:44:29 -04:00
Classic298 48fb2b84ba refac: derive a forked chat's folder from the caller's write access (#30069)
A fork copied the source chat's folder id unchanged. It now keeps that folder only when the caller has write access to it, matching what chat creation and chat moves already do, and is created outside any folder otherwise.
2026-09-16 15:42:03 -04:00
Classic298 befdd86eb1 refac: share one guarded helper for chat branch descent (#30070)
The branch navigation handlers each carried their own copy of the loop that walks a message's childrenIds down to the deepest child, eleven copies in total across the single-response view, the multi-response view and Chat.svelte. They now call one getDeepestChildId helper in the frontend utils, which tracks the ids it has already visited and stops when one repeats, the same way the message list build already does.

The helper also absorbs the "start from the last root message" fallback that two of the call sites repeated inline, so every site is now a single call.
2026-09-16 15:41:56 -04:00
Timothy Jaeryang Baek d9c8de9c39 refac 2026-09-16 10:38:46 -04:00
Timothy Jaeryang Baek fd4fc80536 refac 2026-09-16 10:27:38 -04:00
qwist1233-cpuandqwist1233-cpu a0c8593c92 i18n(tr-TR): translate 1056 missing Turkish strings (#30034)
Co-authored-by: qwist1233-cpu <267938209+qwist1233-cpu@users.noreply.github.com>
2026-09-16 10:26:16 -04:00
tsumon e669f8aefe i18n: improve Simplified Chinese translations (#30043) 2026-09-16 10:19:44 -04:00
Classic298 3348f68778 fix: keep attachment-only messages intact when skills are bound (#30045)
Sending an image or file with no typed text to a model with skills
bound replaced the (empty) message text with the list of skill names,
so the model answered with its skill catalog instead of handling the
attachment. The empty-message guard added for #24929 fired on any
empty last user text without checking for attachments.

The guard now skips the fallback when the current message carries
files, read from the user_message object the frontend sends with the
request. Attachment-only messages then go out exactly as they do on
models without skills. A bare skill selection with no attachment still
gets the fallback, so the provider 400 from #24929 stays fixed.

Request-level files were not usable as the signal: that list carries
the model's knowledge and folder files, so a bare skill selection on a
knowledge model would have gone out empty again.

Fixes #30040
2026-09-16 10:19:28 -04:00
Classic298 82da9093aa fix: default converted Responses API function tools to strict=false (#30046)
Fixes #27750

When a model runs on the Responses API, every Chat Completions style tool (workspace tools, MCP and OpenAPI servers) is converted to the Responses tool shape. Only an explicit strict value was carried over, so tools without one were sent with strict omitted. Chat Completions defaults strict to false, the Responses API defaults it to true, so the conversion silently switched loose schemas into strict mode. The model then filled every optional property (empty strings, zeros, false, empty arrays) and combined mutually exclusive filters, and those arguments went to the tool unchanged. Search and filter tools received meaningful values where omission was intended and rejected the call or returned wrong results.

The converter now defaults strict to false when the source tool does not set it. Explicit true or false values are still preserved and already native Responses tools are passed through untouched, matching what the same tool definition gets on Chat Completions.
2026-09-16 10:19:08 -04:00
Classic298 4611394fa6 refac: correct the ENABLE_CHAT_RESPONSE_STREAM_INPLACE_APPEND comments (#30066)
The comments on the opt-in in-place append made it sound like the fast path
can lose streamed text under normal operation. The only way the field can be
emptied is an allocation failure, which means the host is already out of
memory, and the default path (which copies the whole accumulated string on
every chunk) raises in that situation as well. Reword both comments to state
that condition so the flag is not read as unsafe.
2026-09-16 10:08:46 -04:00
Timothy Jaeryang Baek 3394a10b76 refac 2026-09-16 08:51:02 -04:00
Timothy Jaeryang Baek 88e78b7819 refac 2026-09-16 08:35:01 -04:00
Timothy Jaeryang Baek e9a0164690 refac 2026-09-16 00:34:24 -04:00
Timothy Jaeryang Baek 6d8e63e366 refac 2026-09-16 00:02:48 -04:00
Timothy Jaeryang Baek dfde08aa73 refac 2026-09-15 23:53:41 -04:00
Classic298 3d6598fccb fix: steer models to replace_range in the replace_note_content tool description (#30048)
Asked to add a section or change a few lines, models answer with a whole-note
replace_note_content call and the rest of the note is gone, with nothing to
undo because the editor only records versions on chat inserts. The tool
already supports replace_range operations with an expected guard, but its
description never said when to use them or how the offsets work, so models
defaulted to sending the whole note back.

The docstring, which is the description every model receives for this tool
in note chats and normal chats alike, now states the preference for range
edits and the offset, overlap and expected rules the handler enforces.
Verified the text lands in the generated tool spec unchanged and that range
edits, the expected mismatch rejection and whole-note replace behave as
described against a sqlite data dir.
2026-09-15 22:51:49 -04:00
Classic298 06d9d2e7c7 fix: sync Playwright loader spins a CPU core when a page opens a WebSocket (#30050)
Fetching a single URL through the Playwright loader (fetch_url tool, a URL attached to a
chat, the process/web endpoint) never returned when the page opened a WebSocket. The worker
thread stayed at 100% CPU for the life of the process, and every further hit cost another
core, so the whole instance got slow. Web search was unaffected, it uses the async loader.

The sync loader's websocket route handler called the synchronous close(). Playwright runs
websocket route handlers directly on its dispatcher fiber, so that call waited on the very
loop it was blocking and busy-spun forever. The handler is now a no-op: a routed socket only
reaches the network when the handler asks for it, so the page still cannot dial out, and
nothing in the handler waits on the dispatcher any more.

Aborting the upgrade request from the HTTP route handler instead does not work, page.route
never sees WebSocket handshakes and the connection goes through.

Fixes #30024
2026-09-15 22:51:37 -04:00
Timothy Jaeryang Baek 1cdd7aa459 refac 2026-09-15 22:51:15 -04:00
Timothy Jaeryang Baek 66addbd6b4 refac 2026-09-15 22:47:03 -04:00
Timothy Jaeryang Baek a096961a31 refac 2026-09-14 17:57:09 -04:00
Timothy Jaeryang Baek d25f6c7135 refac 2026-09-14 17:19:00 -04:00
Timothy Jaeryang Baek 9a93b44495 refac 2026-09-14 17:18:34 -04:00
Timothy Jaeryang Baek cc5479d16d refac 2026-09-14 17:18:26 -04:00
Timothy Jaeryang Baek dc98e3023f refac 2026-09-14 17:18:19 -04:00
Timothy Jaeryang Baek 58078ab304 refac 2026-09-14 15:47:50 -04:00
Timothy Jaeryang Baek 924a4a10fb refac 2026-09-14 15:46:31 -04:00
Timothy Jaeryang Baek 113c56fc8c refac 2026-09-14 15:46:23 -04:00
Timothy Jaeryang Baek b6cf23f332 refac 2026-09-13 23:34:35 -04:00
Timothy Jaeryang Baek e69236bccb refac 2026-09-13 23:33:22 -04:00
Classic298 55b7343be8 fix: keep a failed lock release from masking cancellation (#29979)
When Redis is unreachable at shutdown, the socket cleanup tasks do not stop. release_lock is a bare eval called from the finally of both periodic_session_pool_cleanup and periodic_usage_pool_cleanup, so it raises there and replaces the CancelledError already in flight. The usage task's except Exception then catches the Redis error and carries on reaping after shutdown cancelled it, and the session task ends with a ConnectionError in place of its cancellation.

release_lock now logs the failure and returns. aquire_lock sets the key with ex=self.timeout_secs and renew_lock re-expires it with the same value, so a release that never lands costs at most one lock timeout before another node can take over.

The except names both RedisClusterException and RedisError because the cluster-only types subclass Exception directly, and redis_cluster is a supported configuration, so RedisError alone would miss the outage on a cluster. Genuine bugs still propagate.

Verified against a real Redis: acquire, refusal while held, renew, compare-and-delete release and non-owner release are unchanged, a populated pool reaps identically with identical lock TTL lifecycles, and both tasks now cancel cleanly where before one kept running and the other died with the wrong exception.
2026-09-13 20:37:51 -05:00
Timothy Jaeryang BaekandClassic298 d372bec704 refac
Co-Authored-By: Classic298 <27028174+Classic298@users.noreply.github.com>
2026-09-13 21:37:12 -04:00
Classic298 601e0e4345 fix: stop tracking tasks under an empty item id (#29980)
An authenticated socket client can grow Open WebUI's memory without bound by sending ydoc updates for an empty document id. create_task files every task under item_tasks[id] whatever the id, while cleanup_task removes it only for a truthy id, so each update leaves a uuid behind for the process lifetime. normalize_document_id passes an empty id through, and such an id also skips the note access check.

create_task now files the task only when an id is provided, which is what its own comment already described and what the rest of the file does: redis_save_task and redis_cleanup_task both guard on a truthy item id, and stop_task normalizes a falsy one away. Nothing is filed, so nothing leaks, and cleanup_task's existing guard correctly no-ops.

This also settles a disagreement between the two backends. For an empty id, list_task_ids_by_item_id, has_active_tasks and stop_item_tasks answered one way with Redis and another without it; they now match. The visible consequence is that an instance without Redis no longer cancels a pending save for an empty document id, which is how Redis instances already behaved.

Measured over 300 calls with an empty id: 300 stale entries before, none after. Behaviour for a normal id is unchanged, including ordering, cancellation and key removal when the last task finishes.
2026-09-13 20:28:54 -05:00
Classic298 d2e62db69b fix: stop the sign-in rate limiter blocking the loop and leaking memory (#29977)
A slow Redis freezes the whole worker during sign-in, not just the user signing in. RateLimiter held a synchronous redis-py client and signin called is_limited inline from a coroutine, so every attempt did blocking round trips on the event-loop thread, with REDIS_SOCKET_TIMEOUT defaulting to None so nothing bounded the wait. Its Redis methods are now async and take the handle as their first argument, and both handlers pass request.app.state.redis, the async client the lifespan already creates. Building one in the limiter instead would pin its pooled connection to the first event loop that used it.

Without Redis, which is the default single-instance setup, the fallback store leaked. It was keyed by the rate-limit key and pruned a key's expired buckets only when that same key was checked again, so a login email never seen again was never reclaimed, and that email comes straight from an unauthenticated request body. It is now keyed by bucket, so one prune drops every key an expired bucket held, and it lives on the instance: pruning uses the per-instance num_buckets, so a shared store would let a limiter with a short window delete buckets a longer-windowed one still needs.

With a Redis costing a second per call, the widest event-loop tick gap drops from 2.010s to 0.010s and a concurrent request is answered at 0.05s instead of 2.05s, at no cost to the caller's own latency. Across 20,000 distinct keys the store goes from 40,000 entries and 6.4 MB, growing linearly, to a flat 1,004 entries and 100 KB. Rate-limiting decisions are unchanged across 700,000 randomised calls over 14 window, bucket and limit combinations, against a real Redis and the in-memory fallback alike, and sign-in still returns its first 429 on attempt 16.

Two behaviour changes worth naming. Pruning is now global rather than per key, so a wall clock that jumps forward past a full window and back forgets a hit it previously kept. The two limiters also stop sharing a store, which previously let a sign-in attempt with an IP-shaped email touch the token-exchange limiter's counters.
2026-09-13 20:28:41 -05:00
Classic298 263e56e272 fix: keep the session pool reaper alive through a Redis error (#29976)
A single Redis blip permanently stops orphaned websocket sessions from being reaped. periodic_session_pool_cleanup acquires its lock outside the try, and that try has only a finally, so the first timeout or connection reset ends the coroutine for the life of the process. The session pool then only grows, and the sole trace is one "Task exception was never retrieved" at shutdown.

The loop body gets the same try/except Exception its sibling periodic_usage_pool_cleanup already has, which also brings the lock acquire inside the guarded region. The task now logs, releases the lock and retries after the existing delay, so another node can take the lock over meanwhile.

The diff reads long because the body is re-indented one level; nothing changes beyond indentation and the four added lines. Only Redis deployments are affected, since the lock functions are lambda: True otherwise.

Verified by injecting a ConnectionError at each of the four failure points (acquire, renew, the batch scan, the reaping delete), against a real Redis as well: the task survives all four and keeps retrying, where it previously died on the first. Reaping results, lock acquire and release counts, and cancellation at shutdown are unchanged.
2026-09-13 20:28:10 -05:00