fix: skip image, media and font requests in the Playwright web loader (#29742)

Fetching a page with many media files through the Playwright web loader was extremely slow or timed out, while raw Playwright loaded the same page in a couple of seconds. The loader routes every request the page makes through the backend HTTP client and downloads the full body before the browser sees any of it, and the browser cancelling a media request once it has enough never reaches that download. On a page with a few dozen audio players every file was pulled in full for a text extraction that never reads it.

Image, media and font requests are now aborted in the interceptor before any fetch is made. None of them feed the text extraction. Measured on the page from the report with the default timeout on the same connection:

| | requests fetched | bytes downloaded | elapsed |
|---|---|---|---|
| before | 151 | 55.0 MB | 10.1 s |
| after | 45 | 3.9 MB | 2.6 s |

Fixes #29741

backend/open_webui/retrieval/web/utils.py

@@ -304,6 +304,9 @@ _DROPPED_REQUEST_HEADERS = {'accept-encoding', 'connection', 'content-length', '
 # The clients hand us a decoded body, so the sender's framing no longer describes it.
 _DROPPED_RESPONSE_HEADERS = {'connection', 'content-encoding', 'content-length', 'transfer-encoding'}
 
+# The Playwright loader only reads the page HTML, which none of these feed.
+_DROPPED_RESOURCE_TYPES = {'font', 'image', 'media'}
+
 
 def _forwardable_request_headers(headers: Dict[str, str]) -> Dict[str, str]:
     return {name: value for name, value in headers.items() if name.lower() not in _DROPPED_REQUEST_HEADERS}
@@ -725,6 +728,9 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing
 
     def _intercept_navigation_sync(self, route, session):
         req = route.request
+        if req.resource_type in _DROPPED_RESOURCE_TYPES:
+            route.abort()
+            return
 
         hop_cookies: List[Tuple[str, str]] = []
 
@@ -779,6 +785,9 @@ class SafePlaywrightURLLoader(PlaywrightURLLoader, RateLimitMixin, URLProcessing
 
     async def _intercept_navigation(self, route, session):
         req = route.request
+        if req.resource_type in _DROPPED_RESOURCE_TYPES:
+            await route.abort()
+            return
 
         hop_cookies: List[Tuple[str, str]] = []