diff --git a/sandbox/plans/interview-translation-forwarding.md b/sandbox/plans/interview-translation-forwarding.md new file mode 100644 index 000000000000..be2a03f89bf3 --- /dev/null +++ b/sandbox/plans/interview-translation-forwarding.md @@ -0,0 +1,200 @@ +# Brainstorm — Sandbox translation forwarding + +## Topic + +Forward an integration's translations from the sandbox subprocess into the +main HA instance so the frontend renders translated strings for sandboxed +integrations — entity names, entity-state translations, config/options flow, +selectors, services, exceptions, issues, etc. + +## Problem / Why + +The sandbox runs an integration's code (setup, config flow, entities, +services) in an isolated subprocess while main keeps the unified frontend. +Translations in HA are loaded from each integration's on-disk +`translations/.json`, **keyed by integration domain**, and served to the +frontend via `frontend/get_translations`. For sandboxed integrations the +strings don't reach the frontend today: + +- **Built-in** integrations: the translation files *do* exist on main's disk + (same bundled `homeassistant` package), but the domain isn't in main's + `hass.config.components` (it ran in the sandbox), so the entity/state + category never loads for it. Files are present; loading is the gap. +- **Custom** (HACS) integrations: the code — and its `translations/` dir — is + fetched into the *sandbox's* `/custom_components/` per the + integration-source design. The files **do not exist on main at all**. + Strings genuinely have to cross the wire. Core's `title`→`integration.name` + fallback (`translation.py:119-124`) also can't run on main (no `Integration` + object for the custom domain). + +## Grounding (code facts) + +- `homeassistant/helpers/translation.py` + - `_TranslationCache` is lazy and **per-language**; once a (language, domain) + is loaded it's cached forever (translations are never unloaded). + - `_async_load` → `async_get_integrations` (needs the `Integration` object) + → `_async_get_component_strings` (reads `translations/.json` from the + integration's dir). Both steps break for a custom domain absent on main. + - `async_get_translations(..., category, integrations, config_flow)`: + with `config_flow=True` and no integration list, + `components = async_get_config_flows(hass) - hass.config.components` + → a **bulk** set of every config-flow integration. + - Categories are **not a fixed enum** — `build_resources` slices whatever + top-level keys exist in the strings file (`config`, `options`, `selector`, + `title`, `entity`, `state`, `services`, `exceptions`, `issues`, …). + - `title` fallback: missing `title` ⇒ `component..title = + integration.name` (line 119-124). Unavailable for custom on main. +- Frontend (`../frontend`): + - Add-integration **picker**: `dialog-add-integration.ts:606,640` + → `loadBackendTranslation("title", discoveredHandlers, true)` — loads only + the **`title`** category, in **bulk**, for all handlers; names also come + from the static integrations index (`integration.name` / + `domainToName`). No `config` category at picker time. + - **Running** flow: `show-dialog-config-flow.ts:29-46` + → loads `config` + `selector` + `title` for a **single** `handler`. By + then that integration's sandbox is alive. +- Existing extensibility pattern to mirror: `sources.py` + `async_register_sandbox_source_resolver` (HACS-agnostic resolver hook that + maps a custom domain → git source). The `router` attribute on + `ConfigEntries` and the `current_sandbox` ContextVar are the other + precedents for a small declared core hook. + +## Decisions so far + +1. **Scope = both groups, unified path** (user). One conceptual mechanism for + built-in and custom rather than per-group branching — subject to the picker + carve-out below. +2. **Transport = pull / RPC on demand** (user). Main intercepts translation + loading for a sandboxed *running* domain and issues a + `sandbox/get_translations(domain, language)` RPC; the sandbox reads its + local `translations/.json` and returns it. Matches the lazy + per-language cache; only fetches what the frontend asks for; couples + availability to the sandbox being alive (it is, when entities/flows are + active). +3. **Picker is a separate, static seam** (user insight). The add-integration + dialog only needs the **`title`** category and must work when no sandbox is + running. So: + - built-in → main reads `title` from its own disk (unchanged); + - custom → the **source resolver / index** supplies the picker `title` + (and any minimal picker strings), exactly like it already supplies the + git install source. No sandbox spawn to render the picker. + - Once the user *starts* a flow, the sandbox spawns and the full `config` / + `selector` / `title` strings come over the live pull-RPC. + +## Resulting shape (two seams) + +- **Seam A — static picker strings (no live sandbox):** extend the + integration-source resolver/index so a custom integration contributes its + `title` (picker name + minimal strings) to main. Built-in stays disk-served. +- **Seam B — live pull-RPC (sandbox running):** a declared core hook in the + translation loader routes a sandboxed domain's (language) strings request to + the bridge → `sandbox/get_translations` → sandbox reads local file → returns + the whole strings dict; main caches it in the existing `_TranslationCache` + and `build_resources` slices by category as usual. + +## Open questions / edges + +- **RPC granularity:** return the *whole* strings dict for (domain, language) + in one shot (simplest; main slices) vs per-category. Whole-dict matches how + main reads the file today. +- **Domain → sandbox-group resolution on main** for a running domain: derive + from `ConfigEntry.sandbox` / the bridge's registered-entity map. Pre-entry + (flow in progress) the router already knows the group. +- **Cache invalidation on custom-integration update** (new sha → changed + strings): core never invalidates translation cache; a sandbox restart with a + new ref may need an explicit drop of cached (domain, *) strings. +- **Sandbox liveness for non-flow loads:** entity/state/exceptions strings for + a loaded sandboxed entry — sandbox is running, fine. Guard the core hook to + only redirect domains that are actually sandboxed-and-owned, so a + not-running / main domain still reads disk. +- **`title` fallback** (`integration.name`) for custom on main — must come + from Seam A, since main has no `Integration` object. +- **Custom-integration discovery on main** (whether a custom domain even + appears in `async_get_config_flows` when its code lives only in the sandbox) + is an adjacent unknown that Seam A's index likely also has to feed. + +## Research findings (2 agents) + +Full notes: `research/translation-forwarding-core-seam.md`, +`research/translation-forwarding-discovery-and-index.md`. + +### Seam B — live pull-RPC (sandbox running) — well-supported + +- **Silent-vanish today:** `async_get_integrations` returns `IntegrationNotFound` + *as the dict value* (not raised, not cached — `loader.py:1441-1447`); + `_async_load` skips Exception-valued domains (`translation.py:221-227`); + `_async_get_component_strings` does `integrations.get(domain)` → a custom + sandboxed domain silently yields `{}` and its frontend strings just disappear. +- **Recommended seam:** branch **inside `_async_load`** + (`translation.py:208-253`) right after `async_get_integrations`, overlaying the + RPC result onto `translation_by_language_strings` *before* + `_build_category_cache` (which owns EN-fallback expansion + cache bookkeeping + the sandboxed domain must share). + - *Antithesis:* runs under the cache lock → a per-domain RPC serialises + latency-sensitive frontend loads. Mitigate: **batch per group**, and + **degrade to empty on a dead channel** (never block the picker/frontend). +- **RPC shape:** `{language: {domain: raw strings.json dict}}`, un-flattened, + with `title` pre-filled **sandbox-side** (main can't run the + `integration.name` fallback at `translation.py:118-124` for a custom domain). +- **Domain→group is already wired:** `ConfigEntry.sandbox` + (`config_entries.py:432`) → `SandboxData.bridges[group]` + (`sandbox/__init__.py:38-45`); pre-entry (flow in progress) the group comes + from the active `SandboxFlowProxy` / `_assignment_for_new_flow` + (`router.py:189-201`). No new index needed for the live path. +- **Cache invalidation:** there is **no eviction API** — `loaded`/`cache` only + grow (`translation.py:168-171`). A custom integration re-fetched at a new + commit sha (changed strings) needs a new `async_invalidate(components)` to + drop stale `(domain, *)` entries. Minimal addition. + +### Seam A — picker — bigger than translations + +- **Reframe:** the picker does **not** use `async_get_config_flows`. It calls + WS `integration/descriptions` → `async_get_integration_descriptions` + (`loader.py:416-460`), built from the generated `integrations.json` (core) + + `async_get_custom_components` — a **disk scan of `/custom_components`** + (`loader.py:325-343`). A custom integration whose code lives **only in the + sandbox appears in none of these** (descriptions, config-flows, or index). + *"Today the picker works by accident because HACS still drops code on disk; + the stateless-sandbox future breaks it."* So the picker gap is + **discoverability**, not just translation — the `title` strings are a subset + of the catalog metadata main is missing. +- **Recommended shape:** a **separate** `async_register_sandbox_catalog_provider` + hook (eager, enumerable, display-only) rather than overloading the + security-critical, sha-pinned `IntegrationSourceDict` source resolver. Core + merges the catalog into `async_get_integration_descriptions` + the `title` + fallback; HACS fills it. + - *Antithesis:* HACS reliably has the manifest `name` but may **not** have + `translations/*.json` indexed (they live inside the un-fetched tarball) — so + `title_translations` must be **optional**, degrading to `name` via the + existing fallback. A wrong/missing name is cosmetic (unlike `ref`), so no + strict validation needed. + +## Converged approaches + +**Approach 1 — Seam B first (live pull-RPC only).** Self-contained, +shippable. Covers entity names/state, running config/options flow, selectors, +services, exceptions, issues for any sandboxed integration whose sandbox is +alive. Built-in flows work (they run in the sandbox). Custom integrations work +*once their flow is running / entry is loaded*. Does **not** fix the +not-running picker for sandbox-only customs — but that's already broken for +*discoverability* independent of translations, so this doesn't regress +anything. Smallest core surface: one `_async_load` branch + a provider hook + +`async_invalidate`. + +**Approach 2 — Both seams (B + catalog provider A).** Adds +`async_register_sandbox_catalog_provider` so stateless custom integrations are +discoverable *and* titled in the picker without a sandbox. Larger surface; +overlaps the broader "how do sandbox-only custom integrations appear on main" +question, which is arguably its own feature beyond translations. + +**Open sub-decisions (either approach):** +- Built-in in Seam B: read main's local disk (skip the redundant RPC, one + `is_built_in` branch) vs uniform pull. Research leans local-disk for built-in + — main has byte-identical files. +- Cache invalidation now (`async_invalidate` on sha change) vs defer. +- RPC granularity: whole strings dict per (domain, language) — confirmed + simplest, matches how main reads the file. + +## Coverage + +What 2/2 · Why 2/2 · Scope 2/2 · Where 2/2 · How 2/2 · Edge 2/2 (12/12) diff --git a/sandbox/plans/plan-translation-forwarding.md b/sandbox/plans/plan-translation-forwarding.md new file mode 100644 index 000000000000..28bc250a7b96 --- /dev/null +++ b/sandbox/plans/plan-translation-forwarding.md @@ -0,0 +1,191 @@ +# Plan — Sandbox translation forwarding (both seams) + +> Source: `/phx:brainstorm` → `/phx:plan` (adapted to Python / Home Assistant +> core — not Elixir/Phoenix). Brainstorm + research are complete; this plan +> consumes them rather than re-deriving. +> +> Inputs: `interview-translation-forwarding.md`, +> `research/translation-forwarding-core-seam.md`, +> `research/translation-forwarding-discovery-and-index.md`. +> Scratchpad: `scratchpad-translation-forwarding.md`. +> +> **Locked decisions:** both seams in scope · pull/RPC transport · built-in +> reads local disk in the live path (RPC reserved for customs) · separate +> catalog-provider hook for the picker · whole-strings-dict RPC granularity. + +## Problem + +A sandboxed integration runs in an isolated subprocess; its +`translations/.json` is keyed by integration domain and served to the +frontend via `frontend/get_translations`. Today those strings don't reach the +frontend: + +- **Custom** sandboxed integration → main has no `Integration` object; + `async_get_integrations` returns `IntegrationNotFound` *as the dict value* + (`loader.py:1441-1447`), `_async_load` skips Exception-valued domains + (`translation.py:221-227`), so strings silently vanish (`{}`). +- **Picker** → the add-integration dialog is built from `integration/ + descriptions` (a disk scan of `/custom_components`, + `loader.py:325-343,416-460`); a sandbox-only custom appears in *none* of + main's lists — a **discoverability** gap, of which `title` is a subset. + +## Goal & success criteria + +- [ ] Live: entity names/state, running config/options flow, selectors, + services, exceptions, issues resolve for built-in *and* custom sandboxed + integrations whose sandbox is alive. +- [ ] Picker: a sandbox-only custom integration is discoverable + named in the + add-integration dialog with **no sandbox spawn**. +- [ ] No regression for non-sandboxed integrations (disk path unchanged). +- [ ] **Iron Law:** public declared hooks only — no monkey-patching private + internals (the sandbox subsystem's standing rule). + +--- + +## Phase B — live pull-RPC (ship first; self-contained) + +### B1 · Wire protocol `[protocol]` +- [ ] Add `sandbox/get_translations` to `proto/sandbox.proto`; regenerate + `_proto/sandbox_pb2.py(i)` via `proto/generate.sh` (run + `proto/check_drift.sh`). +- [ ] Mirror the constant in **both** `protocol.py` files + (`homeassistant/components/sandbox/protocol.py` + + `sandbox/hass_client/hass_client/protocol.py`). +- [ ] Shapes — request `{ language: str, domains: [str] }` (batched per group); + response `{ language: str, strings: { domain: } }`, + un-flattened, `title` pre-filled. + +### B2 · Sandbox handler `[hass_client]` +- [ ] Register a `sandbox/get_translations` handler in `SandboxRuntime`. +- [ ] For each domain, load raw strings for `language` from the sandbox's own + filesystem — built-in from the bundled package, custom from the fetched + `/custom_components/` — reusing core's + `_async_get_component_strings` / `component_translation_path` against the + sandbox's private `hass`, or reading the file directly. +- [ ] **Pre-fill `title`**: if absent, inject `integration.name` (the sandbox + *has* the `Integration`; `translation.py:118-124` can't run on main for + customs). + +### B3 · Core translation hook `[core: homeassistant/helpers/translation.py]` +- [ ] Add `async_register_sandbox_translation_provider(hass, provider)`, + mirroring the `sources.py` resolver convention (HassKey + unregister + callback). `provider(language, components) -> {language: {domain: raw_strings}}`, + returning only the domains it owns. +- [ ] In `_TranslationCache._async_load` (`:208-253`), **after** + `async_get_integrations` and **before** `_build_category_cache`, call the + provider and overlay its result onto `translation_by_language_strings`. + Provider-claimed domains bypass the disk/`IntegrationNotFound` path; + everything else unchanged. +- [ ] **Batch per group** + **degrade to empty on a dead channel** — the + overlay runs under the cache lock; never block the frontend. +- [ ] Add `_TranslationCache.async_invalidate(components)` (+ module wrapper): + discard from `loaded[*]` and `del cache[*][*][component]` (no eviction API + exists today — `:168-171`). + +### B4 · Provider impl + registration `[sandbox: bridge.py / __init__.py]` +- [ ] Implement the provider: domain → group via `ConfigEntry.sandbox` + (`config_entries.py:432`) for loaded entries, or the active + `SandboxFlowProxy` / `_assignment_for_new_flow` (`router.py:189-201`) for a + flow in progress; group → bridge via `SandboxData.bridges[group]` + (`sandbox/__init__.py:38-45`); issue the batched RPC; `{}` for + unowned/unreachable domains. +- [ ] **Built-in carve-out:** return nothing for `Integration.is_built_in` + domains — main reads its byte-identical disk files. One branch. +- [ ] Register the provider in `async_setup`; unregister on unload. +- [ ] Call `async_invalidate({domain})` on entry reload / sandbox restart at a + new integration-source `ref` (strings may have changed). + +--- + +## Phase A — catalog provider (picker discoverability + title) + +> The picker gap is discoverability, not just translation — `title` is a subset +> of the catalog metadata main lacks for a sandbox-only custom. + +### A1 · Core hook `[sandbox: sources.py (or sibling)]` +- [ ] Add a **separate** `async_register_sandbox_catalog_provider(hass, + provider)` — display-only, enumerable; do **not** overload the sha-pinned, + security-critical `IntegrationSourceDict` (`sources.py:38-56`). Entry shape: + `{ domain, name, config_flow, integration_type, iot_class, + single_config_entry, title_translations?: {lang: str} }`. + `title_translations` **optional** (HACS may not index the un-fetched + tarball's `translations/`); absent ⇒ degrade to `name`. + +### A2 · Merge into descriptions `[core: homeassistant/loader.py]` +- [ ] Append catalog entries to the custom half of + `async_get_integration_descriptions` (`:416-460`) so the picker lists them. +- [ ] Use catalog `name` / `title_translations` in the `title` fallback chain + when no on-disk `Integration` exists for a custom domain. + +### A3 · HACS contract `[docs]` +- [ ] Core exposes the hook; HACS fills it (HACS-agnostic posture, same as the + source resolver). Wrong/missing name is cosmetic — no strict validation + (unlike `ref`). Document in `sandbox/docs/`. + +--- + +## Core surface touched (high review attention) + +| File | Change | Phase | +|---|---|---| +| `homeassistant/helpers/translation.py` | provider hook + `_async_load` overlay + `async_invalidate` | B3 | +| `homeassistant/loader.py` | catalog merge into `async_get_integration_descriptions` + title fallback | A2 | +| `homeassistant/components/sandbox/protocol.py` + `sandbox/hass_client/.../protocol.py` + `proto/sandbox.proto` + `_proto/*pb2*` | `get_translations` message | B1 | +| `homeassistant/components/sandbox/{bridge,__init__,sources}.py` | translation + catalog provider impl & registration | B4, A1 | +| `sandbox/hass_client/.../` runtime | `get_translations` handler + title pre-fill | B2 | + +## Verification + +```bash +# core translation helper +uv run pytest tests/helpers/test_translation.py -q +# sandbox HA-core side +uv run pytest tests/components/sandbox/ --no-cov -q +# client side (separate uv env — no --no-cov) +uv run pytest sandbox/hass_client/ -q +# lint/format on changed files +uv run prek run --files +``` + +- [ ] `tests/components/sandbox/`: assert a sandboxed integration's + `frontend/get_translations` returns `config`/`entity`/`state`/`services`/ + `exceptions` strings — built-in *and* a fixture custom. +- [ ] `async_invalidate` drops stale strings after a simulated ref change. +- [ ] `hass_client`: `get_translations` returns title-prefilled raw dict for + built-in + custom fixture. +- [ ] Catalog: a registered provider makes a sandbox-only custom appear in + `async_get_integration_descriptions` + supplies the picker name. +- [ ] `test_translation.py`: provider overlay + degrade-to-empty on dead + channel; non-sandboxed integrations unaffected. + +## Risks & self-check + +- **Cache lock × RPC latency.** The overlay runs under the cache lock; + per-group batching + degrade-to-empty are load-bearing, not optional. +- **Pre-entry flow translations** for a brand-new custom (no entry, no code on + main): group must come from the live `SandboxFlowProxy`, not `entry.sandbox`. +- **Invalidation correctness** on sha change — get the `loaded` + nested + `cache` eviction keys right, or stale strings persist. +- **Scope creep on Phase A** — it bleeds into the broader "sandbox-only custom + discovery" feature; keep the catalog strictly display metadata. + +*Self-check:* +1. *What could make this wrong?* Translation loads that happen before the + sandbox is up (boot-time) — guard the provider to claim only running/owned + domains so they fall through to disk/empty, not block. +2. *Simplest thing that works?* Phase B alone is shippable and delivers the + bulk of the UX; Phase A can land separately. +3. *What did research explicitly warn?* `IntegrationNotFound` is a dict + *value* not a raise; there is *no* cache-eviction API; HACS may lack + indexed translations — `title_translations` must be optional. + +## Phasing + +1. **Phase B** (B1→B4) — live pull-RPC. Self-contained, biggest win. +2. **Phase A** (A1→A3) — catalog provider; pairs with the broader + stateless-custom-discovery work; can ship independently. + +## Status + +Not started. Next: `/phx:work sandbox/plans/plan-translation-forwarding.md` +(or implement Phase B directly). diff --git a/sandbox/plans/research/translation-forwarding-core-seam.md b/sandbox/plans/research/translation-forwarding-core-seam.md new file mode 100644 index 000000000000..691b0eef3fee --- /dev/null +++ b/sandbox/plans/research/translation-forwarding-core-seam.md @@ -0,0 +1,399 @@ +# Core seam: redirecting a sandboxed integration's translation load to RPC + +All citations are against the working tree at `/home/paulus/dev/hass/core`. +THE file is `homeassistant/helpers/translation.py`. + +--- + +## 1. The redirect seam + +### 1a. What `async_get_integrations` returns for an unknown (custom, sandboxed) domain + +`_TranslationCache._async_load` (`translation.py:208-253`) calls +`async_get_integrations(self.hass, components)` (`:220`) and then iterates, +**skipping any domain whose value is an `Exception`** (`:221-227`): + +```python +ints_or_excs = await async_get_integrations(self.hass, components) +for domain, int_or_exc in ints_or_excs.items(): + if isinstance(int_or_exc, Exception): + _LOGGER.warning("Failed to load integration for translation: %s", int_or_exc) + continue + integrations[domain] = int_or_exc +``` + +`async_get_integrations` (`loader.py:1375-1449`) does **not** raise for an +unknown domain. It resolves custom components first (`:1413-1417`), then +`_resolve_integrations_from_root` (`:1426-1428`); for anything still +unresolved it returns an `IntegrationNotFound` *as the dict value* +(`loader.py:1441-1447`): + +```python +del cache[domain] +exc = IntegrationNotFound(domain) +results[domain] = exc +future.set_result(exc) # not set_exception — value, not raise +``` + +So for a custom sandboxed domain with no code on disk, main gets +`{domain: IntegrationNotFound(domain)}`. The cache deliberately does **not** +memoise the miss (`:1434-1441`), so a later disk appearance is re-resolvable. + +### 1b. What `_async_get_component_strings` does with a missing Integration + +`_async_get_component_strings` (`:86-128`) only ever reads `integrations.get(domain)` +— it never indexes, so a domain absent from the `integrations` dict simply +produces **no file to load** and **no title** (see §3). The two relevant guards: + +- File collection (`:100-107`): a domain is only added to `files_to_load` when + `(integration := integrations.get(domain)) and integration.has_translations`. + A missing integration ⇒ skipped, no `KeyError`. +- Title injection (`:118-124`): `integration := integrations.get(domain)` is + falsy ⇒ no `title` is set. + +Net effect for a custom sandboxed domain on main today: it falls through +silently to `loaded_translations.setdefault(domain, {})` (`:120`) — an **empty +dict**, no warning beyond the `_async_load` one. That empty dict flows into +`translations_by_language[lang][domain] = {}`, `_build_category_cache` finds no +categories for it, and the domain is marked loaded with zero strings. **The +integration's frontend strings are simply missing** — this is the gap to fill. + +### 1c. Recommended seam — split sandboxed domains out *before* `_async_load` + +**Thesis:** redirect inside `_async_load`, by partitioning `components` into +`local` vs `remote` and merging an RPC result into +`translation_by_language_strings` *before* `_build_category_cache` runs. This is +the single cleanest seam because: + +- `_async_load` already owns the `language → languages` fallback expansion + (`:217`), the per-language cache build (`:234-251`), and the `loaded` set + bookkeeping (`:244-253`). A sandboxed domain must go through the *same* + `_build_category_cache` / `loaded` machinery so `get_cached`, the + English-fallback overlay, and `async_is_loaded` keep working uniformly. +- `_async_get_component_strings` is the right *shape* producer to mirror (its + output is the merge target), but it is `Integration`-driven and disk-path + only. Putting the branch there would force passing the sandbox lookup down + one more layer for no benefit. Keep that function untouched (disk-only). + +Concretely, the branch sits right after `async_get_integrations` returns, in +`_async_load` (`:219-231`): + +```python +integrations: dict[str, Integration] = {} +ints_or_excs = await async_get_integrations(self.hass, components) +for domain, int_or_exc in ints_or_excs.items(): + if isinstance(int_or_exc, Exception): + _LOGGER.warning("Failed to load integration for translation: %s", int_or_exc) + continue + integrations[domain] = int_or_exc + +translation_by_language_strings = await _async_get_component_strings( + self.hass, languages, components, integrations +) +# NEW: overlay remote (sandboxed) component strings fetched over RPC. +# For each sandboxed domain in `components`, replace its (empty) entry in +# translation_by_language_strings[lang][domain] with the RPC payload. +``` + +**Why not split before `_async_load` (in `async_load`, `:160-176`)?** That +would bypass the lock-coalescing (`:166-176`) and the +`components - loaded` diff that `async_load` computes, and would need to +duplicate the English-fallback `languages` expansion. The diff/lock belong to +the *whole* component set; only the *strings source* differs per-domain. So +split inside `_async_load`, not above it. + +**Main risk / antithesis:** `_async_load` runs under the cache lock and is +`await`-heavy already; adding a synchronous RPC round-trip per sandboxed domain +serialises translation loads behind the channel. Frontend translation fetches +are frequent and latency-sensitive (`async_get_translations` is called per +category). Mitigations: batch all remote domains of one group into a single RPC +(the channel already multiplexes — `channel.call`, `channel.py:392`), and have +the RPC payload pre-shaped so no extra disk/CPU work happens under the lock. A +channel-down / `IntegrationNotFound`-equivalent must degrade to "empty strings, +domain marked loaded" (today's behavior) rather than raising, so a dead sandbox +never wedges the frontend translation endpoint. + +### 1d. Required RPC return shape + +The merge target is `translation_by_language_strings`, the return of +`_async_get_component_strings` (`:91-128`). Shape is +**`{language: {domain: }}`**: + +```python +translations_by_language: dict[str, dict[str, Any]] = { + "en": { + "light": { # == contents of light/translations/en.json + "title": "...", # injected from integration.name if absent + "config": {...}, + "entity": {...}, + "exceptions": {...}, + ... + }, + }, + "": { "light": {...} }, +} +``` + +`_build_category_cache` (`:300-330`) then walks each component's top-level keys +as *categories* (`config`, `entity`, `exceptions`, `entity_component`, …), +calls `build_resources` (`:71-83`) per category, and `recursive_flatten`s into +`component...<...>` keys. So the RPC must return the **raw, +un-flattened nested `strings.json` structure** for each requested language — +exactly what a `translations/.json` file on disk contains, with `title` +already filled in (the sandbox *has* the `Integration` and can inject it; see +§3). The languages requested are `["en"]` or `["en", ]` (`:217`) — the +RPC should accept that language list and return both, because `_async_load` +loads English as the fallback overlay (`:233-251`). + +--- + +## 2. Cache structure & invalidation + +### Keying + +`_TranslationsCacheData` (`:131-141`) holds two dicts shared across cache +instances: + +- `loaded: dict[str, set[str]]` — per **language** → set of component domains + already loaded. Drives `async_is_loaded` (`:156-158`, + `components.issubset(...)`) and the `components - loaded` diff (`:167`, `:175`). +- `cache: dict[str, dict[str, dict[str, dict[str, str]]]]` — nested + **`language → category → component → {flat_key: value}`** (built in + `_build_category_cache`, `:309-330`; read in `get_cached`, `:196-206`). + +Flat keys are `component...` (`:324`, `:327-328`). + +### "Never unloaded" + +The comment is in `async_load` (`:168-171`): + +```python +# Translations are never unloaded so if there are no components to load +# we can skip the lock which reduces contention ... +``` + +There is **no eviction API anywhere** in `translation.py`. `loaded` only ever +grows (`:166` `setdefault`, `:251`/`:253` `update`); `cache` only ever +`setdefault`/`update` (`:309`, `:318`, `:321`, `:330`). Nothing deletes from +either. (`async_setup`, `:382-412`, only *adds* on language change.) + +### What re-fetch-at-new-sha needs + +For a custom integration re-fetched at a new commit sha whose `strings.json` +changed, stale entries to drop are, for that one `domain` across **all +languages** and **all categories**: + +- every `loaded[lang]` set containing `domain` → `discard(domain)` +- every `cache[lang][category]` dict that has a `domain` key → `del` + +There is **no existing API** to do this. Minimal addition: a single callback +method on `_TranslationCache`, e.g. + +```python +@callback +def async_invalidate(self, components: set[str]) -> None: + """Drop cached + loaded state for the given components (all languages).""" + for loaded in self.cache_data.loaded.values(): + loaded -= components + for by_category in self.cache_data.cache.values(): + for category_cache in by_category.values(): + for component in components & category_cache.keys(): + del category_cache[component] +``` + +plus a thin module-level `async_invalidate_translations(hass, components)` +wrapper that goes through `_async_get_translations_cache(hass)` (`:376-379`), +mirroring the existing `async_load_integrations` wrapper (`:415-419`). The +sandbox would call it on each re-fetch / entry reload of a custom domain, and +the next `async_fetch` re-runs `_async_load` (now via the RPC seam from §1). +Note: invalidation must hold or respect `self.lock` (`:153`, `:172`) if it can +race a concurrent `_async_load`; the simplest correct form is to make it +`async` and take the lock, or to document it as caller-serialised against loads. + +--- + +## 3. `title` fallback needs an `Integration` — confirmed + +`:118-124`: + +```python +for domain in components: + component_translations = loaded_translations.setdefault(domain, {}) + if "title" not in component_translations and ( + integration := integrations.get(domain) + ): + component_translations["title"] = integration.name +``` + +The fallback reads `integration.name`, which requires the `Integration` +object's manifest. For a custom sandboxed domain main has **no `Integration`** +(§1a returns `IntegrationNotFound`), so this branch **cannot run on main** for +such a domain — `integrations.get(domain)` is `None` and `title` stays unset. + +**Implication:** title injection must happen **on the sandbox side**, which +*does* hold the loaded `Integration` (it fetched + imported the code). The RPC +payload should therefore return component strings with `title` already filled +(the sandbox runs the equivalent of `_async_get_component_strings`' title +fallback before serialising). If the sandbox omits it, the integration's +display name is blank on main's frontend. Built-in sandboxed domains are a +non-issue — their `Integration` resolves on main from the bundled package, so +they could even stay on the local disk path; only **custom** domains truly need +the remote title. + +--- + +## 4. Existing core hooks to mirror + +### `async_register_sandbox_source_resolver` (the convention to copy) + +`homeassistant/components/sandbox/sources.py:67-87` — a `@callback` that appends +a resolver to a `HassKey`-stored list and returns an unregister `@callback`: + +```python +SandboxSourceResolver = Callable[[str], IntegrationSourceDict | None] # :56 +DATA_SOURCE_RESOLVERS: HassKey[list[SandboxSourceResolver]] = HassKey( + "sandbox_source_resolvers") # :58-60 + +@callback +def async_register_sandbox_source_resolver(hass, resolver): # :67-87 + resolvers = hass.data.setdefault(DATA_SOURCE_RESOLVERS, []) + resolvers.append(resolver) + @callback + def _unregister() -> None: + resolvers.remove(resolver) + return _unregister +``` + +The consumer (`async_resolve_integration_source`, `:90-114`) short-circuits +built-ins via `Integration.is_built_in` and otherwise consults resolvers in +order, **raising** `SandboxSourceError` if none matches. A "remote translation +provider" hook can follow this exact convention: a `HassKey`-stored provider +keyed by group (or a single resolver `domain → group | None`), registered by +the sandbox integration at setup, consulted from the new `translation.py` +branch. Because `translation.py` is in `homeassistant/helpers/` (core), it must +**not** import the `sandbox` component — the hook lives there as a registration +seam exactly like the source resolver, and the sandbox integration registers +into it. + +### The `router` attribute on `ConfigEntries` + +`config_entries.py:2160-2161` — a single nullable hook attribute: + +```python +# Optional hook for diverting flows and entry setup (used by sandbox). +self.router: ConfigEntryRouter | None = None +``` + +`ConfigEntryRouter` is a `Protocol` (`:2121-2142`) with three `async def` +methods (`async_create_flow`, `async_setup_entry`, `async_unload_entry`), each +returning `None` to fall through. `SandboxFlowRouter` (`router.py:46`) +structurally implements it; it's assigned in `sandbox/__init__.py:96` +(`data.router = router`) and presumably set onto `hass.config_entries.router` +at setup. A translation hook could mirror either style — a list of providers +(source-resolver style) or a single Protocol object (router style). The +**list-of-resolvers** style is the better fit here since translation has no +fall-through chain semantics beyond "which group owns this domain". + +### Where the singleton translation cache lives on hass + +`translation.py:376-379` — stored via the `singleton` decorator under the key +`TRANSLATION_FLATTEN_CACHE = "translation_flatten_cache"` (`:29`): + +```python +@singleton.singleton(TRANSLATION_FLATTEN_CACHE) +def _async_get_translations_cache(hass: HomeAssistant) -> _TranslationCache: + return _TranslationCache(hass) +``` + +Every public entry point (`async_get_translations` `:353`, +`async_get_cached_translations` `:371`, `async_load_integrations` `:417`, +`async_translations_loaded` `:425`) goes through `_async_get_translations_cache(hass)`. +A new `async_invalidate_translations(hass, ...)` (§2) and any provider lookup +hang off the same singleton, so there is exactly one cache + one hook registry +per `hass`. + +--- + +## 5. Liveness guard — knowing a domain is sandboxed and its group, at load time + +Two signals already exist; both are reachable from `hass` without importing the +sandbox internals if a small accessor hook is added (per §4). + +### Loaded-entry case (authoritative) + +`ConfigEntry.sandbox: str | None` (`config_entries.py:432`, declared +`:566-569`) is the group name, set at flow completion (`:1813`, +`async_finish_flow` reads `ConfigFlowResult["sandbox"]`) and read back from +storage (`:2356`). It persists in `as_dict` only when set (`:1221-1224`). So at +translation-load time, the canonical "is `domain` sandboxed and where" is: + +```python +for entry in hass.config_entries.async_entries(domain): + if entry.sandbox is not None: + group = entry.sandbox # this domain is sandboxed into `group` + break +``` + +This is exactly the pattern the router already uses for new flows — +`SandboxFlowRouter._assignment_for_new_flow` (`router.py:189-200`) loops +`async_entries(handler_key)` and returns `existing.sandbox` if set, else falls +back to `classify(integration)`. Translation loading should reuse that resolve +order (entry-wins, classifier-fallback) so it agrees with where the entry +actually ran. + +### Live-channel / bridge case (for issuing the RPC) + +Group → live transport lives in `SandboxData` (`sandbox/__init__.py:38-45`), +stored under `DATA_SANDBOX` (`const.py:12`, +`HassKey[SandboxData](DOMAIN)`): + +- `data.bridges: dict[str, SandboxBridge]` (`:45`) — the per-group bridge owning + the channel; populated in `_on_channel_ready` (`:53-57`). +- `data.channels: dict[str, Channel]` (`:44`) — the raw channel. +- `data.manager.get(group)` (`manager.py:582-584`) returns the + `SandboxProcess` or `None`; `sandbox.channel` may be `None` if down + (router checks this at `router.py:104-105`, `:169`). + +So the RPC issuer resolves `domain → group` (via `entry.sandbox`), then +`group → channel` (via `data.bridges[group].channel` or +`manager.get(group).channel`), then `channel.call(MSG_..., payload)` +(`channel.py:392`). If the group has no running sandbox / no live channel, the +guard degrades to empty strings (the §1c risk note) — never raise into the +frontend translation path. + +### Pre-entry (flow-in-progress) case + +Before any `ConfigEntry` exists (the add-integration flow is mid-render and +wants config-flow translations — `async_get_translations(..., config_flow=True)`, +`:346-347`), `entry.sandbox` is unavailable. Fall back to +`classify(await async_get_integration(hass, domain))` +(`classifier.py:58-76`) — the same fallback the flow router uses +(`router.py:199-200`). For a **custom** domain, `classify` returns +`Sandbox("custom")` (`classifier.py:73-74`) **without importing** the +integration (it uses manifest + `platforms_exists`). Caveat: at flow-start the +custom integration's code may not yet be on disk on main at all, so even +`async_get_integration` can raise `IntegrationNotFound` — in that pre-entry, +no-code state the loader genuinely has nothing, and the config-flow translation +RPC must target whichever group the flow proxy spun up (the flow already routes +through `SandboxFlowProxy`, `router.py:79-83`, which knows its +`sandbox_group`). That is the trickiest corner: config-flow translations for a +brand-new custom integration need the *flow's* group, not an entry's group. + +--- + +## Summary of the recommended change set + +1. **Seam:** branch in `_TranslationCache._async_load` (`translation.py:219-231`), + after `async_get_integrations`, overlaying remote-fetched + `{lang: {domain: raw_strings}}` onto `translation_by_language_strings`. +2. **Hook:** a source-resolver-style registry (mirror `sources.py:58-87`) so + core stays sandbox-agnostic; the sandbox integration registers a + `domain → group` resolver + an RPC fetcher. +3. **RPC shape:** raw un-flattened `strings.json` nesting per requested + language, `title` pre-injected sandbox-side (§3). +4. **Invalidation:** new `async_invalidate(components)` on `_TranslationCache` + (`loaded` discard + `cache[*][*]` del) + module wrapper, called on re-fetch + at a new sha (§2). +5. **Liveness:** resolve group via `entry.sandbox` (entry-wins) → + `classify` fallback (pre-entry); resolve channel via + `DATA_SANDBOX.bridges[group]`; degrade to empty on a down channel. diff --git a/sandbox/plans/research/translation-forwarding-discovery-and-index.md b/sandbox/plans/research/translation-forwarding-discovery-and-index.md new file mode 100644 index 000000000000..6559ded4030c --- /dev/null +++ b/sandbox/plans/research/translation-forwarding-discovery-and-index.md @@ -0,0 +1,316 @@ +# Sandbox translation forwarding — discovery & index research + +Goal: figure out how the *add-integration picker* can show a `title` (display +name) for a **custom/HACS integration whose code lives only in a sandbox**, and +how the not-yet-running config-flow entry can be rendered — **without spawning a +sandbox**. Two seams: + +- **Seam A (cold/picker path):** picker list + `title` strings, no flow running, + no entry loaded, possibly no code on main's disk. +- **Seam B (live pull path):** a flow *is* running or an entry *is* loaded — + main already knows the sandbox group for the domain. + +All `file:line` references are against repo root `/home/paulus/dev/hass/core` +(backend) and `/home/paulus/dev/hass/frontend` (frontend, read-only). + +--- + +## 1. Where the picker's integration list comes from + +**The picker does NOT call `async_get_config_flows`.** That function +(`homeassistant/loader.py:346-367`) is the *flow allow-list* (used by config-flow +init to decide whether a domain may start a flow). The picker UI is fed by a +different function. + +### The picker's actual source: `integration/descriptions` + +- Frontend: `dialog-add-integration.ts:590-644` `_load()` calls + `getIntegrationDescriptions(this.hass)` + (`frontend/src/data/integrations.ts:42-47`), which is the WS command + `integration/descriptions`. +- Backend handler: `homeassistant/components/websocket_api/commands.py:1303-1309` + `handle_integration_descriptions` → `async_get_integration_descriptions`. +- Builder: `homeassistant/loader.py:416-460` + `async_get_integration_descriptions`: + - **core** half: read verbatim from the generated + `homeassistant/generated/integrations.json` + (`loader.py:420-424`). This is a build-time artifact (brands index + + per-integration metadata + a `translated_name` list). Never includes + sandbox-only customs. + - **custom** half: built live from `async_get_custom_components(hass)` + (`loader.py:425-458`). That function (`loader.py:325-343` → + `_get_custom_components`) **scans `/custom_components` on main's + disk**. Each custom integration contributes a metadata dict: + `config_flow`, `integration_type`, `iot_class`, `name`, + `single_config_entry`, `overwrites_built_in` (`loader.py:448-458`). + +### Consequence for a sandbox-only custom integration + +A custom integration whose code lives **only in the sandbox** (fetched on +`entry_setup` per `sandbox/sources.py`; not present under main's +`/custom_components`) is invisible to **both** lists: + +- It is **not** in `async_get_config_flows` — the custom branch + (`loader.py:360-365`) iterates `async_get_custom_components(...).values()`, + which only sees on-disk customs. +- It is **not** in `async_get_integration_descriptions` `custom.integration` — + same on-disk scan (`loader.py:425-431`). +- It is **not** in the generated `integrations.json` (that ships only built-ins). + +So **with stateless sandboxes the picker simply has no row for the integration**, +and `frontend/get_translations` has nothing to load a title from. This is the +core gap the feature must close. + +> Note on the today-state: the stateless-sandbox source model +> (`sandbox/CLAUDE.md` "Stateless sandboxes — integration source") presumes the +> code is fetched at setup time. But the *current* HACS install still drops the +> code under `/custom_components`, so today the picker works by accident +> (the on-disk scan finds it). The research target is the future where the code +> is NOT on main's disk — then the picker breaks unless we feed it from the +> resolver/index. + +--- + +## 2. The `title` strings for the picker + +Two-layer title resolution, both rooted on main's disk: + +1. **Display name in the list** — `dialog-add-integration.ts:266 / 282 / 299 / + 324` use `integration.name || domainToName(localize, domain)`. `name` is the + manifest name carried in the descriptions payload (`loader.py:452`, + `"name": integration.name`). For a sandbox-only custom there is **no manifest + on disk → no `name`**, and `domainToName` only falls back to a prettified + domain string. So the row, if it existed at all, would show e.g. + `My Custom Thing` only if `integration.name` were supplied. + +2. **`title` translation category** — loaded lazily via + `dialog-add-integration.ts:639-643` + `loadBackendTranslation("title", descriptions.core.translated_name, true)` + (and `:606` for in-progress discovered handlers). Backend: + `frontend/get_translations` (`frontend/__init__.py:987-1009`) → + `async_get_translations` → `_async_get_component_strings` + (`translation.py:86-127`). Title strings come from: + - the integration's on-disk `translations/.json` + (`translation.py:100-106`, `integration.file_path / "translations" / + .json`), **OR** + - fallback to `integration.name` (manifest name) when the `title` key is + missing (`translation.py:118-124`): + ```python + if "title" not in component_translations and ( + integration := integrations.get(domain) + ): + component_translations["title"] = integration.name + ``` + But `integrations.get(domain)` is the loaded `Integration` object — for a + sandbox-only custom, `async_get_integrations` cannot load it (no code on + disk), so even the fallback has nothing. + +**Confirmed gap:** + +| Integration kind | manifest `name` on main | `translations/*.json` on main | +|---|---|---| +| built-in, sandboxed | ✅ bundled | ✅ bundled | +| custom, code on disk (today's HACS) | ✅ | ✅ (HACS ships them) | +| **custom, sandbox-only (target)** | ❌ | ❌ | + +**Minimal data the picker actually needs** (the whole reason we don't need a +sandbox): per custom domain, a tiny static descriptor — +- `name` (display name; what feeds both the list label *and* the `title` + translation fallback), +- the picker metadata it already wants: `config_flow: true`, `integration_type` + (so it lands in the right bucket — `integration` vs `helper`), `iot_class` + (cloud badge), optionally `single_config_entry`. + +That is exactly the subset `async_get_integration_descriptions` writes today +(`loader.py:448-458`). No `config`/`selector` schema, no description body — just +enough to render a row and a name. (See §5: the picker never loads `config`.) + +--- + +## 3. Extending the resolver / index to carry the picker `title` + +### What exists + +`async_register_sandbox_source_resolver(hass, resolver)` +(`sandbox/sources.py:67-87`). A resolver is `Callable[[str], IntegrationSourceDict +| None]` (`sources.py:56`) — domain in, git source out. `IntegrationSourceDict` +(`sources.py:38-51`) is purely *code-location* data: `kind, url, ref, tag, +domain, subdir`. There is **no name/title/metadata** anywhere in the contract. +Resolvers are consulted **lazily, per-domain, only at `entry_setup`** via +`async_resolve_integration_source` (`sources.py:90-114`) — i.e. only once you +already know the domain you want. That shape is wrong for the picker, which needs +to *enumerate* unknown domains up front. + +### The contract decomposition: core EXPOSES, HACS FILLS + +Mirror the existing resolver philosophy (core HACS-agnostic; HACS registers). +Two cleanly-separable concerns: + +- **Code location** (existing): `domain → {kind:git,url,ref,...}`, lazy, + per-domain, security-critical (`ref` must be a sha). Keep as-is. +- **Picker metadata** (new): the full *set* of custom domains plus their + display metadata, eager, enumerable, NOT security-critical (it's just a + display string). This wants a **listing** hook, not a per-domain resolver. + +**Cleanest shape — a parallel "catalog provider" hook**, registered the same way: + +```python +class SandboxIntegrationDescriptor(TypedDict, total=False): + domain: str + name: str # display name → list label + title fallback + integration_type: str # "integration" | "helper" | ... + config_flow: bool + iot_class: str | None + single_config_entry: bool + # title translations are optional; see thesis/antithesis below + title_translations: dict[str, str] | None # {lang: title} + +SandboxCatalogProvider = Callable[[], list[SandboxIntegrationDescriptor]] + +@callback +def async_register_sandbox_catalog_provider(hass, provider) -> Callable[[], None]: + ... +``` + +Then `async_get_integration_descriptions` (`loader.py:416-460`) — or a thin +sandbox-aware wrapper — merges these descriptors into the `custom.integration` / +`custom.helper` buckets exactly where the on-disk scan does today +(`loader.py:431-458`), de-duping by domain against on-disk customs. And +`frontend/get_translations` for category `title` gains a sandbox source: when a +domain isn't loadable on disk, pull `name` (and/or `title_translations[lang]`) +from the catalog instead of `integration.name` (the fallback at +`translation.py:118-124`). + +Why a **separate** provider and not extending `IntegrationSourceDict`: + +- The source resolver is *lazy per-domain* and *security-critical* (sha pinning, + `sources.py:19-22,130-135`). The catalog is *eager enumerable* and *display- + only*. Fusing them would force the security-critical path to also be a full + listing API and would drag display strings through the sha-validation code. +- Keeps the wire/`entry_setup` proto (`pb.IntegrationSource`) untouched — title + strings never need to cross to the sandbox; they're a main-side display + concern. + +### Thesis — extend the index to carry `title` + +HACS already maintains a full catalog of installed (and installable) custom +integrations including their repo, version/sha (it supplies `ref` today), and +the `manifest.json` `name`. Surfacing `name` (+ optionally cached +`translations/en.json` `title`) per domain is nearly free for HACS and is the +single source of truth. Core stays agnostic (just a registry + a merge point); +the picker renders custom rows and a sensible name with zero sandbox spin-up. +Matches the established "core exposes hook, HACS fills" precedent exactly +(`sources.py` module docstring lines 10-14). + +### Antithesis — what if HACS doesn't have translations indexed? + +- HACS reliably knows the **manifest `name`** (it parses `manifest.json` to + validate installs) and the sha. It does **not** necessarily have the + integration's `translations/*.json` indexed — those live inside the repo + tarball, which under the stateless model is only fetched at `entry_setup`. So + `title_translations` may be empty for most/all domains. +- Mitigation built into the fallback chain we already rely on: if + `title_translations[lang]` is absent, the picker degrades to + `integration.name` (manifest name) — which is exactly the existing + `translation.py:118-124` behavior and the `integration.name || + domainToName(...)` chain in `dialog-add-integration.ts:266`. So + `title_translations` should be **optional**; the load-bearing field is `name`. + A localized title is a nice-to-have, not a requirement, for the picker. +- Risk: a custom integration that *only* defines its display name in + `translations/.json` `title` (no good manifest `name`) would show a + prettified domain. Acceptable for v1; HACS could later cache `en.json` title + at install time if it wants better names. +- Trust boundary: unlike `ref` (sha-pinned, security-critical), a wrong/missing + `name` is cosmetic — no need for the strict validation the source path has. + +**Recommendation:** ship the catalog provider carrying `name` + the small +picker-metadata subset; make `title_translations` optional. This unblocks the +picker with the data HACS definitely has, and the title localization can improve +later without a contract change. + +--- + +## 4. Domain → sandbox-group resolution for the LIVE pull path (seam B) + +When a flow IS running or an entry IS loaded, main already maps domain → group; +no catalog needed. The existing lookups: + +- **`ConfigEntry.sandbox: str | None`** — the routing tag stored on the entry. + Declared `config_entries.py:432`; `__init__` kwarg `:448`; written to + `as_dict` `:1223-1224`; read back from storage `:1813` / `:2356`; plumbed via + `async_finish_flow`/`async_update_entry` (`:2576`, `:2597`, `:2616`). Its value + is the **group name** (e.g. `"custom"` / `"built_in"`), not a per-domain id. +- **New-flow assignment** — `router.py:189-201` `_assignment_for_new_flow`: + first an *existing entry's* `sandbox` wins + (`async_entries(handler_key)` → `existing.sandbox`, `:196-198`); otherwise + `classify(integration)` decides (`classifier.py:58-76`: system/ALWAYS_MAIN/ + incompatible-platform → MAIN; `not is_built_in → GROUP_CUSTOM`; else + `GROUP_BUILT_IN`). The classifier is *also* a domain→group function and works + off the `Integration` object, **but it needs the integration loadable** — so + for a sandbox-only custom not on disk, `classify` would fail to load it. For + seam B that's fine because an existing entry's `sandbox` field already pins the + group (the entry was created when the code was present / fetched). +- **Group → live bridge / channel** — `SandboxData` + (`__init__.py:38-45`): `bridges: dict[str, SandboxBridge]` and + `channels: dict[str, Channel]`, keyed by **group name**. Populated on channel + ready (`__init__.py:53-57`). The router uses `self._data.bridges.get(group)` + (`router.py:183-184`) and `self._manager.get(group)` / `ensure_started(group)` + (`router.py:91`, `:168`). `entry_setup`/`unload` resolve group via + `entry.sandbox` (`router.py:87`, `:165`). +- **Bridge-level domain tracking** — `bridge.py` keys its owned + `EntityPlatform`s by `(entry_id, domain)` (`bridge.py:179-183`) and tracks + mirrored `(domain, service)` pairs (`bridge.py:183`). This is reverse-mapping + (group→domains it owns), used for entity/service mirroring, not for the + picker's forward domain→group lookup. + +**Net:** seam B's forward lookup already exists end-to-end: +`entry.sandbox` (group) → `SandboxData.bridges[group]` / `manager.get(group)`. +For a *running flow* the group is carried on `SandboxFlowProxy(sandbox_group=...)` +created in `async_create_flow` (`router.py:79-83`). So **a translation pull for a +live/in-progress sandboxed domain can be routed to the right bridge using the +existing group keying** — no new index needed for seam B; the catalog (§3) is +only for the cold picker (seam A). + +--- + +## 5. Does the picker ever load category `config` in bulk? + +**No.** The picker loads **only `title`**: + +- `dialog-add-integration.ts:606` — + `loadBackendTranslation("title", discoveredHandlers, true)` (discovered flow + handlers only). +- `dialog-add-integration.ts:639-643` — + `loadBackendTranslation("title", descriptions.core.translated_name, true)`. + +No `config` / `selector` / fragment load anywhere in `dialog-add-integration.ts`. + +`config`/`selector` are loaded **only once a specific flow starts**, in +`show-dialog-config-flow.ts`: + +- `:29-33` (initial step): + `loadFragmentTranslation("config")`, `loadBackendTranslation("config", handler)`, + `loadBackendTranslation("selector", handler)`, `loadBackendTranslation("title", + handler)`. +- `:40-46` (subsequent step): same set for `step.handler`. + +So the heavy `config`/`selector` payload is per-flow and lazy. **For the picker we +only ever need `title` (which collapses to a display `name`)** — confirming §2's +"minimal data" conclusion: the catalog provider only has to carry a name, not +flow schema. + +--- + +## Summary of the contract to build + +- **Seam A (picker / cold):** new `async_register_sandbox_catalog_provider` + (parallel to the source resolver). HACS fills a list of + `{domain, name, integration_type, config_flow, iot_class, + single_config_entry, title_translations?}`. Core merges it into + `async_get_integration_descriptions` custom buckets (`loader.py:431-458`) and + into the `title` fallback in `translation.py:118-124`. `name` is the + load-bearing field; `title_translations` optional. +- **Seam B (live):** nothing new — reuse `entry.sandbox` → `SandboxData.bridges` + / `manager.get(group)` (`router.py`, `__init__.py:38-45`). +- Keep the security-critical source resolver (`sources.py`) untouched and + separate from the display-only catalog. diff --git a/sandbox/plans/scratchpad-translation-forwarding.md b/sandbox/plans/scratchpad-translation-forwarding.md new file mode 100644 index 000000000000..34f159dff833 --- /dev/null +++ b/sandbox/plans/scratchpad-translation-forwarding.md @@ -0,0 +1,43 @@ +# Scratchpad — translation forwarding + +Decisions, rejected paths, and things to remember while implementing +`plan-translation-forwarding.md`. + +## Decisions (locked) +- **Both seams** — live pull-RPC (B) + catalog provider (A). +- **Pull/RPC**, not push — matches the lazy per-language `_TranslationCache`; + only fetch what the frontend asks for. +- **Built-in reads local disk** in the live path — files are byte-identical on + main; RPC reserved for customs. One `is_built_in` branch in the provider. +- **Whole-strings-dict** RPC granularity — main slices via `build_resources`, + matching how it reads the file today. +- **Separate** `async_register_sandbox_catalog_provider` for the picker — do + NOT overload the sha-pinned `IntegrationSourceDict` source resolver. + +## Rejected / not chosen +- Push-at-setup / hybrid transport — needs a language-set decision + still + needs pull on language switch. Dropped. +- Pure unified pull (RPC for built-in too) — redundant data + needless sandbox + spawn for the picker. Replaced by the built-in disk carve-out. +- Spawning every sandbox to render the bulk picker — rejected; picker only + needs `title`, served statically (disk for built-in, catalog for custom). + +## Landmines (from research, file:line) +- `async_get_integrations` returns `IntegrationNotFound` as the dict **value**, + never raises, never caches the miss (`loader.py:1441-1447`). +- `_async_load` skips Exception-valued domains (`translation.py:221-227`); + missing domain ⇒ silent `{}`. +- No cache-eviction API — `loaded`/`cache` only grow (`translation.py:168-171`). + Must add `async_invalidate`. +- `title`→`integration.name` fallback (`translation.py:118-124`) needs an + `Integration` object — impossible on main for customs ⇒ pre-fill sandbox-side. +- Picker uses `integration/descriptions` (disk scan, `loader.py:325-343, + 416-460`), NOT `async_get_config_flows`. Sandbox-only custom is in none. +- Picker loads only `title` (`dialog-add-integration.ts:606,639-643`); + `config`/`selector` load per-flow (`show-dialog-config-flow.ts:29-46`). + +## Open questions to resolve during work +- Exact RPC batching unit: per group is the intent — confirm the provider + groups `components` by `ConfigEntry.sandbox` before issuing RPCs. +- Boot-time translation loads before any sandbox is up — verify the provider + returns `{}` (fall through to disk/empty), never blocks.