Commit Graph
112855 Commits
Author SHA1 Message Date
Paulus Schoutsen c11f85b048 sandbox: translation returned-domains gate (Phase 4)
The provider overlay (_TranslationCache._async_overlay_sandbox_strings)
splices every domain a sandbox returns, narrowed only by the broad
requested components set — so a compromised sandbox could return strings
for a co-requested victim domain (hue, http) and poison its frontend
strings. async_get_translations now keeps only the requested ∩ returned
intersection (domains & strings.keys()), the set this group was actually
asked to resolve, before handing strings to the cache.
2026-07-07 15:12:24 -04:00
Paulus Schoutsen 53453a69ef sandbox: entry/group ownership on register_entity (Phase 3)
_handle_register_entity now requires entry.sandbox == self.group, not
just that the entry_id resolves: a compromised sandbox may only register
entities for entries main routed to *this* group. Without this it could
attach entities and pre-create devices against a victim integration's
config entry.

Also reject a device pre-create that would merge (via shared
identifiers/connections) into a device already owned by a config entry
outside this group — _reject_foreign_device_merge — closing the
device-registry hijack vector. entry.sandbox is set by main at flow
completion, never by the sandbox.

entry_setup/entry_unload are main-initiated (main supplies the entry_id),
and the store server is scoped one-dir-per-channel by construction, so no
further entry_id trust points need gating.

Test entry fixtures across bridge/entity_query/domain_proxies/
crash_recovery/proto_transport updated to tag their entries sandbox="built-in".
2026-07-07 15:12:24 -04:00
Paulus Schoutsen 2cc2ccf2cc sandbox: register_service ownership check (Phase 2)
_handle_register_service now requires the service domain to be one this
group owns (same main-side _owned_domains() derivation as the fire_event
gate), so a compromised sandbox can no longer squat
persistent_notification.* or any unclaimed domain.service. Unowned
domains are rejected with a HomeAssistantError → remote-error frame; the
existing refuse-to-clobber-an-existing-handler check is kept.

Existing register_service tests updated to own their mock domains via a
MockConfigEntry(sandbox="built-in").
2026-07-07 15:12:24 -04:00
Paulus Schoutsen 58eb70d3c8 sandbox: main-side fire_event domain gate (Phase 1)
Enforce on main the same <owned_domain>_ rule the sandbox claimed to:
_handle_fire_event now drops any event that is not in the <domain>_
namespace of a domain this group owns, plus a hard deny-list of core
control-plane events (homeassistant_*, call_service, state_changed, ...)
so an owned domain can never alias a core event.

Owned-domain trust is derived purely from main-side state via the new
_owned_domains() helper (entries with entry.sandbox == self.group plus
registered proxy platform domains) — never from a sandbox-supplied
identifier. The helper is the linchpin reused by the register_service
gate in the next phase.

Drops (never raises) on a forged push, so the dispatch loop is unaffected.
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 b237df6872 sandbox: STATUS — plan-review-crash-recovery landing note
Documents all six phases shipped, the /phx:work→intent deviation, the
SETUP_RETRY→SETUP_ERROR fallback (true retry not feasible from the router
seam), the new EntityComponent.async_unregister_remote_platform public
hook, the channel.py both-mirrors note, test coverage, and final
verification (224 + 87 + 30 passed; drift clean; prek clean).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 6cd7d0512d sandbox: crash-recovery regression tests (Phase 6)
Guards every fix in the cluster:

- test_crash_recovery.py (new): crash→respawn→re-register (Phase 1),
  sandbox-dies-availability + recovery (Phase 2), unload-while-down
  releases the platform (Phase 4).
- test_manager.py: stop()-during-spawn completes without hang (Phase 3) —
  a _PausingProcess lands stop() while _spawn is suspended so
  self._process is still None.
- test_channel.py: Channel.close() after EOF still closes the transport
  and awaits inflight exactly once, and is idempotent (Phase 5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 4a887fdeb9 sandbox: fix Channel.close() no-op + honest SETUP_ERROR (Phase 5)
Channel.close() early-returned on `if self._closed: return`. But the read
loop's EOF `finally` already sets `_closed=True` (and cancels, never
awaits, inflight tasks), so a close() after EOF returned immediately —
transport.close() and the inflight gather never ran, leaking the stdin
pipe / unix connection every restart cycle. Split "already closed" (set
_closed, fail pending — idempotent) from "teardown not yet done" (close
transport + await inflight, guarded by a new _close_done flag that runs
exactly once regardless of who set _closed first).

channel.py is hand-mirrored — the identical fix is applied to BOTH copies
(homeassistant/components/sandbox/channel.py and
sandbox/hass_client/hass_client/channel.py).

SETUP_RETRY non-retry: the router runs outside ConfigEntry.async_setup,
so the SETUP_RETRY timer (async_call_later) is never armed for a sandbox
entry — a router-set SETUP_RETRY wedged the entry in a retry state that
never fires (and a later async_setup raised OperationNotAllowed). The
ChannelClosedError-during-entry_setup case now reports SETUP_ERROR
honestly (recoverable via manual reload); ARCHITECTURE.md §5 updated and
a router-driven true retry flagged as a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 54041a6a22 sandbox: tear down main-side proxies on unload-while-down (Phase 4)
router.async_unload_entry returned True without any cleanup when the
sandbox/channel was down, leaking the proxy entities and the
EntityComponent platform registration → a later re-setup hit "has
already been setup!".

Extract a shared _async_unload_main_side helper (delegates to
bridge.async_unload_entry, which now uses the Phase 1 public-hook
teardown) and call it on every exit that should release main-side state:
the sandbox-down early return and a ChannelClosedError mid-unload both
skip the (impossible) remote RPC but still remove the proxies + platform.
A live sandbox that refuses the unload (ChannelRemoteError) still returns
False with the proxies left in place.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 bec9b1cf91 sandbox: bound the shutdown/respawn hangs in manager (Phase 3)
Two paths could hang HA shutdown or wedge a sandbox forever:

- stop() spawn-in-progress race: a stop() that landed inside _spawn read
  self._process as None, terminated nothing, then awaited the supervisor
  forever while the freshly-spawned healthy child ran unsupervised. Fix:
  a post-spawn _stopping check in _run_one_{stdio,unix} kills the child
  stop() missed, and stop()'s await is bounded with a SIGKILL+cancel
  backstop. Terminate logic is extracted into a shared _terminate helper.

- Respawn had no ready-timeout: ready_timeout was only applied in the
  first start(). _supervise_until_exit now bounds the ready handshake on
  every attempt; a child that opens its channel but never signals ready
  is killed and counts against the restart budget instead of leaving the
  sandbox 'starting' forever.

Also: ensure_started no longer hands back a 'starting' zombie — it awaits
readiness (bounded) via the new async_wait_until_ready and raises
SandboxFailedError if the in-flight spawn never becomes running.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 4a7b3683c3 sandbox: mark proxies unavailable when sandbox dies (Phase 2)
sandbox_set_available() had zero callers, so a dead sandbox's proxies
kept serving their last state — a crashed integration's light read "on"
forever to automations and the UI.

- Add SandboxBridge.async_mark_all_unavailable() (flips every owned proxy
  to unavailable via the existing sandbox_set_available).
- Add a manager on_channel_closed callback fired right after the control
  channel is closed on process exit; __init__ wires it to mark the
  group's live bridge unavailable. Proxies flip back to available on
  respawn through the normal register/state_changed round-trip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 46cf39b212 sandbox: tear down old bridge on restart (crash-recovery Phase 1)
Keystone fix for crash/restart recovery: a respawn used to overwrite
data.bridges with a fresh SandboxBridge but never released the old one's
proxy entities or its EntityComponent platform slots, so the first
register_entity after respawn raised ValueError("… has already been
setup!") and every entity for the entry failed permanently.

- Add the public inverse hook EntityComponent.async_unregister_remote_platform
  (mirrors async_register_remote_platform; no private _platforms poke —
  the bridge's old SLF001 poke in async_unload_entry is replaced by it).
- Add SandboxBridge.async_teardown() + a shared _async_teardown_entry
  helper; async_unload_entry now routes through it.
- __init__: on restart, stash the displaced bridge in
  SandboxData.pending_teardown and, when the fresh process goes ready,
  tear it down and re-drive entry_setup (async_schedule_reload) for the
  group's loaded entries. A new manager on_ready callback is the trigger
  (fires on every (re)spawn after MSG_READY); capturing loaded entries
  synchronously keeps a first start from being treated as a respawn.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 e73251f8e6 sandbox: stop tracking generated test .storage; match dev
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 21dcf8ddc7 sandbox: add code-review follow-up plans + intro presentation
Capture the 2026-06-12 sandbox code review as actionable artifacts:

- plans/plan-review-INDEX.md + five phased fix plans covering every verified
  finding — crash/restart recovery, trust-boundary hardening, client-side
  races + int->float wire fidelity, config-flow forwarding fidelity, and
  simplification/dedup. Index records execution order and a finding->plan
  coverage map; plans cross-reference each other by name.
- PRESENTATION.md — a 10-slide intro to the sandbox concept (what data
  represents an integration in another instance -> entity/registry/device,
  service control, event listening, action forwarding, entity RPC for
  query-shaped APIs, then sandboxes as ephemeral one-domain isolates).

The documentation findings from the review were fixed directly (prior commit);
the remaining code findings are planned here, not yet implemented.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 e3f263727a sandbox/docs: consolidate to one architecture doc; drop OVERVIEW + v1
Delete OVERVIEW.md — it duplicated ARCHITECTURE.md at a different altitude and
was the source of every doc-vs-doc disagreement found in review (31-vs-32
proxies, SETUP_RETRY vs SETUP_ERROR, classify-at-entry-setup). ARCHITECTURE.md
is now the single architecture reference; its missing "where to look in the
code" map moved in as §15.

Scrub all v1 references from the live docs: the changelog removal line, the
rename row wording, README's "v1 kept for reference" prose, and CLAUDE's
v1-removal paragraph, Iron-Law cautionary tale, and "v1 removal. DONE" bullet.

Fix ARCHITECTURE drift: crash-budget exhaustion is SETUP_ERROR (not
SETUP_RETRY); classify() runs at flow creation only. Rewrite README.md to a
slim entry point (real stdio:// quick-start, no ws://token/RemoteStore),
re-point CLAUDE.md's OVERVIEW links to ARCHITECTURE.md, and correct CLAUDE.md
to list five core-HA surfaces.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 df7ebf1dc1 sandbox/docs: add doc-audit research notes (ARCHITECTURE/OVERVIEW vs code)
Dated audit snapshots (2026-06-05) cross-checking every concrete name /
RPC / routing rule / table row in ARCHITECTURE.md and OVERVIEW.md against
the implementation. Kept as research artifacts under plans/research/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 74d96ddfd0 sandbox: STATUS — plan-query-rpc landing note
Record what shipped per phase (service-path + EntityQuery request/response),
what stays deferred (subscription/push primitive, todo, browse_media
media-source caveat), the deviations (search via async_internal_search_media,
JSON-safe sandbox response, callerless raise_not_proxied retained), and the
green verification summary lines.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:24 -04:00
Paulus SchoutsenandClaude Opus 4.8 0e14f55874 sandbox/docs: query RPCs are implemented; subscriptions still open
Reflect the shipped request/response query RPCs across the docs: the
server-side query + WS-only mutation entity APIs now answer with real data
(service-path return_response + the generic entity_query RPC), so the
catalogue's status column, ARCHITECTURE §8/§14, OVERVIEW's "still open"
bullet, and the CLAUDE.md follow-up all move from "raises" to "wired". Kept
accurate as still-open: the subscription/push primitive (the */subscribe
one-shot-only rows + todo item-list push) and the media_player.browse_media
media-source caveat.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 e89455414a sandbox: test EntityQuery serialization fidelity + error paths
Round-trip rebuild tests for SearchMedia and Segment (the as_dict /
dataclass-asdict-vs-constructor asymmetry), per-op EntityQuery proxy tests
(media search, release notes, vacuum segments, calendar update/delete) that
assert the rebuilt typed object and the forwarded method + args, and the two
error paths: a sandbox-side ServiceValidationError translating to a
HomeAssistantError on main, and a closed channel degrading to a clean
HomeAssistantError. Client-side coverage for the EntityQuery handler:
method invocation + kwarg passing, unknown entity_id, unknown method, and a
raising method propagating its exception type on the error frame.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 07a7b918ca sandbox: wire the service-less query ops onto EntityQuery
Replace the remaining raise_not_proxied stubs with EntityQuery forwards +
typed rebuilds, so every query-shaped entity API now answers with real data:

- media_player.async_search_media -> async_internal_search_media (which
  rebuilds the SearchMediaQuery from flat kwargs on the sandbox side, so the
  query crosses as plain JSON); rebuilds SearchMedia, reusing the BrowseMedia
  helper for its result list.
- update.async_release_notes -> async_release_notes (plain str/None).
- vacuum.async_get_segments -> async_get_segments; rebuilds list[Segment].
- calendar.async_update_event / async_delete_event -> the matching WS-only
  entity methods (None result).

The sandbox-side serialisation is the as_dict-aware JSON encoder already
added with the handler, so SearchMedia/BrowseMedia/Segment cross verbatim.
raise_not_proxied is now callerless but kept exported for the still-deferred
subscription/todo-push primitive.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 50a2d16d9d sandbox: add the generic EntityQuery request/response RPC
The fire-and-forget call_service channel can command an entity but can't
ask it a server-side question that has no SupportsResponse service to ride.
Add one generic EntityQuery RPC for those, mirroring the call_service path
end to end (proto -> codec registry -> bridge sender + error translation ->
sandbox handler -> proxy helper):

- proto: EntityQuery {sandbox_entity_id, method, args, context_id} and
  EntityQueryResult {result} (the return wrapped as {"value": ...} so
  scalar/list/None are all representable). Gencode regenerated into both
  _pb2 mirrors; drift guard passes.
- MSG_ENTITY_QUERY constant + REGISTRY entry added to both protocol/messages
  mirrors.
- SandboxBridge.async_entity_query builds the request, remembers the context
  before the id is reduced to a wire value, translates remote/closed errors
  through the existing paths, and unwraps {"value": ...}.
- EntryRunner._handle_entity_query resolves the entity on the private hass,
  invokes the named method with the decoded kwargs, and serialises the return
  through the as_dict-aware JSON encoder; raised HA/voluptuous errors
  propagate as channel error frames so main rebuilds the same shape.
- SandboxProxyEntity._entity_query is the proxy-side companion to
  _call_service.

No proxy op is wired onto it yet — that is the next phase.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 29f0c31b4f sandbox: proxy response queries via the call_service path
Wire the three query-shaped entity APIs that have a SupportsResponse
service onto the existing call_service + return_response channel, so a
sandboxed entity answers them with real data instead of raising:

- calendar.async_get_events -> calendar.get_events service, rebuilding
  list[CalendarEvent] from the response (explicit field mapping, ISO
  date/datetime parse — not a **dict splat).
- weather.async_forecast_{daily,hourly,twice_daily} -> weather.get_forecasts
  service; Forecast is a plain TypedDict, returned verbatim.
- media_player.async_browse_media -> media_player.browse_media service,
  rebuilding the recursive BrowseMedia from its frontend-shaped as_dict.

SandboxProxyEntity._call_service grows a return_response flag that decodes
the CallServiceResult response into a dict. The sandbox-side call_service
handler now runs rich service responses (e.g. a BrowseMedia object keyed by
entity_id) through the as_dict-aware JSON encoder before packing the Struct,
yielding the exact wire shape main rebuilds from.

Caveat documented at the browse_media call site: a sandboxed player's browse
surfaces only its own sources; the media_source tree is empty inside the
sandbox (media_source runs on main). Round-trip rebuild unit tests cover the
as_dict-vs-constructor asymmetry first (plan Risk #2).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 5b0a73ebb5 sandbox/docs: catalogue query-shaped RPC gap + request/response plan
Document the unproxied query/subscribe/WS-only entity APIs, their interim
raise behaviour, and the two missing primitives (request/response +
subscription RPC) in docs/query-shaped-rpcs.md. Add the implementation
plan (plan-query-rpc.md): a generic EntityQuery RPC for the service-less
ops + reuse of the existing call_service return_response path for ops
that have a SupportsResponse service. Note the media_player.browse_media
caveat (no media_source tree inside the sandbox). Cross-reference from
ARCHITECTURE/OVERVIEW/CLAUDE.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 932b54cc0e sandbox: fail loudly on unproxied query-shaped entity APIs; block todo
The server-side query / subscribe / WS-only-mutation entity APIs the
fire-and-forget call_service bridge can't express (calendar listings +
event update/delete, weather forecasts, media browse/search, update
release notes, vacuum segments) previously returned empty/None silently.
Add entity.raise_not_proxied and have those proxy methods raise
HomeAssistantError instead, so the gap fails loudly until a real query
RPC lands.

todo is a special case: its To-do panel reads the sync todo_items
property that also feeds TodoListEntity.state, so it can't be a query at
all. Route it to main via SANDBOX_INCOMPATIBLE_PLATFORMS and drop the
proxy (matching the camera/image precedent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 7246269118 sandbox/docs: document translation forwarding; fix OVERVIEW drift
Both docs now describe the translation-forwarding subsystem in the body,
not just the goal: live pull (sandbox/get_translations RPC + provider
overlay) and the picker catalog hook.

- OVERVIEW: add a Translation forwarding section + "where to look" row +
  v1-diff row. Fix pre-existing drift: ALWAYS_MAIN is 24 entries across
  three groups (was listed as 6), failed-sandbox setup is SETUP_ERROR
  (not SETUP_RETRY), and the manager runs no periodic ping loop.
- ARCHITECTURE: add §11 Translation forwarding (renumber following
  sections), list translation.py/catalog.py in §2, and correct the core
  touch surface from three to five hooks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 d049ab23f9 sandbox/docs: mention translations in the unified view; drop architecture.html
Translation forwarding (live pull-RPC + catalog provider) now puts the
sandboxed integration's translations on main alongside its entities,
services, and events — note it in the OVERVIEW + ARCHITECTURE goals.
Remove the generated architecture.html; the architecture is published to
a gist instead of carrying a rendered artifact in the tree.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 01dac485fc sandbox: STATUS — plan-translation-forwarding Phase A landing note
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 266bbbe5e0 sandbox: A3 — document the catalog-provider HACS contract
Add sandbox/docs/catalog-provider-contract.md describing the display-only
picker catalog hook: the discoverability gap it closes, the
async_register_sandbox_catalog_provider API, and the contract — separate
from the sha-pinned source resolver, name load-bearing, title_translations
optional, no validation, display-only scope, and how it complements the
Phase B live RPC for the cold picker case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 4c2feccd78 sandbox: A2 — merge catalog into descriptions + title fallback
Wire the A1 catalog hook into the two display paths a sandbox-only custom
falls through today:

- async_get_integration_descriptions (loader.py): append catalog
  descriptors to the custom integration/helper buckets so the add-
  integration picker lists them. On-disk customs carry richer metadata,
  so the disk scan wins on a domain collision.
- _async_get_component_strings (helpers/translation.py): when a domain
  has no on-disk Integration (IntegrationNotFound on main), take its
  "title" from the catalog — a localized title_translations[lang] if
  present, otherwise degrading to the descriptor name.

Tests: catalog entry appears in descriptions with picker name + defaults
+ helper-bucket routing; on-disk custom wins a collision; title fallback
uses title_translations and degrades to name when absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 7b8fc84459 sandbox: A1 — catalog-provider hook for picker discoverability
Add a separate, display-only catalog hook so a custom integration whose
code lives only in a sandbox (never on main's disk) can be listed and
named in the add-integration picker without spawning a sandbox.

Core (homeassistant/loader.py) owns the registry because core consumes
it: SandboxIntegrationDescriptor, SandboxCatalogProvider,
DATA_SANDBOX_CATALOG_PROVIDERS, async_register_sandbox_catalog_provider,
async_get_sandbox_catalog. This mirrors the Phase B translation-provider
precedent (hook + consumer co-located in core).

homeassistant/components/sandbox/catalog.py re-exports the hook so HACS
registers through a sandbox namespace parallel to the source resolver —
but the catalog stays deliberately separate from the sha-pinned, security-
critical source resolver: it is eager, enumerable and cosmetic only.

Wired into descriptions + title fallback in A2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 01718e3f84 sandbox: STATUS — plan-translation-forwarding Phase B landing note
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 118914e2ec sandbox: B4 — main-side translation provider impl + registration
Implement SandboxTranslationProvider and register it into core's translation
hook from async_setup (unregistered on stop). For each requested component it:

- resolves the owning sandbox group — a loaded entry's .sandbox field wins,
  else the live SandboxFlowProxy of a brand-new custom's in-progress flow
  (new sandbox_group accessor on the proxy);
- carves out built-ins (Integration.is_built_in ⇒ main reads its byte-identical
  disk copy, never the wire);
- batches each group's custom domains into one get_translations RPC per
  language (5s timeout), and degrades to empty strings on a down/closed/slow
  channel so the cache-lock overlay never blocks the frontend.

router.async_unload_entry now invalidates a sandboxed entry's cached
translations, so a reload at a new integration-source ref re-pulls fresh
strings on the next fetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 d955595517 sandbox: B3 — core translation provider hook + overlay + invalidation
Add a sandbox-agnostic seam to the translation cache, mirroring the
sandbox.sources source-resolver convention:

- async_register_sandbox_translation_provider(hass, provider): a HassKey-backed
  registry with an unregister callback. The provider is awaited inside the
  cache load and returns {language: {domain: raw_strings}} for only the domains
  it owns.
- _TranslationCache._async_load overlays the provider result onto
  translation_by_language_strings after async_get_integrations and before
  _build_category_cache, so sandboxed strings flow through the same flatten /
  English-fallback / loaded machinery as on-disk strings. A custom sandboxed
  domain (IntegrationNotFound on main) thus stops resolving to {}.
- _TranslationCache.async_invalidate + async_invalidate_translations wrapper:
  the first eviction API (translations were never unloaded), called by the
  sandbox when a custom integration is re-fetched at a new ref.

Core never raises on a provider; degrade-to-empty is the provider's contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 6b33d76434 sandbox: B2 — get_translations runtime handler + string loader
Register a sandbox/get_translations handler in SandboxRuntime. It loads raw
translation strings for the requested domains from the sandbox's own
filesystem (built-in from the bundled package, custom from the fetched
<config>/custom_components/<domain>) by reusing core's
_async_get_component_strings against the sandbox-private hass — which also
pre-fills 'title' from integration.name. Main cannot run that fallback for a
custom domain because it holds no Integration, so the title must be injected
here. Replies with {language, strings: {domain: raw dict}}.

Tests cover built-in title pass-through, custom title injection, the empty
case, the Struct packing, and the no-flow-runner guard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 ca7b7a05a4 sandbox: B1 — get_translations wire protocol
Add the sandbox/get_translations message pair to the control-channel proto
and regenerate the checked-in gencode for both no-cross-import mirrors.
Mirror MSG_GET_TRANSLATIONS in both protocol.py files and register the
message pair in both messages.py REGISTRY copies.

Request {language, domains[]}; result {language, strings: {domain: raw
strings.json dict}} — main batches a group's custom domains into one call;
built-in domains never cross the wire.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 b7d9bf6a5b sandbox/plans: add translation-forwarding brainstorm + plan
Brainstorm → plan for forwarding a sandboxed integration's translations
into main: live pull-RPC (Phase B) for running integrations + a catalog
provider (Phase A) for picker discoverability. Includes interview,
research notes, scratchpad, and the phased plan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 595c7077ac sandbox: PLAN_RUNNER — brief is just piped, no file references
Rewrote the briefing so it never frames the brief as a file or mentions
the former tempfile handoff: compose it, pipe it straight into the session
(heredoc), claude-screen pastes it as one message. Dropped the file-pipe
example and the "no tempfile dance" aside.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 490f873b26 sandbox: PLAN_RUNNER — drop file-handoff framing from briefing
Now that claude-screen pastes multi-line directly, the brief no longer
needs a tempfile + "Read /tmp/X" pointer. Step 1 reads "Compose the brief"
(source it from a heredoc or any scratch file) instead of "Write the
brief"; step 2 shows both heredoc and file pipes. The brief is just stdin,
not a handoff artifact.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 30c4b3a63d sandbox: PLAN_RUNNER — STATUS marker now lives under status/
Follow-up to the status/ reorg: the brief's STATUS path and the monitor
until-loop both point at sandbox/status/STATUS-<plan>.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 6f72c79be9 sandbox: move STATUS files into status/
Tidy the directory: the 29 per-phase + per-plan landing records
(STATUS-phase-*.md, STATUS-plan-*.md) move out of the sandbox/ root into
sandbox/status/ (git mv, blame preserved). Live current-state docs
(CLAUDE.md, README.md, OVERVIEW.md, FOLLOWUPS.md, architecture.html, the
docker-compose harness comment) now point at status/. Historical records
(the STATUS bodies themselves, plans/*.md, plan.md) keep their original
text by convention.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 48f6990c2d sandbox: PLAN_RUNNER — multi-line briefs now paste directly, no tempfile
claude-screen pastes multi-line prompts directly: bracketed-paste markers
keep embedded newlines literal, and the submit \r is sent as a separate
keystroke a beat later (the concatenated \r was what raced the paste and
submitted mid-prompt). Verified live — a 3-line prompt lands as one
message. Doc no longer mentions any file-handoff.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 41711fe573 sandbox: PLAN_RUNNER — claude-screen now auto-handles multi-line briefs
The single-line file-handoff is now built into claude-screen itself (it
detects a newline in the piped prompt, stashes the brief to a tempfile,
and pastes a pointer). So the doc just pipes the brief straight in; the
manual "write to /tmp + echo a single line" dance is gone.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 5b3571b454 sandbox: PLAN_RUNNER — plans MUST be executed via the phx:work skill
Make explicit that each sub-session steps through its plan with the
phx:work skill (task-by-task with per-step compile/test verification),
not ad-hoc edits. Added as a brief hard rule + a why-this-shape bullet.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.7 83763c94b3 sandbox: add PLAN_RUNNER.md — the per-plan sub-session workflow
Documents the loop used to build this batch: write a brief to a tempfile,
spawn a fresh Claude in a screen window via single-line file-handoff, watch
for a STATUS marker, verify independently, push, kill the window. Captures
the gotchas that bit (single-line stdin, prompt-submit confirmation,
prefix-match window names, orchestrator-only push).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 16a3c17397 sandbox/tests: rename test_phase1_spike_late_additions_pin_to_main
Drop the build-phase scaffolding from the test name; it just verifies ai_task
and image pin to ALWAYS_MAIN. -> test_ai_task_and_image_pin_to_main.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 5433c8a558 sandbox: trim the compat lane to one runner
The one-shot full cross-sweep that produced the original backlog
(run_compat_full.py + categorize_failures.py + generate_backlog.py) and its
machine-generated outputs (COMPAT_FULL.md/.csv, COMPAT_LATEST.md, COMPAT.csv,
BACKLOG_FAILURES.json) were Phase-16 measurement scaffolding; the gate is long
cleared. Keep the single ongoing runner (run_compat.py) and the two curated
summaries (COMPAT.md, BACKLOG.md). Git-ignore the per-run machine output so it
stops being checked in. Living docs updated; recover the full-sweep tooling
from git history if a fresh tree-wide sweep is ever needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 63ead8828c sandbox/bridge: route the service forwarder through _raw_call_service
The registered-service forwarder (_build_service_forwarder._forward) rebuilt its
own pb.CallService request and duplicated the ChannelRemoteError/
ChannelClosedError translation that _raw_call_service already does. With the
batcher gone, _raw_call_service is the single low-level send helper — have
_forward call it and keep only its response-extraction logic. No behaviour
change (the channel-closed error message is now the shared generic one).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 21dde0ab15 sandbox: remove the Option A/B spike harness
The spike (hass_client/spike/: bridge_a, bridge_b, rig, synthetic_light,
transport + tests/components/sandbox/test_spike.py) was a one-off bake-off to
choose between entity-bridge designs. Option B was chosen and shipped long ago;
nothing in production imports the spike, only its own test did. Delete it.

docs/entity-bridge-decision.md keeps the rationale and the measured numbers as
the decision record, with a note that the harness is recoverable from git
history.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 b8249ec9d9 sandbox: drop the call-service batcher; keep the first iteration simple
Each proxy entity service call now forwards as its own single
`sandbox/call_service` RPC. The per-loop-tick coalescing batcher
(_CallServiceBatcher / _BatchBucket) added complexity the first iteration
doesn't need, so it is removed; async_call_service calls _raw_call_service
directly. Behaviour is unchanged except a multi-entity area call now pays one
RPC per entity instead of one coalesced RPC.

Coalescing same-tick calls is recorded as a future optimisation in
docs/FOLLOWUPS.md (with the 200-light perf benchmark that validated it). Living
docs updated; the phase-history records are left as-is.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 f62efecc0f sandbox/bridge: correct misleading 'fire-and-forget' wording
A service call is never fire-and-forget: each batched caller awaits the
coalesced RPC's completion via its future, which resolves with the result or
the raised error, so every caller learns when its call finished. Batching only
shares the *wire* call, not the await; only a response *value* can't be
coalesced (hence the response bypass). Wording-only; no behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00
Paulus SchoutsenandClaude Opus 4.8 ee74f766bb sandbox: drop development-phase references from code
The final deliverable should not carry the scaffolding of the phases it was
built in. Reword comments, docstrings, and generated-output strings that named
build phases (Phase N / T1-T3 / Phase A1-A2) to describe what the code does,
and rename the phase-numbered test files:

  test_phase4_subprocess  -> test_subprocess
  test_phase9_shutdown    -> test_shutdown
  test_phase13_proxies    -> test_domain_proxies
  test_phase14            -> test_schema_and_unload
  test_phase19_devices    -> test_device_registry

Comments/docstrings/filenames only; no logic changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 15:12:23 -04:00