Discovery flows feed the proxy non-JSON objects — a *ServiceInfo dataclass
(IPv4Address fields, sets) as the first-step user_input and a DiscoveryKey in
context — which dict_to_struct couldn't hold, crashing the flow unhandled
(the except only caught channel errors). The router routes discovery-sourced
flows to the sandbox with no source filter, so this was reachable.
Main side: _to_jsonable walks context + first-step payload into Struct-safe
primitives (dataclasses -> field dicts, IPs -> str, sets -> lists) before
dict_to_struct. Broadened the except to abort cleanly on any unmapped payload
that still trips the marshaller.
Sandbox side: _rehydrate_discovery rebuilds the real DiscoveryKey + the
source's BaseServiceInfo (zeroconf/homekit, ssdp, dhcp, usb, hassio, mqtt) so
async_step_<source> receives the type it expects. Unmapped sources (bluetooth)
leave a dict; reconstruction failures degrade to the dict with the proxy's
clean abort as the outer backstop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two fixes in the flow proxy:
1. MENU support end-to-end. async_show_menu is common and marshals like a
form. Added FlowResult.menu_options (ListValue) + sort to the proto, marshal
it sandbox-side, and re-issue async_show_menu on main. A dict-form menu
(id -> label) crosses as ordered [id, label] pairs so labels survive; a
menu selection is forwarded as the sandbox flow's {"next_step_id": <chosen>}
navigation choice.
2. Leak fix. The unsupported-result-type branch set _terminated=True before
aborting, which made async_remove skip the flow_abort RPC — leaving the
sandbox-side flow in progress and wedging retries on already_in_progress if
it had set a unique_id. The non-terminal result types (external-step,
progress) no longer set _terminated, so async_remove reaps the sandbox flow.
Regenerated proto gencode for both mirrors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The proxy CREATE_ENTRY path read only data/title/description and called
async_create_entry, which stamps the proxy class defaults VERSION=1 /
MINOR_VERSION=1 / options={}. A sandboxed flow with VERSION>1 lost its
schema version (spurious migration on next setup) and dropped options.
async_create_entry reads self.VERSION/self.MINOR_VERSION off the *instance*
(config_entries.py async_create_entry), so override the proxy instance's
values from the wire result before the call, and plumb options= through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records the 7-phase client-runtime + wire-fidelity batch: per-phase
summaries, the messages.py both-mirrors byte-identical note, the Phase 7
no-narrowing decision, the Phase 3 correctness-fix approach + plan-5
writer-queue handoff, tests added, and final green verification.
_handle_shutdown set the shutdown event via call_soon so the reply lands
on the wire first, but call_soon only buys one loop turn — if the reply
write suspended (write-lock contention from unload pushes, drain
backpressure on a large restore_state), run()'s finally closed the channel
and cancelled the in-flight reply task, so main lost restore_state.
Add Channel.drain_inflight(timeout): wait for in-flight inbound handler
tasks (the shutdown reply included) to finish before close, without
cancelling anything. run()'s finally now drains before close(), so the
reply completes its write instead of being cancelled mid-flush.
Write-lock check (Phase 4 bullet 2): _write_lock is a plain mutex held
only across one frame write, so the shutdown reply and the unload-driven
_push_unregister writes serialize — no circular wait, no deadlock.
drain_inflight added to both hand-mirrored channel.py copies. Regression
test stalls the reply's drain and asserts main still receives it (verified
it fails with ChannelClosedError when the drain step is removed).
protobuf.Struct stores all numbers as double, so _value_to_py returned
entry.data ints, service_data ints, and the store envelope's version/
minor_version as floats — breaking socket()/isinstance(int)/range(...).
_value_to_py now coerces whole-number floats back to int
(int(v) if v.is_integer() else v); genuinely fractional values (0.5) keep
their float type. Reviewed every struct_to_dict / listvalue_to_list
consumer (flow/entry data, service_data, target, entity_query args/result,
store data, state attributes, capabilities, event_data, translations) —
none needs a whole-number float to stay float, so the global coercion is
safe; no field narrowing required.
Applied byte-identically to BOTH hand-mirrored copies:
homeassistant/components/sandbox/messages.py
sandbox/hass_client/hass_client/messages.py
(verified identical via diff). The explicit int32 proto fields (version,
minor_version, supported_features on EntrySetup) bypass the Struct and are
unaffected.
A failed async_setup left the rebuilt ConfigEntry in the sandbox's
config_entries (only unload popped it), so main's later retry of the same
entry_id was rejected with 'entry already loaded'. On both failure paths
(async_setup raised / returned False) the entry is now popped before
returning ok=False, so a re-sent entry_setup starts clean.
Sandbox-side half of plan 1's SETUP_RETRY decision: main shipped honest
SETUP_ERROR + manual reload and remains the only retry driver. The
sandbox-side ConfigEntryNotReady timer is deliberately NOT enabled (the
sandbox hass is never async_started).
EntityBridge._register added an approval refcount per registered entity
(approved.add(domain)) but nothing ever decremented it, so a platform
domain stayed approved for the process lifetime — the service/event gate
stayed open for domains with zero owning entities.
Track each entity's contributed domain (_approved_domain) and release it
(_release_approval -> approved.remove) on both unregister paths: the
new_state-is-None removal and the Phase 3 removal-while-pending flush.
Symmetric with the per-entity add; the refcount drops to absent when the
last entity of a domain unregisters. EntryRunner's per-entry
approved.remove(entry.domain) on entry_unload already balances the
per-entry add, so it is unchanged.
_on_state_changed dropped any state_changed while an entity sat in
_pending (register RPC in flight), and the register task pushed only the
snapshot captured at task-creation. A fast second update was lost; a
removal in the window left a ghost proxy on main (the entity wasn't in
_registered yet, so the removal was dropped).
After the register RPC resolves, the task now:
- if a removal raced (new _removed_while_pending flag), unregisters the
entity it just registered, so main keeps no ghost proxy;
- otherwise re-reads hass.states and pushes a state_changed when the live
state differs from the registered snapshot, flushing the coalesced gap.
This is the correctness fix only. Plan 5 (simplification) builds a
single-writer queue on top of the entity push path; when it lands it
should subsume this flush into the queue's ordering guarantees (noted in
code).
SandboxRuntime.run pushed Ready (manager flips to running, router sends
entry_setup), awaited the warm-load store_load, and only then registered
MSG_ENTRY_SETUP et al. An entry_setup arriving in that window hit
ChannelUnknownType -> SETUP_ERROR.
Now: register every inbound handler first, run the (outbound) warm-load
store_load, then push Ready as the last frame. This both removes the
no-handler race and preserves the warm-load-before-entry_setup invariant
(Ready timing still gates entry_setup until the restore cache is warm).
EVENT_SERVICE_REGISTERED fires synchronously while a service is
registered inside async_setup_entry, but EntryRunner only approves the
domain after async_setup returns, so ServiceMirror dropped those early
registrations with a warning and never replayed them.
ApprovedDomains now fires approve-listeners on the first (absent->present)
add; ServiceMirror subscribes async_sync_domain, which re-mirrors every
already-registered service of the freshly-approved domain (skipping any
already in _mirrored). Covers both the entry-runner approve path and the
entity-bridge per-entity approve path.
EventMirror is left as-is: owned events are transient, so a past event
cannot be replayed (noted in plan Phase 1).
All trust-boundary gates this plan proposed shipped (Phases 1-6, none
deferred), so the malicious-sandbox guarantees the architecture asserts now
match enforced reality. Added a one-line 'enforced on main in
bridge.py/channel.py' note for each gate, traceable to code:
* §4 channel read-backpressure shedding (both mirrors)
* §8 register_entity entry ownership + foreign-device-merge refusal
* §8 register_service / fire_event owned-domain gates + core deny-list
* §8 context-cache eviction bound on the resolve path
* §9 store key-length + value/total/key-count quotas
* §11 translation overlay narrowed to requested ∩ returned
No 'Known trust-boundary gaps' subsection is needed — nothing was deferred.
Added a changelog row. README.md/CLAUDE.md make no overstated boundary
claims (only 'isolated subprocesses', accurate), so no softening needed.
One forged-frame test per trust-boundary gate:
* fire_event: core (homeassistant_stop/call_service/state_changed) and
unowned-domain (zha_event/hue_event) events are dropped, never reach the bus.
* register_service: an unowned domain (persistent_notification) is rejected.
* register_entity: a foreign entry_id (entry.sandbox != group) is rejected;
a device_info colliding with a foreign entry's device is refused (no merge).
* translation: a forged foreign domain returned alongside the owned one is
dropped; only the requested ∩ returned survives.
* store_save: an overlong key and an oversized value are rejected; nothing
hits disk.
* context cache: a flood of distinct unknown context_ids stays bounded by
_CONTEXT_CACHE_MAX.
* channel backpressure: over the max_queued cap, inbound calls are shed with
a ChannelOverloaded error and the inflight set stops growing.
make_channel_pair gained max_queued_a/b passthrough for the backpressure test.
Two unbounded-growth vectors closed:
1. Context cache on resolve. _resolve_context minted a fresh Context per
unknown context_id but never enforced _CONTEXT_CACHE_MAX (only
_remember_context did), so a sandbox flooding distinct unknown ids grew
the cache without bound. Factor a single _store_context() helper used by
both paths so the cap + expiry-ordering apply uniformly.
2. Channel read backpressure (BOTH mirrors). The reader create_task'd a
handler per inbound frame; the inflight semaphore caps *running*
handlers but queued tasks — each pinning a decoded payload up to
MAX_FRAME_SIZE — grew without bound under a flood. _dispatch now sheds
over a DEFAULT_MAX_QUEUED cap on inflight handler tasks: inbound calls
are rejected with a ChannelOverloaded error frame, pushes dropped.
Responses are always handled inline above the gate, so backpressure
never starves a reply.
The channel.py edit is applied byte-identically to both hand-mirrored
copies (homeassistant/components/sandbox/channel.py and
sandbox/hass_client/hass_client/channel.py), rebased on top of plan
#1's Channel.close() fix, in this separate commit.
Design note: shed (reject/drop over a bounded cap) rather than block the
reader on the semaphore. Blocking the shared reader would deadlock the
documented nested-call pattern — a handler that issues channel.call() and
awaits its reply through the same reader would stall it once all slots are
held by such handlers (real on the client mirror: a call_service handler
doing a store_save round-trip to main). Shedding bounds memory without
that liveness hazard and stays safe in both mirrors.
_validate_key now caps key length (128, well under NAME_MAX). async_save
caps each value (4 MB) and enforces a per-group dir quota (32 MB total,
256 keys) via _enforce_group_quota before the atomic write, so a
compromised sandbox can no longer exhaust the host disk through the
store-routing channel. Limits are commented, generous-but-finite
constants.
Over-quota writes raise HomeAssistantError → remote-error frame; the
sandbox-side async_store_save already catches ChannelRemoteError and logs
(keeping its in-memory data), so a rejected flush degrades, not crashes.
The provider overlay (_TranslationCache._async_overlay_sandbox_strings)
splices every domain a sandbox returns, narrowed only by the broad
requested components set — so a compromised sandbox could return strings
for a co-requested victim domain (hue, http) and poison its frontend
strings. async_get_translations now keeps only the requested ∩ returned
intersection (domains & strings.keys()), the set this group was actually
asked to resolve, before handing strings to the cache.
_handle_register_entity now requires entry.sandbox == self.group, not
just that the entry_id resolves: a compromised sandbox may only register
entities for entries main routed to *this* group. Without this it could
attach entities and pre-create devices against a victim integration's
config entry.
Also reject a device pre-create that would merge (via shared
identifiers/connections) into a device already owned by a config entry
outside this group — _reject_foreign_device_merge — closing the
device-registry hijack vector. entry.sandbox is set by main at flow
completion, never by the sandbox.
entry_setup/entry_unload are main-initiated (main supplies the entry_id),
and the store server is scoped one-dir-per-channel by construction, so no
further entry_id trust points need gating.
Test entry fixtures across bridge/entity_query/domain_proxies/
crash_recovery/proto_transport updated to tag their entries sandbox="built-in".
_handle_register_service now requires the service domain to be one this
group owns (same main-side _owned_domains() derivation as the fire_event
gate), so a compromised sandbox can no longer squat
persistent_notification.* or any unclaimed domain.service. Unowned
domains are rejected with a HomeAssistantError → remote-error frame; the
existing refuse-to-clobber-an-existing-handler check is kept.
Existing register_service tests updated to own their mock domains via a
MockConfigEntry(sandbox="built-in").
Enforce on main the same <owned_domain>_ rule the sandbox claimed to:
_handle_fire_event now drops any event that is not in the <domain>_
namespace of a domain this group owns, plus a hard deny-list of core
control-plane events (homeassistant_*, call_service, state_changed, ...)
so an owned domain can never alias a core event.
Owned-domain trust is derived purely from main-side state via the new
_owned_domains() helper (entries with entry.sandbox == self.group plus
registered proxy platform domains) — never from a sandbox-supplied
identifier. The helper is the linchpin reused by the register_service
gate in the next phase.
Drops (never raises) on a forged push, so the dispatch loop is unaffected.
Documents all six phases shipped, the /phx:work→intent deviation, the
SETUP_RETRY→SETUP_ERROR fallback (true retry not feasible from the router
seam), the new EntityComponent.async_unregister_remote_platform public
hook, the channel.py both-mirrors note, test coverage, and final
verification (224 + 87 + 30 passed; drift clean; prek clean).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Guards every fix in the cluster:
- test_crash_recovery.py (new): crash→respawn→re-register (Phase 1),
sandbox-dies-availability + recovery (Phase 2), unload-while-down
releases the platform (Phase 4).
- test_manager.py: stop()-during-spawn completes without hang (Phase 3) —
a _PausingProcess lands stop() while _spawn is suspended so
self._process is still None.
- test_channel.py: Channel.close() after EOF still closes the transport
and awaits inflight exactly once, and is idempotent (Phase 5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Channel.close() early-returned on `if self._closed: return`. But the read
loop's EOF `finally` already sets `_closed=True` (and cancels, never
awaits, inflight tasks), so a close() after EOF returned immediately —
transport.close() and the inflight gather never ran, leaking the stdin
pipe / unix connection every restart cycle. Split "already closed" (set
_closed, fail pending — idempotent) from "teardown not yet done" (close
transport + await inflight, guarded by a new _close_done flag that runs
exactly once regardless of who set _closed first).
channel.py is hand-mirrored — the identical fix is applied to BOTH copies
(homeassistant/components/sandbox/channel.py and
sandbox/hass_client/hass_client/channel.py).
SETUP_RETRY non-retry: the router runs outside ConfigEntry.async_setup,
so the SETUP_RETRY timer (async_call_later) is never armed for a sandbox
entry — a router-set SETUP_RETRY wedged the entry in a retry state that
never fires (and a later async_setup raised OperationNotAllowed). The
ChannelClosedError-during-entry_setup case now reports SETUP_ERROR
honestly (recoverable via manual reload); ARCHITECTURE.md §5 updated and
a router-driven true retry flagged as a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
router.async_unload_entry returned True without any cleanup when the
sandbox/channel was down, leaking the proxy entities and the
EntityComponent platform registration → a later re-setup hit "has
already been setup!".
Extract a shared _async_unload_main_side helper (delegates to
bridge.async_unload_entry, which now uses the Phase 1 public-hook
teardown) and call it on every exit that should release main-side state:
the sandbox-down early return and a ChannelClosedError mid-unload both
skip the (impossible) remote RPC but still remove the proxies + platform.
A live sandbox that refuses the unload (ChannelRemoteError) still returns
False with the proxies left in place.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two paths could hang HA shutdown or wedge a sandbox forever:
- stop() spawn-in-progress race: a stop() that landed inside _spawn read
self._process as None, terminated nothing, then awaited the supervisor
forever while the freshly-spawned healthy child ran unsupervised. Fix:
a post-spawn _stopping check in _run_one_{stdio,unix} kills the child
stop() missed, and stop()'s await is bounded with a SIGKILL+cancel
backstop. Terminate logic is extracted into a shared _terminate helper.
- Respawn had no ready-timeout: ready_timeout was only applied in the
first start(). _supervise_until_exit now bounds the ready handshake on
every attempt; a child that opens its channel but never signals ready
is killed and counts against the restart budget instead of leaving the
sandbox 'starting' forever.
Also: ensure_started no longer hands back a 'starting' zombie — it awaits
readiness (bounded) via the new async_wait_until_ready and raises
SandboxFailedError if the in-flight spawn never becomes running.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
sandbox_set_available() had zero callers, so a dead sandbox's proxies
kept serving their last state — a crashed integration's light read "on"
forever to automations and the UI.
- Add SandboxBridge.async_mark_all_unavailable() (flips every owned proxy
to unavailable via the existing sandbox_set_available).
- Add a manager on_channel_closed callback fired right after the control
channel is closed on process exit; __init__ wires it to mark the
group's live bridge unavailable. Proxies flip back to available on
respawn through the normal register/state_changed round-trip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keystone fix for crash/restart recovery: a respawn used to overwrite
data.bridges with a fresh SandboxBridge but never released the old one's
proxy entities or its EntityComponent platform slots, so the first
register_entity after respawn raised ValueError("… has already been
setup!") and every entity for the entry failed permanently.
- Add the public inverse hook EntityComponent.async_unregister_remote_platform
(mirrors async_register_remote_platform; no private _platforms poke —
the bridge's old SLF001 poke in async_unload_entry is replaced by it).
- Add SandboxBridge.async_teardown() + a shared _async_teardown_entry
helper; async_unload_entry now routes through it.
- __init__: on restart, stash the displaced bridge in
SandboxData.pending_teardown and, when the fresh process goes ready,
tear it down and re-drive entry_setup (async_schedule_reload) for the
group's loaded entries. A new manager on_ready callback is the trigger
(fires on every (re)spawn after MSG_READY); capturing loaded entries
synchronously keeps a first start from being treated as a respawn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Capture the 2026-06-12 sandbox code review as actionable artifacts:
- plans/plan-review-INDEX.md + five phased fix plans covering every verified
finding — crash/restart recovery, trust-boundary hardening, client-side
races + int->float wire fidelity, config-flow forwarding fidelity, and
simplification/dedup. Index records execution order and a finding->plan
coverage map; plans cross-reference each other by name.
- PRESENTATION.md — a 10-slide intro to the sandbox concept (what data
represents an integration in another instance -> entity/registry/device,
service control, event listening, action forwarding, entity RPC for
query-shaped APIs, then sandboxes as ephemeral one-domain isolates).
The documentation findings from the review were fixed directly (prior commit);
the remaining code findings are planned here, not yet implemented.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete OVERVIEW.md — it duplicated ARCHITECTURE.md at a different altitude and
was the source of every doc-vs-doc disagreement found in review (31-vs-32
proxies, SETUP_RETRY vs SETUP_ERROR, classify-at-entry-setup). ARCHITECTURE.md
is now the single architecture reference; its missing "where to look in the
code" map moved in as §15.
Scrub all v1 references from the live docs: the changelog removal line, the
rename row wording, README's "v1 kept for reference" prose, and CLAUDE's
v1-removal paragraph, Iron-Law cautionary tale, and "v1 removal. DONE" bullet.
Fix ARCHITECTURE drift: crash-budget exhaustion is SETUP_ERROR (not
SETUP_RETRY); classify() runs at flow creation only. Rewrite README.md to a
slim entry point (real stdio:// quick-start, no ws://token/RemoteStore),
re-point CLAUDE.md's OVERVIEW links to ARCHITECTURE.md, and correct CLAUDE.md
to list five core-HA surfaces.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dated audit snapshots (2026-06-05) cross-checking every concrete name /
RPC / routing rule / table row in ARCHITECTURE.md and OVERVIEW.md against
the implementation. Kept as research artifacts under plans/research/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Record what shipped per phase (service-path + EntityQuery request/response),
what stays deferred (subscription/push primitive, todo, browse_media
media-source caveat), the deviations (search via async_internal_search_media,
JSON-safe sandbox response, callerless raise_not_proxied retained), and the
green verification summary lines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reflect the shipped request/response query RPCs across the docs: the
server-side query + WS-only mutation entity APIs now answer with real data
(service-path return_response + the generic entity_query RPC), so the
catalogue's status column, ARCHITECTURE §8/§14, OVERVIEW's "still open"
bullet, and the CLAUDE.md follow-up all move from "raises" to "wired". Kept
accurate as still-open: the subscription/push primitive (the */subscribe
one-shot-only rows + todo item-list push) and the media_player.browse_media
media-source caveat.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-trip rebuild tests for SearchMedia and Segment (the as_dict /
dataclass-asdict-vs-constructor asymmetry), per-op EntityQuery proxy tests
(media search, release notes, vacuum segments, calendar update/delete) that
assert the rebuilt typed object and the forwarded method + args, and the two
error paths: a sandbox-side ServiceValidationError translating to a
HomeAssistantError on main, and a closed channel degrading to a clean
HomeAssistantError. Client-side coverage for the EntityQuery handler:
method invocation + kwarg passing, unknown entity_id, unknown method, and a
raising method propagating its exception type on the error frame.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the remaining raise_not_proxied stubs with EntityQuery forwards +
typed rebuilds, so every query-shaped entity API now answers with real data:
- media_player.async_search_media -> async_internal_search_media (which
rebuilds the SearchMediaQuery from flat kwargs on the sandbox side, so the
query crosses as plain JSON); rebuilds SearchMedia, reusing the BrowseMedia
helper for its result list.
- update.async_release_notes -> async_release_notes (plain str/None).
- vacuum.async_get_segments -> async_get_segments; rebuilds list[Segment].
- calendar.async_update_event / async_delete_event -> the matching WS-only
entity methods (None result).
The sandbox-side serialisation is the as_dict-aware JSON encoder already
added with the handler, so SearchMedia/BrowseMedia/Segment cross verbatim.
raise_not_proxied is now callerless but kept exported for the still-deferred
subscription/todo-push primitive.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fire-and-forget call_service channel can command an entity but can't
ask it a server-side question that has no SupportsResponse service to ride.
Add one generic EntityQuery RPC for those, mirroring the call_service path
end to end (proto -> codec registry -> bridge sender + error translation ->
sandbox handler -> proxy helper):
- proto: EntityQuery {sandbox_entity_id, method, args, context_id} and
EntityQueryResult {result} (the return wrapped as {"value": ...} so
scalar/list/None are all representable). Gencode regenerated into both
_pb2 mirrors; drift guard passes.
- MSG_ENTITY_QUERY constant + REGISTRY entry added to both protocol/messages
mirrors.
- SandboxBridge.async_entity_query builds the request, remembers the context
before the id is reduced to a wire value, translates remote/closed errors
through the existing paths, and unwraps {"value": ...}.
- EntryRunner._handle_entity_query resolves the entity on the private hass,
invokes the named method with the decoded kwargs, and serialises the return
through the as_dict-aware JSON encoder; raised HA/voluptuous errors
propagate as channel error frames so main rebuilds the same shape.
- SandboxProxyEntity._entity_query is the proxy-side companion to
_call_service.
No proxy op is wired onto it yet — that is the next phase.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the three query-shaped entity APIs that have a SupportsResponse
service onto the existing call_service + return_response channel, so a
sandboxed entity answers them with real data instead of raising:
- calendar.async_get_events -> calendar.get_events service, rebuilding
list[CalendarEvent] from the response (explicit field mapping, ISO
date/datetime parse — not a **dict splat).
- weather.async_forecast_{daily,hourly,twice_daily} -> weather.get_forecasts
service; Forecast is a plain TypedDict, returned verbatim.
- media_player.async_browse_media -> media_player.browse_media service,
rebuilding the recursive BrowseMedia from its frontend-shaped as_dict.
SandboxProxyEntity._call_service grows a return_response flag that decodes
the CallServiceResult response into a dict. The sandbox-side call_service
handler now runs rich service responses (e.g. a BrowseMedia object keyed by
entity_id) through the as_dict-aware JSON encoder before packing the Struct,
yielding the exact wire shape main rebuilds from.
Caveat documented at the browse_media call site: a sandboxed player's browse
surfaces only its own sources; the media_source tree is empty inside the
sandbox (media_source runs on main). Round-trip rebuild unit tests cover the
as_dict-vs-constructor asymmetry first (plan Risk #2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the unproxied query/subscribe/WS-only entity APIs, their interim
raise behaviour, and the two missing primitives (request/response +
subscription RPC) in docs/query-shaped-rpcs.md. Add the implementation
plan (plan-query-rpc.md): a generic EntityQuery RPC for the service-less
ops + reuse of the existing call_service return_response path for ops
that have a SupportsResponse service. Note the media_player.browse_media
caveat (no media_source tree inside the sandbox). Cross-reference from
ARCHITECTURE/OVERVIEW/CLAUDE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The server-side query / subscribe / WS-only-mutation entity APIs the
fire-and-forget call_service bridge can't express (calendar listings +
event update/delete, weather forecasts, media browse/search, update
release notes, vacuum segments) previously returned empty/None silently.
Add entity.raise_not_proxied and have those proxy methods raise
HomeAssistantError instead, so the gap fails loudly until a real query
RPC lands.
todo is a special case: its To-do panel reads the sync todo_items
property that also feeds TodoListEntity.state, so it can't be a query at
all. Route it to main via SANDBOX_INCOMPATIBLE_PLATFORMS and drop the
proxy (matching the camera/image precedent).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both docs now describe the translation-forwarding subsystem in the body,
not just the goal: live pull (sandbox/get_translations RPC + provider
overlay) and the picker catalog hook.
- OVERVIEW: add a Translation forwarding section + "where to look" row +
v1-diff row. Fix pre-existing drift: ALWAYS_MAIN is 24 entries across
three groups (was listed as 6), failed-sandbox setup is SETUP_ERROR
(not SETUP_RETRY), and the manager runs no periodic ping loop.
- ARCHITECTURE: add §11 Translation forwarding (renumber following
sections), list translation.py/catalog.py in §2, and correct the core
touch surface from three to five hooks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Translation forwarding (live pull-RPC + catalog provider) now puts the
sandboxed integration's translations on main alongside its entities,
services, and events — note it in the OVERVIEW + ARCHITECTURE goals.
Remove the generated architecture.html; the architecture is published to
a gist instead of carrying a rendered artifact in the tree.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add sandbox/docs/catalog-provider-contract.md describing the display-only
picker catalog hook: the discoverability gap it closes, the
async_register_sandbox_catalog_provider API, and the contract — separate
from the sha-pinned source resolver, name load-bearing, title_translations
optional, no validation, display-only scope, and how it complements the
Phase B live RPC for the cold picker case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the A1 catalog hook into the two display paths a sandbox-only custom
falls through today:
- async_get_integration_descriptions (loader.py): append catalog
descriptors to the custom integration/helper buckets so the add-
integration picker lists them. On-disk customs carry richer metadata,
so the disk scan wins on a domain collision.
- _async_get_component_strings (helpers/translation.py): when a domain
has no on-disk Integration (IntegrationNotFound on main), take its
"title" from the catalog — a localized title_translations[lang] if
present, otherwise degrading to the descriptor name.
Tests: catalog entry appears in descriptions with picker name + defaults
+ helper-bucket routing; on-disk custom wins a collision; title fallback
uses title_translations and degrades to name when absent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a separate, display-only catalog hook so a custom integration whose
code lives only in a sandbox (never on main's disk) can be listed and
named in the add-integration picker without spawning a sandbox.
Core (homeassistant/loader.py) owns the registry because core consumes
it: SandboxIntegrationDescriptor, SandboxCatalogProvider,
DATA_SANDBOX_CATALOG_PROVIDERS, async_register_sandbox_catalog_provider,
async_get_sandbox_catalog. This mirrors the Phase B translation-provider
precedent (hook + consumer co-located in core).
homeassistant/components/sandbox/catalog.py re-exports the hook so HACS
registers through a sandbox namespace parallel to the source resolver —
but the catalog stays deliberately separate from the sha-pinned, security-
critical source resolver: it is eager, enumerable and cosmetic only.
Wired into descriptions + title fallback in A2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implement SandboxTranslationProvider and register it into core's translation
hook from async_setup (unregistered on stop). For each requested component it:
- resolves the owning sandbox group — a loaded entry's .sandbox field wins,
else the live SandboxFlowProxy of a brand-new custom's in-progress flow
(new sandbox_group accessor on the proxy);
- carves out built-ins (Integration.is_built_in ⇒ main reads its byte-identical
disk copy, never the wire);
- batches each group's custom domains into one get_translations RPC per
language (5s timeout), and degrades to empty strings on a down/closed/slow
channel so the cache-lock overlay never blocks the frontend.
router.async_unload_entry now invalidates a sandboxed entry's cached
translations, so a reload at a new integration-source ref re-pulls fresh
strings on the next fetch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a sandbox-agnostic seam to the translation cache, mirroring the
sandbox.sources source-resolver convention:
- async_register_sandbox_translation_provider(hass, provider): a HassKey-backed
registry with an unregister callback. The provider is awaited inside the
cache load and returns {language: {domain: raw_strings}} for only the domains
it owns.
- _TranslationCache._async_load overlays the provider result onto
translation_by_language_strings after async_get_integrations and before
_build_category_cache, so sandboxed strings flow through the same flatten /
English-fallback / loaded machinery as on-disk strings. A custom sandboxed
domain (IntegrationNotFound on main) thus stops resolving to {}.
- _TranslationCache.async_invalidate + async_invalidate_translations wrapper:
the first eviction API (translations were never unloaded), called by the
sandbox when a custom integration is re-fetched at a new ref.
Core never raises on a provider; degrade-to-empty is the provider's contract.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register a sandbox/get_translations handler in SandboxRuntime. It loads raw
translation strings for the requested domains from the sandbox's own
filesystem (built-in from the bundled package, custom from the fetched
<config>/custom_components/<domain>) by reusing core's
_async_get_component_strings against the sandbox-private hass — which also
pre-fills 'title' from integration.name. Main cannot run that fallback for a
custom domain because it holds no Integration, so the title must be injected
here. Replies with {language, strings: {domain: raw dict}}.
Tests cover built-in title pass-through, custom title injection, the empty
case, the Struct packing, and the no-flow-runner guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
