Files
Paulus SchoutsenandClaude Fable 5 81e9881211 sandbox: archive 2026-07-08 compat baseline + failure clustering
The honest full-tree compat baseline (all 1148 integration suites through
the real in-process lane) and its analysis, preserved under
sandbox/reports/2026-07-08/ so a re-run can't clobber it:

- COMPAT.csv / COMPAT_LATEST.md — baseline at 9c0d69d8f3 (60.5%
  test-level pass across the 763 engaged suites).
- COMPAT-postfix.csv / .md — re-sweep after the reauth/reconfigure fix
  (402e7987bd): 60.5% -> 62.4% pass, errors -887.
- clusters/ — failure clustering (clusters.md + clusters.json) over the
  --tb=line dumps.
- FINDINGS.md — the diagnosis: eight cross-cutting root-cause clusters
  ranked by leverage, with the fix each suggests. Cluster 1 (reauth/
  reconfigure) is the one already fixed; its writeback residual, the
  SETUP_RETRY gap (333 integrations), and the snapshot refresh are the
  next levers. runtime_data (50 integrations) is documented as an
  inherent white-box divergence, not a bug.

Tooling: sandbox/cluster_failures.py (reusable clusterer, reads
$SANDBOX_ERRORS_DIR), run_compat.py gains --jobs parallelism, --tb=line
+ wide columns for full failure reasons, and the tagged-vs-engaged
'main'/'no_op' bucket split. reports/README.md documents the convention;
the generated report archive is excluded from codespell/prettier.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCotUYum6AoisyrxshoiJJ
2026-07-08 03:17:13 -04:00
..

Home Assistant Sandbox

Runs Home Assistant integrations in isolated subprocesses while the main instance keeps a single, unified view of devices, entities, services, events, and translations — as if everything ran locally.

  • ARCHITECTURE.md — the full architecture: routing, lifecycle, config-flow forwarding, entity bridge, service/event mirror, store routing, translations, auth, shutdown, and the core-HA touch surface. Start here.
  • CLAUDE.md — orientation for working in this directory: repository layout, the core-HA files modified, open follow-ups, and how to run the tests.
  • docs/ — per-decision design write-ups (entity-bridge spike, state-sharing design, query-shaped RPCs, …).
  • status/ — per-phase / per-plan landing notes: the authoritative record of what each phase shipped, deferred, and flagged forward.

Layout

  • hass_client/ — the Python client library (its own uv env). Hosts SandboxRuntime, the flow / entity / service / event runners, the channel-backed store bridge, and the two pytest plugins. Also carries the runtime's Docker test image (see hass_client/docs/docker.md).
  • run_compat.py + COMPAT.md / BACKLOG.md — compat-lane runner and curated reports.

The HA Core side of the integration lives at ../homeassistant/components/sandbox/.

Quick start

cd sandbox/hass_client
uv sync
uv run pytest

# Run the runtime by hand against a local HA (debugging only — the manager
# normally spawns the subprocess for you, over stdio).
uv run python -m hass_client.sandbox --name built-in --url stdio://

The runtime holds no credential: it never opens a connection back to main and never acts on main's behalf (see ARCHITECTURE.md §10). In production the integration spawns the subprocess automatically once the first flow or entry routes to a given group; --url selects the transport (stdio:// default, unix://<path> opt-in; ws:// is reserved and not yet implemented).

Running HA Core's tests through the sandbox

# In-process plugin (fast, freezer-safe)
cd sandbox/hass_client
uv run python -m pytest -p hass_client.testing.pytest_plugin \
    ../../tests/components/input_boolean/test_init.py -v

# Real-subprocess plugin (pins the subprocess boundary)
uv run python -m pytest -p hass_client.testing.conftest_sandbox \
    ../../tests/components/input_boolean/test_init.py -v

# Or drive the compat lane runner
cd sandbox
python run_compat.py input_boolean light switch

COMPAT.md is the curated compat-lane report; per-failure output lands in ${SANDBOX_ERRORS_DIR:-/tmp/sandbox_errors}.

Status

Phases 020 plus the boundary-hardening closing batch have landed: the concurrent channel dispatcher, all 31 domain proxies, schema / unique_id / unload-hook marshalling, the ConfigEntry.sandbox first-class field, device-registry bridging, the protobuf wire with pluggable transports, stateless sha-pinned integration sources, translation forwarding, and the request/response query RPCs. The ARCHITECTURE.md changelog summarises the closing batch; docs/FOLLOWUPS.md tells the narrative; the status/ landing notes are the authoritative per-phase record.

What's still open is tracked in ARCHITECTURE.md §14 (and CLAUDE.md): the state-sharing subscription consumer, query-shaped subscriptions/push, cross-sandbox in-process dependencies, and pip/egress validation for containerised custom-integration dependencies.