Files
openclaude/scripts/externals.ts
T
2edec9a140 fix(deps): ship a zero-warning, minimal install (#1784)
* fix(deps): ship a zero-warning, minimal install

The published package declared 62 runtime `dependencies`, but `dist/cli.mjs`
is a fully-bundled esbuild output that inlines almost all of them. End users
therefore installed ~476 transitive packages — including three subtrees the
bundle never needs at install time, each emitting an install warning:

  - node-domexception (deprecated) via google-auth-library
  - protobufjs (allow-scripts)     via @grpc/* (already bundled into dist)
  - sharp (allow-scripts)          native image module

The repo's `overrides`/`allowScripts` silence these locally, but those are
root-only npm settings and are ignored when the package is installed as a
dependency — so end users saw the warnings.

Core changes:
  - package.json: runtime dependencies trimmed 62 -> 3 (@orama/orama,
    @orama/plugin-data-persistence, @vscode/ripgrep). Bundled packages, plus
    the optional sharp/google-auth-library, move to devDependencies so they
    are built/tested but not shipped.
  - package.json: @anthropic-ai/sdk, @modelcontextprotocol/sdk, react and
    react-reconciler declared as OPTIONAL peerDependencies — externalized by
    the ./sdk bundle but bundled into the CLI. Optional peers keep the CLI
    install minimal and warning-free while still resolving for ./sdk consumers.
  - externals.ts: sharp, google-auth-library and @anthropic-ai/bedrock-sdk
    marked OPTIONAL_RUNTIME_EXTERNALS (loaded on demand, not shipped).
  - validate-externals.ts: runtime deps validate against externals; bundled
    deps validate against dependencies + devDependencies.
  - client.ts: load @anthropic-ai/bedrock-sdk via the runtime importer so
    esbuild no longer inlines it and hoists its static @aws-sdk import into
    the CLI bundle (that was a startup crash for default installs).

Optional-dependency UX (consistent, actionable errors):
  - New src/utils/optionalRuntimeModule.ts exports importRuntimeModule and
    importOptionalRuntimeModule. The optional variant translates a missing
    package (code === 'ERR_MODULE_NOT_FOUND', specifier present in message)
    into "<feature> requires "<pkg>" ... Run `npm i -g <pkg>`". Generic so
    typed call sites keep their module types.
  - Routed ALL optional-package load sites through it (previously only one
    did): google-auth-library (client.ts, auth.ts, geminiAuth.ts),
    @anthropic-ai/foundry-sdk + @azure/identity (client.ts), and the
    @aws-sdk/* Bedrock paths (model/bedrock.ts, tokenEstimation.ts, aws.ts).
  - imageProcessor.ts: sharp-missing error now says `npm i -g sharp`.
  - docs/advanced-setup.md: new "Optional provider packages" table and a
    Vertex note documenting the on-demand installs.
  - Unit test for the helper (friendly error, success path, specifier match,
    raw passthrough).
  - knip.json: ignore google-auth-library (now loaded via runtime string).

Verified on the current tree:
  - tsc, build/validate-externals, knip, and tests all pass.
  - npm pack + install --omit=dev adds 8 packages, zero deprecation/
    allow-scripts/funding warnings; --version/--help/mcp list run.
  - With packages absent, CLAUDE_CODE_USE_BEDROCK and CLAUDE_CODE_USE_VERTEX
    print the friendly `npm i -g <pkg>` error (verified end-to-end).
  - ./sdk imports once its optional peers are present (24 exports, no warns).
  - Bundled ajv + ajv-formats validate with no ajv installed; no unguarded
    native runtime requires (fsevents absent in chokidar 4; bun:sqlite Bun-only).

Trade-off: image reads, AWS Bedrock, Azure Foundry and GCP/Vertex now prompt
a one-time `npm i -g <pkg>` instead of being shipped to every user.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

Review fixes (CodeRabbit + jatmn):
- validate-externals: the INTENTIONALLY_BUNDLED exemption is now scoped per
  bundle. The CLI exempts every bundled package; the SDK does NOT exempt
  packages declared as peerDependencies (keyed on package.json, an independent
  source of truth) so dropping react/@anthropic-ai/sdk from SDK_EXTERNALS now
  fails validation instead of silently passing. Added an explicit minimal-
  install contract check: bundled packages must be devDependencies-only — never
  in `dependencies`, and only the SDK-external subset may be optional peers.
  Validation logic extracted to scripts/externalsValidation.ts + tests.
- FileReadTool oversized-image fallback now loads via the shared
  getImageProcessor() (not a raw import('sharp')) and re-throws
  ImageProcessorUnavailableError, so a missing processor surfaces the
  `npm i -g sharp` install hint instead of returning an over-budget image.
- optionalRuntimeModule: match the missing specifier as a QUOTED token, not a
  raw substring, so a missing transitive package whose name contains the
  requested one (sharp vs sharp-libvips, @aws-sdk/client-bedrock vs
  @aws-sdk/client-bedrock-runtime) no longer triggers the wrong install hint.
  Predicate extracted to isMissingSpecifierError() with regression tests.
- docs/advanced-setup.md: the Vertex auth section now shows both documented
  paths (gcloud ADC and a GOOGLE_APPLICATION_CREDENTIALS service-account file).

Review fixes (round 2, CodeRabbit):
- validate-externals: assert the optional-peer install contract — every
  peerDependency must be { optional: true } in peerDependenciesMeta
  (validateOptionalPeers), so losing that flag fails the build instead of
  silently reintroducing install warnings.
- validate-externals: hard-check OPTIONAL_RUNTIME_EXTERNALS placement
  (validateOptionalRuntimeexternals). Anything esbuild can see statically must
  stay external in BOTH bundles (dropping sharp/google-auth-library now fails);
  the runtime-indirection-only subset (new RUNTIME_INDIRECTION_ONLY_EXTERNALS)
  must stay OUT of externals so esbuild never re-exposes their static imports.
- Deeper-dig fix: @anthropic-ai/foundry-sdk was misclassified as
  INTENTIONALLY_BUNDLED, but it is loaded only through the Function indirection
  (esbuild never sees it, so it was never actually bundled) — its sole presence
  in dist is the specifier string. Per the PR's own "Azure Foundry now prompts"
  trade-off it is on-demand, so it now lives in OPTIONAL_RUNTIME_EXTERNALS +
  RUNTIME_INDIRECTION_ONLY_EXTERNALS (mirroring bedrock-sdk). sandbox-runtime is
  genuinely statically imported, so it stays bundled.
- Provider-routing coverage (scripts/optionalRuntimeSpecifiers.test.ts): a
  static scan asserts every importOptionalRuntimeModule specifier is a declared
  OPTIONAL_RUNTIME_EXTERNAL and never also INTENTIONALLY_BUNDLED — the
  invariant that keeps a provider's optional package loadable on demand.
- All new validators extracted to scripts/externalsValidation.ts with tests.

Review fixes (round 3, CodeRabbit):
- client.ts: gate the Vertex google-auth-library import behind the non-skip
  branch. CLAUDE_CODE_SKIP_VERTEX_AUTH (proxy/test) uses a mock GoogleAuth and
  must not require the optional package; it was loaded unconditionally before.
- optionalRuntimeModule: drop the hard-coded `npm i -g`. The helper backs both
  the global CLI and project-local ./sdk consumers, so the hint is now
  context-neutral ("npm install <pkg>" / add -g for the global CLI).
- validate-externals: every SDK_ONLY_EXTERNALS entry must STAY a
  peerDependency (a dropped peer leaves runtimeDeps while the SDK still
  externalizes it); and OPTIONAL_RUNTIME_EXTERNALS must never be shipped (fail
  on overlap with dependencies/peerDependencies). Both with tests + live-verified.
- optionalRuntimeSpecifiers.test: pin the EXACT set of optionally-loaded
  specifiers instead of a >=5 count (a count passes even if a provider path
  regresses).
- attachments: extract tryReadEditedImageAttachment() — background watched-file
  image attachments DEGRADE to null on any failure (incl.
  ImageProcessorUnavailableError) so a missing optional package never aborts a
  turn, while the explicit FileReadTool path still surfaces the install hint.
  Deterministic regression test (bad path -> null).
- docs: Bedrock row notes profile-based auth also needs
  @aws-sdk/credential-providers; install-hint wording matches the new message.

Review fixes (round 4, CodeRabbit):
- attachments: stop sending the raw file path through the analytics
  bypass-cast (tengu_watched_file_compression_failed). Send only the safe
  file extension via getFileExtensionForAnalytics, matching the existing
  tengu_file_read_dedup pattern, so no usernames/project paths can leak.
- externals.ts: corrected the OPTIONAL_RUNTIME_EXTERNALS header comment,
  which still claimed all entries "remain in COMMON_EXTERNALS" — no longer
  true since the indirection-only subset (bedrock/foundry) must stay OUT of
  the externals lists.

(Other CodeRabbit comments on this push re-surface items already addressed in
prior commits: the peerDependenciesMeta-optional check (validateOptionalPeers),
the SDK-peers-present and optional-not-shipped validator rules, the
exact-specifier-set test, the attachments degrade contract + test, and the
context-neutral install hint are all present. The "assert every optional
external is a devDependency" suggestion is intentionally NOT applied: @aws-sdk/*
and @azure/identity are transitive devDeps via bedrock-sdk/foundry-sdk, so a
blanket assertion would be incorrect; source resolution is covered by the
build + tests that import these packages.)

Review fixes (round 5, CodeRabbit):
- attachments: stop leaking file paths via logError in the background-image
  degrade path. readImageWithTokenBudget can throw path-bearing messages
  (e.g. "Image file is empty: <path>") and logError persists message/stack, so
  log only the error TYPE name now. (Analytics payload was already sanitized.)
- attachments: tryReadEditedImageAttachment takes an injectable reader so the
  degrade contract is tested for the EXACT error types — ImageProcessorUnavailableError
  and a path-bearing read error both degrade to null (not just ENOENT) — plus a
  success case. No mocking.
- validate-externals: enforce the source-install half of the optional contract.
  Non-transitive OPTIONAL_RUNTIME_EXTERNALS must be devDependencies so `bun
  install` source builds resolve them. The new TRANSITIVE_OPTIONAL_EXTERNALS
  documents the exemption (@aws-sdk/* via @anthropic-ai/bedrock-sdk, @azure/identity
  via @anthropic-ai/foundry-sdk — provided transitively, not direct devDeps). A
  blanket "all optionals are devDeps" check would have wrongly failed on those.
  Tests + live-verified (dropping sharp from devDependencies now fails).

Review fixes (round 6, CodeRabbit + jatmn):
- optionalRuntimeSpecifiers.test: the call-site scan regex missed
  generic-annotated calls (importOptionalRuntimeModule<...>(...)) in
  model/bedrock.ts and tokenEstimation.ts, so the exact-set assertion was
  incomplete. Regex now allows an optional generic; EXPECTED_SPECIFIERS adds
  @aws-sdk/client-bedrock and @aws-sdk/client-bedrock-runtime (7 total).
- importOptionalRuntimeModule default generic is now <T = unknown> (was any),
  so destructured imports are no longer silently any. Every call site now
  supplies its module type — typeof import('<pkg>') where the package is
  type-resolvable (bedrock-sdk, foundry-sdk, @aws-sdk/credential-providers,
  google-auth-library), and a named minimal-shape alias for @azure/identity
  (not a direct devDep, so typeof import can't resolve it). This gives
  compile-time verification of each provider's module contract (export names,
  shapes) — the structural answer to the "cover the provider branches" ask.
- attachments: tryReadEditedImageAttachment takes injectable {read,log,track};
  a new test asserts the sanitized-telemetry contract directly — the logError
  payload is path-free and the analytics payload carries only `ext`, never the
  edited-image path.

* fix(deps): address optional runtime review findings

* test(deps): isolate optional runtime importer mocks

* fix(deps): clarify AWS optional auth labels

* fix(deps): close optional runtime review gaps

---------

Co-authored-by: jatmn <the@jat.mn>
2026-07-07 13:19:39 +08:00

202 lines
7.7 KiB
TypeScript

/**
* Shared external dependency lists for CLI and SDK bundles.
*
* Used by build.ts and validate-externals.ts.
* When adding a new dependency to package.json, check if it should be
* added here (large packages, native modules, or packages with many exports).
*/
// Packages that should be kept external in ALL bundles (CLI + SDK).
// NOTE: some entries here are ALSO in OPTIONAL_RUNTIME_EXTERNALS below
// (sharp, google-auth-library, @aws-sdk/*, @azure/identity). That overlap is
// intentional: membership here means "never inline into the bundle", while
// membership in OPTIONAL_RUNTIME_EXTERNALS additionally means "not shipped in
// the default install — loaded on demand". A package can be both.
export const COMMON_EXTERNALS: string[] = [
// Native image processing
'sharp',
// Cloud provider SDKs
'@aws-sdk/client-bedrock',
'@aws-sdk/client-bedrock-runtime',
'@aws-sdk/client-sts',
'@aws-sdk/credential-provider-node',
'@aws-sdk/credential-providers',
'@smithy/core',
'@smithy/node-http-handler',
'@azure/identity',
'google-auth-library',
// @vscode/ripgrep ships a platform-specific binary alongside its
// index.js and resolves the path via __dirname at runtime. Bundling
// would freeze the build host's absolute path into dist/cli.mjs, so we
// keep it external and rely on the npm package being installed.
'@vscode/ripgrep',
// Orama search engine
'@orama/orama',
'@orama/plugin-data-persistence',
// web-tree-sitter ships a WASM file alongside its JS and resolves the
// path via require.resolve at runtime; bundling would freeze the build
// host's absolute path, so keep it external.
'web-tree-sitter',
// tree-sitter-wasms ships per-language .wasm files resolved via
// require.resolve at runtime — same bundling concern as web-tree-sitter.
'tree-sitter-wasms',
]
// Additional packages external only in the SDK bundle (TUI + heavy deps)
export const SDK_ONLY_EXTERNALS: string[] = [
'react',
'react-reconciler',
'@anthropic-ai/sdk',
'@modelcontextprotocol/sdk',
]
// Optional runtime packages: dynamically imported only when a provider/feature
// needs them, and NOT listed in package.json `dependencies`, so a default
// `npm install -g @gitlawb/openclaude` stays small and warning-free.
//
// Two shapes (see RUNTIME_INDIRECTION_ONLY_EXTERNALS below):
// - Most stay external in both bundles (in COMMON_EXTERNALS) so esbuild never
// inlines them — they ARE referenced where esbuild can see them.
// - The indirection-only subset (@anthropic-ai/{bedrock,foundry}-sdk) is the
// opposite: loaded purely via the runtime importer, so esbuild never sees a
// static reference and they must stay OUT of the externals lists.
export const OPTIONAL_RUNTIME_EXTERNALS: string[] = [
// Cloud provider SDKs (dynamically imported per-provider)
'@aws-sdk/client-bedrock',
'@aws-sdk/client-bedrock-runtime',
'@aws-sdk/client-sts',
'@aws-sdk/credential-provider-node',
'@aws-sdk/credential-providers',
'@smithy/core',
'@smithy/node-http-handler',
'@azure/identity',
// Anthropic Bedrock client — loaded via the runtime importer in
// services/api/client.ts. Not bundled (it statically imports @aws-sdk) and
// not shipped; Bedrock users install it on demand (it pulls @aws-sdk itself).
'@anthropic-ai/bedrock-sdk',
// Anthropic Foundry client — also loaded only via the runtime importer in
// services/api/client.ts (CLAUDE_CODE_USE_FOUNDRY). The Function indirection
// means esbuild never sees it, so it is not bundled; Foundry users install it
// on demand. (It is NOT in COMMON_EXTERNALS for the same reason as bedrock.)
'@anthropic-ai/foundry-sdk',
// GCP/Vertex auth — loaded via runtime import in services/api/client.ts.
// Optional: only Vertex users need it. Its transitive tree (gaxios →
// node-fetch → fetch-blob → node-domexception) is what triggered the
// deprecation warning on install, so we no longer ship it by default.
'google-auth-library',
// Native image processing — loaded via dynamic import in the image tools.
// Optional: only image reads need it, and it carries a native install
// script. Kept opt-in so default installs run no install scripts.
'sharp',
]
// OPTIONAL_RUNTIME_EXTERNALS that are loaded ONLY through the runtime importer
// (the `new Function` indirection in src/utils/optionalRuntimeModule.ts), so
// esbuild never sees a static reference to them. These must NOT appear in the
// externals lists: marking @anthropic-ai/bedrock-sdk external would let esbuild
// keep (and at startup evaluate) its static `@aws-sdk/client-bedrock-runtime`
// import, which is exactly the default-install crash this design avoids. Every
// OTHER optional external IS referenced somewhere esbuild can see (e.g. sharp's
// dynamic import in imageProcessor.ts) and therefore must stay external.
export const RUNTIME_INDIRECTION_ONLY_EXTERNALS: string[] = [
'@anthropic-ai/bedrock-sdk',
'@anthropic-ai/foundry-sdk',
]
// OPTIONAL_RUNTIME_EXTERNALS that are NOT direct devDependencies because they
// are pulled transitively by another optional package's dependency tree, so
// source builds/tests still resolve them. Every OTHER optional external must be
// a direct devDependency (validated) so `bun install` source/dev builds keep
// working.
export const TRANSITIVE_OPTIONAL_EXTERNALS: string[] = [
'@aws-sdk/client-bedrock-runtime',
'@aws-sdk/credential-providers',
]
// Computed full lists
export const CLI_EXTERNALS: string[] = COMMON_EXTERNALS
export const SDK_EXTERNALS: string[] = [...COMMON_EXTERNALS, ...SDK_ONLY_EXTERNALS]
// Packages intentionally bundled (not external, not flagged by validation)
// These are small utilities that are fine to inline into the output bundle.
export const INTENTIONALLY_BUNDLED: string[] = [
// Anthropic provider variants (bundled, not the main SDK).
// NOTE: @anthropic-ai/bedrock-sdk AND @anthropic-ai/foundry-sdk are
// intentionally NOT bundled — they are loaded only via the runtime importer in
// services/api/client.ts (esbuild never sees the specifier), so they live in
// OPTIONAL_RUNTIME_EXTERNALS / RUNTIME_INDIRECTION_ONLY_EXTERNALS and Bedrock /
// Foundry users install them on demand. @anthropic-ai/sandbox-runtime IS
// statically imported (utils/sandbox/sandbox-adapter.ts), so esbuild bundles it.
'@anthropic-ai/sandbox-runtime',
// CLI / TUI utilities
'@alcalzone/ansi-tokenize',
'@commander-js/extra-typings',
'bidi-js',
'chalk',
'cli-boxes',
'cli-highlight',
'commander',
'emoji-regex',
'env-paths',
'figures',
'get-east-asian-width',
'indent-string',
'supports-hyperlinks',
'wrap-ansi',
// Data formats
'jsonc-parser',
'yaml',
'marked',
'turndown',
'xss',
// Data utilities
'ajv',
'auto-bind',
'diff',
'fflate',
'fuse.js',
'ignore',
'lodash-es',
'lru-cache',
'p-map',
'picomatch',
'proper-lockfile',
'qrcode',
'semver',
'shell-quote',
'signal-exit',
'type-fest',
// Networking
'axios',
'cross-spawn',
'duck-duck-scrape',
'execa',
'https-proxy-agent',
'tree-kill',
'undici',
'ws',
// React ecosystem (react/react-reconciler are SDK_ONLY_EXTERNALS, bundled in CLI)
'react',
'react-compiler-runtime',
'react-reconciler',
'usehooks-ts',
// Anthropic SDK (external in SDK bundle, bundled in CLI)
'@anthropic-ai/sdk',
// MCP SDK (external in SDK bundle, bundled in CLI)
'@modelcontextprotocol/sdk',
// Schema validation
'zod',
// gRPC (bundled into CLI, not external)
'@grpc/grpc-js',
'@grpc/proto-loader',
// Language server protocol
'vscode-languageserver-protocol',
// File watching
'chokidar',
// Graph algorithms (repo map PageRank)
'graphology',
'graphology-metrics',
// Tokenizer for repo map token budgeting
'js-tiktoken',
]