11 Commits
Author SHA1 Message Date
anushka ✩ 294bd9a1df Add optional Sentry error reporting (env-driven, opt-in) (#2139)
* Add optional Sentry error reporting (env-driven, opt-in)

* Fix Sentry init to use dynamic import instead of require (ESM compatibility)

* Document SENTRY_DSN setup in advanced-setup docs

* Disable Sentry default integrations; document runtime install requirement

* Wire reportErrorToSentry into top-level error handlers; add sentry.test.ts
2026-08-19 13:51:37 +08:00
Kevin Codexandjatmn 2edec9a140 fix(deps): ship a zero-warning, minimal install (#1784)
* fix(deps): ship a zero-warning, minimal install

The published package declared 62 runtime `dependencies`, but `dist/cli.mjs`
is a fully-bundled esbuild output that inlines almost all of them. End users
therefore installed ~476 transitive packages — including three subtrees the
bundle never needs at install time, each emitting an install warning:

  - node-domexception (deprecated) via google-auth-library
  - protobufjs (allow-scripts)     via @grpc/* (already bundled into dist)
  - sharp (allow-scripts)          native image module

The repo's `overrides`/`allowScripts` silence these locally, but those are
root-only npm settings and are ignored when the package is installed as a
dependency — so end users saw the warnings.

Core changes:
  - package.json: runtime dependencies trimmed 62 -> 3 (@orama/orama,
    @orama/plugin-data-persistence, @vscode/ripgrep). Bundled packages, plus
    the optional sharp/google-auth-library, move to devDependencies so they
    are built/tested but not shipped.
  - package.json: @anthropic-ai/sdk, @modelcontextprotocol/sdk, react and
    react-reconciler declared as OPTIONAL peerDependencies — externalized by
    the ./sdk bundle but bundled into the CLI. Optional peers keep the CLI
    install minimal and warning-free while still resolving for ./sdk consumers.
  - externals.ts: sharp, google-auth-library and @anthropic-ai/bedrock-sdk
    marked OPTIONAL_RUNTIME_EXTERNALS (loaded on demand, not shipped).
  - validate-externals.ts: runtime deps validate against externals; bundled
    deps validate against dependencies + devDependencies.
  - client.ts: load @anthropic-ai/bedrock-sdk via the runtime importer so
    esbuild no longer inlines it and hoists its static @aws-sdk import into
    the CLI bundle (that was a startup crash for default installs).

Optional-dependency UX (consistent, actionable errors):
  - New src/utils/optionalRuntimeModule.ts exports importRuntimeModule and
    importOptionalRuntimeModule. The optional variant translates a missing
    package (code === 'ERR_MODULE_NOT_FOUND', specifier present in message)
    into "<feature> requires "<pkg>" ... Run `npm i -g <pkg>`". Generic so
    typed call sites keep their module types.
  - Routed ALL optional-package load sites through it (previously only one
    did): google-auth-library (client.ts, auth.ts, geminiAuth.ts),
    @anthropic-ai/foundry-sdk + @azure/identity (client.ts), and the
    @aws-sdk/* Bedrock paths (model/bedrock.ts, tokenEstimation.ts, aws.ts).
  - imageProcessor.ts: sharp-missing error now says `npm i -g sharp`.
  - docs/advanced-setup.md: new "Optional provider packages" table and a
    Vertex note documenting the on-demand installs.
  - Unit test for the helper (friendly error, success path, specifier match,
    raw passthrough).
  - knip.json: ignore google-auth-library (now loaded via runtime string).

Verified on the current tree:
  - tsc, build/validate-externals, knip, and tests all pass.
  - npm pack + install --omit=dev adds 8 packages, zero deprecation/
    allow-scripts/funding warnings; --version/--help/mcp list run.
  - With packages absent, CLAUDE_CODE_USE_BEDROCK and CLAUDE_CODE_USE_VERTEX
    print the friendly `npm i -g <pkg>` error (verified end-to-end).
  - ./sdk imports once its optional peers are present (24 exports, no warns).
  - Bundled ajv + ajv-formats validate with no ajv installed; no unguarded
    native runtime requires (fsevents absent in chokidar 4; bun:sqlite Bun-only).

Trade-off: image reads, AWS Bedrock, Azure Foundry and GCP/Vertex now prompt
a one-time `npm i -g <pkg>` instead of being shipped to every user.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

Review fixes (CodeRabbit + jatmn):
- validate-externals: the INTENTIONALLY_BUNDLED exemption is now scoped per
  bundle. The CLI exempts every bundled package; the SDK does NOT exempt
  packages declared as peerDependencies (keyed on package.json, an independent
  source of truth) so dropping react/@anthropic-ai/sdk from SDK_EXTERNALS now
  fails validation instead of silently passing. Added an explicit minimal-
  install contract check: bundled packages must be devDependencies-only — never
  in `dependencies`, and only the SDK-external subset may be optional peers.
  Validation logic extracted to scripts/externalsValidation.ts + tests.
- FileReadTool oversized-image fallback now loads via the shared
  getImageProcessor() (not a raw import('sharp')) and re-throws
  ImageProcessorUnavailableError, so a missing processor surfaces the
  `npm i -g sharp` install hint instead of returning an over-budget image.
- optionalRuntimeModule: match the missing specifier as a QUOTED token, not a
  raw substring, so a missing transitive package whose name contains the
  requested one (sharp vs sharp-libvips, @aws-sdk/client-bedrock vs
  @aws-sdk/client-bedrock-runtime) no longer triggers the wrong install hint.
  Predicate extracted to isMissingSpecifierError() with regression tests.
- docs/advanced-setup.md: the Vertex auth section now shows both documented
  paths (gcloud ADC and a GOOGLE_APPLICATION_CREDENTIALS service-account file).

Review fixes (round 2, CodeRabbit):
- validate-externals: assert the optional-peer install contract — every
  peerDependency must be { optional: true } in peerDependenciesMeta
  (validateOptionalPeers), so losing that flag fails the build instead of
  silently reintroducing install warnings.
- validate-externals: hard-check OPTIONAL_RUNTIME_EXTERNALS placement
  (validateOptionalRuntimeexternals). Anything esbuild can see statically must
  stay external in BOTH bundles (dropping sharp/google-auth-library now fails);
  the runtime-indirection-only subset (new RUNTIME_INDIRECTION_ONLY_EXTERNALS)
  must stay OUT of externals so esbuild never re-exposes their static imports.
- Deeper-dig fix: @anthropic-ai/foundry-sdk was misclassified as
  INTENTIONALLY_BUNDLED, but it is loaded only through the Function indirection
  (esbuild never sees it, so it was never actually bundled) — its sole presence
  in dist is the specifier string. Per the PR's own "Azure Foundry now prompts"
  trade-off it is on-demand, so it now lives in OPTIONAL_RUNTIME_EXTERNALS +
  RUNTIME_INDIRECTION_ONLY_EXTERNALS (mirroring bedrock-sdk). sandbox-runtime is
  genuinely statically imported, so it stays bundled.
- Provider-routing coverage (scripts/optionalRuntimeSpecifiers.test.ts): a
  static scan asserts every importOptionalRuntimeModule specifier is a declared
  OPTIONAL_RUNTIME_EXTERNAL and never also INTENTIONALLY_BUNDLED — the
  invariant that keeps a provider's optional package loadable on demand.
- All new validators extracted to scripts/externalsValidation.ts with tests.

Review fixes (round 3, CodeRabbit):
- client.ts: gate the Vertex google-auth-library import behind the non-skip
  branch. CLAUDE_CODE_SKIP_VERTEX_AUTH (proxy/test) uses a mock GoogleAuth and
  must not require the optional package; it was loaded unconditionally before.
- optionalRuntimeModule: drop the hard-coded `npm i -g`. The helper backs both
  the global CLI and project-local ./sdk consumers, so the hint is now
  context-neutral ("npm install <pkg>" / add -g for the global CLI).
- validate-externals: every SDK_ONLY_EXTERNALS entry must STAY a
  peerDependency (a dropped peer leaves runtimeDeps while the SDK still
  externalizes it); and OPTIONAL_RUNTIME_EXTERNALS must never be shipped (fail
  on overlap with dependencies/peerDependencies). Both with tests + live-verified.
- optionalRuntimeSpecifiers.test: pin the EXACT set of optionally-loaded
  specifiers instead of a >=5 count (a count passes even if a provider path
  regresses).
- attachments: extract tryReadEditedImageAttachment() — background watched-file
  image attachments DEGRADE to null on any failure (incl.
  ImageProcessorUnavailableError) so a missing optional package never aborts a
  turn, while the explicit FileReadTool path still surfaces the install hint.
  Deterministic regression test (bad path -> null).
- docs: Bedrock row notes profile-based auth also needs
  @aws-sdk/credential-providers; install-hint wording matches the new message.

Review fixes (round 4, CodeRabbit):
- attachments: stop sending the raw file path through the analytics
  bypass-cast (tengu_watched_file_compression_failed). Send only the safe
  file extension via getFileExtensionForAnalytics, matching the existing
  tengu_file_read_dedup pattern, so no usernames/project paths can leak.
- externals.ts: corrected the OPTIONAL_RUNTIME_EXTERNALS header comment,
  which still claimed all entries "remain in COMMON_EXTERNALS" — no longer
  true since the indirection-only subset (bedrock/foundry) must stay OUT of
  the externals lists.

(Other CodeRabbit comments on this push re-surface items already addressed in
prior commits: the peerDependenciesMeta-optional check (validateOptionalPeers),
the SDK-peers-present and optional-not-shipped validator rules, the
exact-specifier-set test, the attachments degrade contract + test, and the
context-neutral install hint are all present. The "assert every optional
external is a devDependency" suggestion is intentionally NOT applied: @aws-sdk/*
and @azure/identity are transitive devDeps via bedrock-sdk/foundry-sdk, so a
blanket assertion would be incorrect; source resolution is covered by the
build + tests that import these packages.)

Review fixes (round 5, CodeRabbit):
- attachments: stop leaking file paths via logError in the background-image
  degrade path. readImageWithTokenBudget can throw path-bearing messages
  (e.g. "Image file is empty: <path>") and logError persists message/stack, so
  log only the error TYPE name now. (Analytics payload was already sanitized.)
- attachments: tryReadEditedImageAttachment takes an injectable reader so the
  degrade contract is tested for the EXACT error types — ImageProcessorUnavailableError
  and a path-bearing read error both degrade to null (not just ENOENT) — plus a
  success case. No mocking.
- validate-externals: enforce the source-install half of the optional contract.
  Non-transitive OPTIONAL_RUNTIME_EXTERNALS must be devDependencies so `bun
  install` source builds resolve them. The new TRANSITIVE_OPTIONAL_EXTERNALS
  documents the exemption (@aws-sdk/* via @anthropic-ai/bedrock-sdk, @azure/identity
  via @anthropic-ai/foundry-sdk — provided transitively, not direct devDeps). A
  blanket "all optionals are devDeps" check would have wrongly failed on those.
  Tests + live-verified (dropping sharp from devDependencies now fails).

Review fixes (round 6, CodeRabbit + jatmn):
- optionalRuntimeSpecifiers.test: the call-site scan regex missed
  generic-annotated calls (importOptionalRuntimeModule<...>(...)) in
  model/bedrock.ts and tokenEstimation.ts, so the exact-set assertion was
  incomplete. Regex now allows an optional generic; EXPECTED_SPECIFIERS adds
  @aws-sdk/client-bedrock and @aws-sdk/client-bedrock-runtime (7 total).
- importOptionalRuntimeModule default generic is now <T = unknown> (was any),
  so destructured imports are no longer silently any. Every call site now
  supplies its module type — typeof import('<pkg>') where the package is
  type-resolvable (bedrock-sdk, foundry-sdk, @aws-sdk/credential-providers,
  google-auth-library), and a named minimal-shape alias for @azure/identity
  (not a direct devDep, so typeof import can't resolve it). This gives
  compile-time verification of each provider's module contract (export names,
  shapes) — the structural answer to the "cover the provider branches" ask.
- attachments: tryReadEditedImageAttachment takes injectable {read,log,track};
  a new test asserts the sanitized-telemetry contract directly — the logError
  payload is path-free and the analytics payload carries only `ext`, never the
  edited-image path.

* fix(deps): address optional runtime review findings

* test(deps): isolate optional runtime importer mocks

* fix(deps): clarify AWS optional auth labels

* fix(deps): close optional runtime review gaps

---------

Co-authored-by: jatmn <the@jat.mn>
2026-07-07 13:19:39 +08:00
JATMNandgnanam1990 fb40d49e68 feat: add repo map codebase intelligence (#1867)
* feat: add Codebase Intelligence — repo map with PageRank-ranked structural summaries

Adds a new module that builds a structural map of the repository by parsing
source files with tree-sitter, building a cross-file reference graph weighted
by IDF, ranking files with PageRank, and rendering a token-budgeted summary
of the most important files and their signatures.

Surface:
- RepoMap tool the model can call on-demand, with focus_files / focus_symbols
- /repomap slash command with --tokens, --focus, --stats, --invalidate
- Auto-injection into session system context, gated by REPO_MAP=1 env var
  (compile-time feature('REPO_MAP') flag stays off in scripts/build.ts)

How it works:
  git ls-files → tree-sitter WASM parse → extract defs/refs →
  IDF-weighted directed graph → PageRank → render top files until token budget

Files imported by many others rank highest. Common symbol names (get, set,
map, value) are down-weighted via IDF. Results cached to disk keyed by
(path, mtime, size) — only changed files are re-parsed.

Supported languages: TypeScript, JavaScript, Python.

Tree-sitter tag queries are inlined as string constants in queries.ts so
they ship inside dist/cli.mjs and work after npm install — the .scm source
files are kept for readability/Aider attribution but are not required at
runtime. A drift-guard test (queries.test.ts) asserts byte-equality between
the inlined strings and the .scm source files.

Dependencies added: web-tree-sitter, tree-sitter-wasms, graphology,
graphology-pagerank, graphology-operators, js-tiktoken.

* fix(repomap): invalidate rendered cache on file edits + Windows test fix

- computeMapHash now folds per-file mtime+size into the cache key so a
  source edit (without changing the file list) no longer returns the
  prior rendered map. Adds a regression test that edits a file and
  confirms the second build reflects the new symbol without manual
  invalidateCache().
- queries.test.ts byte-for-byte drift guard normalizes CRLF -> LF when
  reading the .scm source so Windows checkouts pass. .gitattributes
  also pins *.scm to LF on future checkouts.
- Externals: declare web-tree-sitter, tree-sitter-wasms, graphology*,
  and js-tiktoken in scripts/externals.ts so build validation passes.

* fix(repomap): expand directory focus paths

* fix(repomap): satisfy deadcode check

* Fix repo map review findings

* Resolve remaining repo map review findings

* fix(repomap): address review findings

* fix(repomap): address review findings

* fix(repomap): resolve smoke and review follow-ups

* fix(repomap): preserve cached tag order

* fix(repomap): resolve review follow-ups

* fix(repomap): satisfy query promise lint

* Fix repo map context timeout cleanup

* fix: address repo map review findings

* fix: cancel timed-out repo map context builds

* fix(repomap): preserve git file path whitespace

* fix(repomap): handle graph and parsing edge cases

* fix(repomap): preserve shell token positions

* fix(repomap): respect configured cache home

* fix(repomap): address review findings

- Add explicit 10000ms timeout to the feature-flag-off context test to avoid cold-import flakes.

- Add --focus-symbols flag to /repomap and forward it to buildRepoMap, matching the RepoMap tool.

- Add parsing/command tests and docs coverage for --focus-symbols.

---------

Co-authored-by: gnanam1990 <gnanasekaran.sekareee@gmail.com>
2026-07-07 11:09:41 +08:00
9c0d5c61e2 fix(deps): remove deprecated uuid install path by replacing vertex-sdk with local client (#1771)
* fix(deps): remove deprecated uuid install path

* fix(api): address PR #1232 review — type the local Vertex client surface

Resolves the blocker raised by @Vasanthdev2004, @gnanam1990, and @jatmn: the
in-repo AnthropicVertex replacement compiled under bun (no type-check) but added
4 `tsc --noEmit` errors that the upstream typed SDK did not.

- Declare `messages`/`beta` as typed class fields (BaseAnthropic doesn't, but
  the upstream @anthropic-ai/vertex-sdk client did), so typed consumers —
  client.ts `new AnthropicVertex(...)` and the SDK calling `.messages` — keep
  the resource surface. (vertexClient.ts:145/146, test:53)
- Widen the header-merge helpers to accept the base client's request header type
  (HeadersLike), and handle the NullableHeaders shape it actually passes so the
  merge stays correct, not just type-clean. (vertexClient.ts:182)

Also drops the now-stale `@anthropic-ai/vertex-sdk` entries left behind by the
dependency removal:
- scripts/externals.ts INTENTIONALLY_BUNDLED (P3)
- knip.json ignoreDependencies

Testing: `tsc --noEmit` clean; vertex/client/gemini tests 51 pass; smoke green
(INTENTIONALLY_BUNDLED back in sync, 57 entries); knip clean.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(api): address CodeRabbit review on PR #1232 — auth precedence + coverage

- Security (vertexClient.ts): merge resolved Google auth headers LAST so a
  caller-supplied Authorization / x-goog-user-project can't override the Vertex
  credential and send the wrong token upstream. Other request headers still
  pass through unchanged.
- Tests (vertexClient.test.ts): add focused regression coverage for the
  previously-unguarded routing/auth branches —
    * streaming → :streamRawPredict path (+ model stripped, stream preserved)
    * count_tokens → count-tokens:rawPredict path rewrite
    * auth-header precedence: caller Authorization does NOT override the Vertex
      token (guards the fix above + exercises the NullableHeaders merge branch).

Testing: tsc clean; vertexClient tests 5 pass; full src/services/api 840 pass;
smoke + knip green.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(api): validate and encode Vertex model before building the URL

CodeRabbit follow-up on PR #1232: `model` was interpolated straight into the
Vertex endpoint path, so a missing/non-string model would silently route to
`.../models/undefined:rawPredict` instead of failing fast. Now throw a clear
error on a missing/empty model and encodeURIComponent the value before building
the path. Adds a focused test for the missing-model case.

Testing: tsc clean; vertexClient tests 6 pass; smoke + knip green.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(api): address remaining review items from PR #1232

- [P2] Fix count_tokens method guard: apply method==='post' to both paths
- [P3] Remove unused accessToken option from AnthropicVertex
- [P3] Narrow batches type on messages/beta resources with Omit

* test: add count_tokens?beta=true routing regression test

---------

Co-authored-by: Kevin Codex <kevin@gitlawb.com>
Co-authored-by: OpenClaude <openclaude@gitlawb.com>
Co-authored-by: Gravirei <gravirei@users.noreply.github.com>
2026-06-24 13:03:29 +08:00
Kevin CodexandOpenClaude f4c3be850e chore: remove dead code and add knip gate to CI check (#1612)
Delete 32 unreferenced source files (~4,000 lines) verified dead by
import-specifier grep and knip: test-only token utilities, orphaned hooks
(useTaskListWatcher, useSkillImprovementSurvey + its component), the
removed DevBar and ConfigTool UIs, unregistered bundled skills (stuck,
verifyContent), unused analytics sinks, the benchmark command, and
stale migrations/helpers.

Remove unused dependencies code-excerpt, stack-utils, and tsx from
package.json plus their entries in build stub/external lists.

Add knip with a tuned knip.json (entrypoints, build-time stub targets,
subprocess-launched fixtures, and runtime-string-imported SDKs ignored;
providerAutoDetect kept intentionally as provider pre-wiring) and wire
`bun run deadcode` into the `check` script so dead code stays dead.

Co-authored-by: OpenClaude <openclaude@gitlawb.com>
2026-06-14 10:13:37 +08:00
JATMNandJATMN 492cde2619 Remediate audit findings, replace vulnerable Firecrawl SDK, and harden release validation (#1030)
* Harden release publish checks and remove vulnerable Firecrawl SDK

Add a post-publish npm verification step to the release workflow so GitHub releases fail if the npm latest tag does not resolve to the expected version within the retry window.

Update dependency pins to remediate the audit findings by moving axios to 1.16.0, upgrading the Anthropic SDK to 0.94.0, and bumping the Bedrock and Vertex wrapper packages so Bun installs dedupe onto the patched SDK.

Replace the @mendable/firecrawl-js dependency with a small in-repo fetch-based Firecrawl client used by WebFetchTool and the Firecrawl web-search provider. Preserve self-hosted support, add transient 502 retry/backoff behavior, and cover the new client with focused tests.

Validation:
- bun test src/tools/firecrawl/client.test.ts src/tools/WebSearchTool/providers/firecrawl.test.ts
- bun run build
- bun run smoke
- packed-install npm audit --omit dev --json returned 0 vulnerabilities

* Harden Bun test isolation for release validation

Fix shared-module test leaks that were breaking providerProfile in the full serialized Bun suite.

- preserve full module surfaces when mocking env/provider modules
- remove unnecessary env/envUtils mocks from user/install surface tests
- use a fresh providerProfile module import for the Codex OAuth cleanup regression
- relax the Windows-only permission assertion in providerProfile tests

Validation:
- bun install --frozen-lockfile
- bun test --max-concurrency=1
- bun run smoke
- bun run build
- npm pack

* Complete execa mock coverage in user test

Fix the remaining cross-file Bun mock leak reported in review by expanding the persisted execa mock in src/utils/user.test.ts to include execaSync.

This keeps later imports that touch secure-storage and exec helpers from failing or hanging when bun test runs files serially after user.test.ts.

Validation:
- bun test src/utils/user.test.ts src/utils/effort.codex.test.ts
- bun test --max-concurrency=1
- bun run build
- bun run smoke
- npm pack

* Preserve full module surfaces in user test mocks

Convert the auth, config, cwd, and execa mocks in src/utils/user.test.ts into pass-through mocks with targeted overrides.

This fixes the remaining Bun process-global mock leakage where later suites could fail or hang after user.test.ts because leaked partial mocks were missing exports such as auth/config helpers or execaSync.

Validation:
- bun test src/utils/user.test.ts src/utils/effort.codex.test.ts
- bun test src/utils/user.test.ts src/utils/openclaudeInstallSurfaces.test.ts
- bun test --max-concurrency=1

* Override ip-address to 10.2.0

Add a top-level override for ip-address and refresh bun.lock so the MCP SDK -> express-rate-limit path resolves to ip-address@10.2.0 instead of 10.1.0.

This keeps the branch's audit-remediation scope aligned with the remaining transitive advisory path without changing the direct MCP SDK pin.

Validation:
- bun pm why ip-address
- bun audit

* fix: use cleanup-safe Firecrawl timeouts

* Isolate attribution settings tests

* test: remove stale provider profile import

---------

Co-authored-by: JATMN <12479882+jatmn@users.noreply.github.com>
2026-06-09 06:19:44 +08:00
ArkhAngelLifeJiggy 3bf6ccd6d8 fix: preserve raw mode across component re-renders (issue #843) (#1198)
* fix: preserve raw mode across component re-renders (issue #843)

* fix(input): only reset raw mode on explicit isActive=false, not on MCP re-render churn (issue #843)

* fix: balance raw mode for isActive false transitions + add regression test

Fixes the issue where cleanup closes over stale isActive=true and returns
early without calling setRawMode(false), leaving rawModeEnabledCount
incremented after UI no longer has active useInput.

Changes:
- Use a ref to track whether raw mode was actually enabled
- Check the ref in cleanup instead of stale isActive closure value
- Add 6 regression tests covering the true->false/unmount paths

Addresses jatmn's review feedback: 'fix raw mode balance for isActive: false transitions'

* fix(input): debounce raw-mode reset to survive MCP re-render churn (issue #843)

* fix: add react-test-renderer dep and fix use-input test for CI

- Add react-test-renderer devDependency (required by @testing-library/react-hooks)
- Add @testing-library/react-hooks to INTENTIONALLY_BUNDLED in externals.ts
- Fix use-input.test.ts 'MCP re-render churn' test to use isActive rerender
  instead of separate renderHook calls (refs don't persist across instances)

* fix: address P1 raw-mode counter imbalance and P2 test-dep scope (PR #1196)

P1 (use-input.ts:64-68): skip setRawMode(true) on isActive false->true
when a deferred reset is pending, preventing counter over-increment
that leaked raw mode on final unmount. Test updated to assert
balanced 1-then-1 call pattern (no redundant setRawMode(true)).

P2 (package.json, externals.ts): move @testing-library/react-hooks
from dependencies to devDependencies; remove from INTENTIONALLY_BUNDLED.
2026-06-03 19:54:00 +08:00
SukeshP1995 db6017a8b7 chore: replace strip-ansi with util.stripVTControlCharacters (#1380) 2026-06-01 17:02:45 +08:00
Kevin CodexandOpenClaude 2c71e09394 chore(build): clean up external dependency validation warnings (#1124)
* chore(build): clean up external dependency validation warnings

Remove 2 unused externals (@opentelemetry/sdk-trace-node, ink) and add
12 missing packages to package.json that are dynamically imported at
runtime but weren't declared as dependencies. Also remove the unused
@opentelemetry/sdk-trace-node dependency.

This eliminates all 13 build validation warnings:
- 8 missing OTel exporter deps (http, proto, grpc variants + prometheus)
- 4 missing AWS SDK deps (bedrock, bedrock-runtime, sts, credential-providers)
- 1 missing Azure dep (@azure/identity)
- ink external pointed to local reimplementation, not npm package
- sdk-trace-node was declared external but never imported

Build validation now passes cleanly with 0 warnings.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* chore(build): eliminate external validation warnings

Remove unused @opentelemetry/sdk-trace-node from externals and package.json
(it's not imported anywhere in src/). Remove ink from SDK_ONLY_EXTERNALS
(the project reimplements ink locally at src/ink/). Add OPTIONAL_RUNTIME_EXTERNALS
list for packages that are dynamically imported but intentionally not direct
deps — OTel protocol exporters and cloud provider SDKs are resolved from
transitive deps or installed by users who need them.

Validation now passes with 0 warnings instead of 13.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* feat(telemetry): full OpenTelemetry purge — remove all tracking dependencies

Replace all @opentelemetry/* runtime dependencies with no-op stubs,
delete OTel-only source modules, and remove 10 @opentelemetry packages
from package.json plus @growthbook/growthbook.

Key changes:
- Delete 5 OTel-only modules (instrumentation, betaSessionTracing,
  bigqueryExporter, logger, firstPartyEventLoggingExporter)
- Replace 9 modules with no-op stubs (sessionTracing, events,
  telemetryAttributes, firstPartyEventLogger, growthbook, index,
  sink, datadog, sinkKillswitch, perfettoTracing)
- Remove all @opentelemetry/* imports from bootstrap/state.ts,
  entrypoints/init.ts, and ~20 caller files
- Remove all OTel counter types, meter/provider state from state.ts
- Clean externals.ts: remove 27 @opentelemetry/* entries
- Clean build.ts: remove OTel native-stub namespace exports
- Simplify no-telemetry-plugin.ts: remove redundant source-level stubs
- Remove 10 @opentelemetry/* + @growthbook/growthbook from package.json
- GrowthBook stub reads local ~/.claude/feature-flags.json for overrides

Co-Authored-By: OpenClaude (zai-org-glm-5-1) <openclaude@gitlawb.com>

* fix format

* chore: regenerate lockfile after OTel dependency removal

Co-Authored-By: OpenClaude (zai-org-glm-5-1) <openclaude@gitlawb.com>

* fix(growthbook): route gate helpers through local flag overrides

checkStatsigFeatureGate_CACHED_MAY_BE_STALE() and
checkGate_CACHED_OR_BLOCKING() now resolve from
~/.claude/feature-flags.json like getFeatureValue_* does,
so gates like tengu_thinkback, tengu_ccr_bridge, and
VS Code upsells can be flipped on locally. Security gates
(checkSecurityRestrictionGate) remain hard-false.

Also adds 5 tests covering gate helper override behavior
and unifies JSDoc wording for _getFlagValue-routed functions.

Co-Authored-By: OpenClaude (zai-org-glm-5-1) <openclaude@gitlawb.com>

---------

Co-authored-by: OpenClaude <openclaude@gitlawb.com>
2026-05-13 17:07:45 +08:00
3kin0x 5873bc6714 feat(knowledge): introduce local Orama persistence (feature-flagged) (#1015)
* feat(knowledge): introduce local Orama persistence (clean phase 1)

- Added @orama/orama and persistence plugin.
- Implemented optional local-only Orama backend in knowledgeGraph.ts.
- Gated Orama logic behind OPENCLAUDE_KNOWLEDGE_ORAMA=1.
- Converted knowledge and conversation arc functions to async.
- Fixed circular dependency between knowledgeGraph and sessionStorage by moving getProjectsDir to envUtils.
- Updated all call sites and tests to handle async Knowledge API.
- Verified build and tests pass on latest main.

* fix: address PR review comments for knowledge feature (async finalizeArcTurn and Orama cleanup)

* test: add comprehensive stress and edge case testing for Orama Knowledge Graph

* fix: prevent test pollution by restoring Orama env flag in stress test

* refactor: harden Knowledge architecture with concurrency locks, optimized I/O, and consolidated state
2026-05-09 22:35:37 +08:00
Ali AlakbarliandAli Alakbarli 60c76b6599 feat: SDK Runtime — Query Engine, Sessions, and Build Pipeline (#984)
* feat(sdk): add SDK foundation — type declarations, errors, and utilities

Adds standalone SDK building blocks with no SDK source dependencies:
- sdk.d.ts: ambient type declarations for SDK bundle
- coreSchemas.ts + coreTypes.generated.ts: Zod schemas and generated types
- errors.ts: SDK-specific error classes
- validation.ts: input validation utilities
- messageFilters.ts: extracted message filter logic
- handlePromptSubmit.ts: imports from messageFilters
- 16 generated-types tests

* fix(sdk): narrow assertFunction type from broad Function to callable signature

Code review finding: assertFunction used `asserts value is Function` which
accepts any function-like value without narrowing. Changed to
`(...args: any[]) => any` for better type safety.

* fix(sdk): update sdk.d.ts header — manually maintained, not generated

Reviewer noted the header said "Generated from index.ts" but no generator
produces this file. Updated to "Manually maintained — keep in sync with
index.ts". Drift detection added in validate-externals.ts (PR 3).

* fix(sdk): align sdk.d.ts types with canonical coreTypes.generated.ts

Tighten SDK public type contract to resolve reviewer blockers:

- PermissionResult: unknown[] → precise 6-shape discriminated union
  (addRules/replaceRules/removeRules/setMode/addDirectories/removeDirectories)
- SDKSessionInfo: snake_case → camelCase (sessionId, lastModified, etc.)
- ForkSessionResult: session_id → sessionId
- SDKPermissionRequestMessage: uuid + session_id now required
- SDKPermissionTimeoutMessage: added uuid + session_id
- SessionMessage: parent_uuid → parentUuid
- SDKMessage/SDKUserMessage/SDKResultMessage: replaced loose inline
  definitions with re-exports from coreTypes.generated.ts

* feat(sdk): wire existing code modules + SDK shared utilities

Modifies core modules for SDK integration:
- QueryEngine, tools, state, commands: SDK type hooks
- SDK shared utilities (shared.ts, permissions.ts)
- 21 SDK tests (shared-utils, permissions)

Stack: main ← pr1-foundation ← pr2-sdk-core

* feat(sdk): add snake_case ↔ camelCase key mapping utilities

casing.ts provides recursive key transformation for the SDK boundary
layer. Internal runtime uses snake_case; public API exposes camelCase.
Will be used by shared.ts, sessions.ts, query.ts at export boundaries.

* test(sdk): add tests for snake_case ↔ camelCase mapping utilities

Covers snakeToCamel, camelToSnake, mapKeysToCamel, mapKeysToSnake
including nested objects, arrays, null/undefined, and round-trips.

* feat(sdk): add SDK runtime — query engine, sessions, build pipeline

Completes the SDK implementation:
- SDK build target (dist/sdk.mjs) with TUI dependency stubbing
- External dependency lists (scripts/externals.ts)
- SDK type generation from Zod schemas (scripts/generate-sdk-types.ts)
- External validation (scripts/validate-externals.ts)
- SDK source: index, query, v2, sessions modules
- agentSdkTypes: re-exports SDK functions (query, createSession, etc.)
- 136 SDK tests + 7 build scanner tests

Stack: main ← pr1-foundation ← pr2-sdk-core ← pr3-sdk-runtime

* fix(sdk): align internal SDK types with camelCase public contract

shared.ts: SDKSessionInfo, ForkSessionResult, SessionMessage fields
now use camelCase matching sdk.d.ts. SDKPermissionRequestMessage and
SDKPermissionTimeoutMessage gain required uuid + session_id fields.

permissions.ts: onPermissionRequest/onTimeout callbacks now include
uuid and session_id in emitted messages.

* fix(sdk): update runtime modules to use camelCase field names

sessions.ts: toSDKSessionInfo outputs camelCase keys, entryToSessionMessage
uses parentUuid, forkSession returns sessionId.

query.ts: reads sessionId from listSessions/forkSession results
instead of snake_case session_id.

* fix(test): update session tests to use camelCase field names

session_id → sessionId in forkSession result assertions and
getSessionMessages calls.

* fix(sdk): prevent permission timeout race condition with once-only resolve wrapper

Add createOnceOnlyResolve utility to prevent double-resolution of promises
when timeout and host response happen simultaneously. This ensures
deterministic behavior in the permission handling flow.

* fix(sdk): improve race condition test robustness

* fix(sdk): handle consecutive underscores in snakeToCamel conversion

Changes:
- Use _+([a-z]) regex to match multiple consecutive underscores before letters
- Add lookahead (?=. ) to preserve underscore-letter pairs at string end
- Handle dunder names (__proto__, __typename) by stripping wrapper and capitalizing
- Add tests for consecutive underscores and trailing underscore preservation

* fix(sdk): include original error message in permission callback denial

When a canUseTool callback throws an error, the catch block now
includes the original error message in the denial message, making
debugging easier for SDK consumers.

* feat(sdk): add optional timeout to env mutex for deadlock prevention

Add timeout parameter to acquireEnvMutex() to prevent infinite waits
in deadlock scenarios. The timeout is optional and defaults to no timeout
(wait forever) for backward compatibility.

Returns a MutexAcquireResult object with acquired status and optional
timeout reason for failed acquisitions.

* fix(sdk): remove timed-out callback from mutex queue to prevent deadlock

* test(sdk): add missing error path and timeout scenario tests

Add tests for timeout scenarios when host doesn't respond to permission
requests, fallback behavior when no onPermissionRequest callback, and
MCP connection edge cases for undefined/empty config.

* fix(sdk): address code review issues - race conditions, validation, error handling

- Add createPermissionTarget() factory that applies onceOnlyResolve at
  registration time, fixing race condition where timeout and host response
  could both try to resolve the same promise
- Add try-catch to releaseEnvMutex() to prevent permanent lock if callback throws
- Extract DEFAULT_PERMISSION_TIMEOUT_MS constant (30 seconds)
- Add MCP config validation rejecting null, non-objects, and arrays
- Preserve error stack traces in MCP connection failures
- Add runtime validation to mapMessageToSDK for null/non-object/invalid type
- Update tests to use createPermissionTarget and add validation tests

* fix(sdk): syntax fixes and MCP connection error handling

- Remove extra closing parenthesis in permissions.ts
- Remove extra closing braces in shared.ts type definitions
- Wrap MCP connection in try/catch to continue without MCP tools on failure

* fix(sdk): syntax fixes, MCP error handling, and logic clarity

- Remove extra closing parenthesis in permissions.ts
- Remove extra closing braces in shared.ts type definitions
- Wrap MCP connection in try/catch to continue without MCP tools on failure
- Clarify thinkingConfig logic: use ?? true instead of !== false
- Add explanatory comment about thinkingEnabled default behavior
- Apply createOnceOnlyResolve wrapper in QueryImpl.registerPendingPermission

* fix(sdk): comprehensive error handling and resource cleanup

- Add try-catch around injectAgents() to gracefully handle plugin agent
  tool validation failures (prevents test crashes from unknown 'LS' tool)
- Add console.warn logging to agent loading/injection catch blocks for
  debugging visibility (matches v2.ts pattern)
- Add pendingPermissionPrompts.clear() to close() and interrupt() methods
  in both query.ts and v2.ts to prevent memory accumulation
- Add close() method to SDKSession interface and SDKSessionImpl
- Wrap MCP connection in query.ts with try-catch (matches v2.ts behavior)
- Add timeoutQueue cleanup in finally blocks (query.ts + v2.ts)
- Remove error.stack from MCP error messages to prevent internal path leak

All 208 SDK tests pass. TypeScript errors are pre-existing.

* fix(sdk): address code review non-blocking issues

- Add SDKAgentLoadFailureMessage type for agent load failure events
- Emit agent definition/injection failures to SDK message stream
- Add tool name to permission timeout denial message
- Replace 'as any' casts with proper typed state access
- Fix supportedCommands to use correct mcp.commands/plugins.commands paths
- Update test for correct AppState structure

* fix(sdk): address code review blocking and non-blocking issues

Blocking Issues Fixed:
- MCP cleanup missing on session/query close - now disconnects MCP clients
  to prevent resource leaks in long-running processes with multiple sessions
- Engine reference not cleared on close - now sets _engine = null to prevent
  memory leaks
- Added MCP cleanup tests (9 new tests covering cleanup scenarios)

Non-Blocking Issues Fixed:
- Removed redundant catch block that just rethrew errors (query.ts)
- Fixed inconsistent timeout denial message format (permissions.ts)
- Fixed hardcoded tool name 'Bash' in test (permissions.test.ts)
- Exported PermissionResolveDecision type for SDK consumers (index.ts)

All 217 SDK tests pass.

* fix(sdk): address code review type consistency issues

- Add close() method to SDKSession interface (documented but missing from type)
- Fix SDKSessionInfo, ForkSessionResult, SessionMessage field naming:
  snake_case → camelCase to match sdk.d.ts public contract and implementation
- Add uuid and session_id to SDKPermissionTimeoutMessage for correlation
- Fix JSDoc comment in forkSession to use sessionId (not session_id)

These changes align internal types (shared.ts) with the public SDK contract
(sdk.d.ts) and actual implementation output. The merge from origin/main
introduced snake_case types that mismatched camelCase implementation and tests.

* fix: restore openclaude.json comment in REPL.tsx

Merge 0f3aa7a incorrectly took main's side for this comment, reverting
PR2 fix c725c48. Project has migrated to ~/.openclaude.json, not ~/.claude.json.

This is the only PR2 fix lost during merge - all other PR2 fixes
(permissions.ts race conditions, state.ts parentSessionId, etc.)
are preserved in PR3 via subsequent fix commits.

* fix(sdk): add missing type declarations to sdk.d.ts

Add SDKAgentLoadFailureMessage and PermissionResolveDecision to sdk.d.ts
to resolve type declaration drift detected by build validation.

- SDKAgentLoadFailureMessage: Agent loading failure notification
  (stage: definitions/injection, error_message)
- PermissionResolveDecision: SDK-specific permission resolution result
  (allow with updatedInput, deny with message + decisionReason)

Build validation now passes: 56 exports match between index.ts and sdk.d.ts.

* fix(sdk): resource leak and null safety in close/interrupt paths

- unstable_v2_prompt: wrap session in try/finally to guarantee
  session.close() on both success and error paths, preventing
  MCP connection and engine resource leaks
- QueryImpl.interrupt(): add null guard on _engine so calling
  interrupt() after close() is a safe no-op instead of throwing
- SDKSessionImpl.interrupt(): add matching null guard for v2
  sessions, consistent with the Query fix
- QueryImpl.close(): call this.interrupt() before cleanup to
  properly stop in-flight engine operations, matching v2's close()
  pattern and ensuring engine.interrupt() runs before nulling

* fix(sdk): abort AbortController in SDKSessionImpl.close() to prevent resource leak

SDKSessionImpl.close() was not aborting the AbortController, unlike
QueryImpl.close() which does. This meant in-flight HTTP requests and
async operations could continue running after session closure.

- Store AbortController reference via _abortController field + late-bind setter
- Abort and null the controller in close(), mirroring QueryImpl pattern
- Also null _appStateStore in close() to release state snapshots
- Wire abortController through createEngineFromOptions return value

* fix(sdk): index ALL entries in byUuid for compact preserved segment

The byUuid map must index system compact_boundary entries, not just
user/assistant. When anchorUuid === boundary.uuid, the relink walk
needs to find the boundary in byUuid.

Changes:
- query.ts: Index ALL non-sidechain entries (user, assistant, system)
- v2.ts: Same fix — index ALL entries, leaf selection user/assistant only
- Add regression test: boundary.uuid as anchorUuid scenario

Test verifies preserved messages kept, stale pre-compact dropped,
post-boundary chain intact when anchorUuid points to boundary itself.

* fix(sdk): complete preserved segment handling for compact resumes

Multiple fixes for compact-aware transcript loading:

1. Index ALL entries in byUuid (including system compact_boundary)
   - Needed when anchorUuid === boundary.uuid

2. Keep anchorUuid when pruning preserved segment entries
   - The anchor is the parent of preserved head after relink
   - Deleting it breaks the conversation chain

3. Filter system entries from final messages
   - compact_boundary is metadata, shouldn't pass to engine

4. Fix test timestamp format (ISO 8601 requires 2-digit hours)
   - '2025-01-04T0:00:00Z' → '2025-01-04T00:00:00Z'

5. Update test expectations for anchor inclusion
   - When anchor is a stale entry, it appears in messages
   - preserved(4) + anchor(1) + post(4) = 9 max

All 224 SDK tests pass.

* fix(sdk): MCP type:sdk tools properly convert SdkMcpToolDefinition to Tool

- Import MCPTool base from tools/MCPTool/MCPTool.js
- Spread MCPTool properties for proper Tool interface compliance
- Add tools field to SdkMcpSdkConfig type declaration
- Add regression tests for type:sdk tools wiring

Fix ensures in-process SDK tools match Tool interface expected
by QueryEngine and permission handlers.

* test(sdk): strengthen preserved segment and MCP tools tests

Preserved segment test improvements:
- Fix content extraction (access message.content, not message)
- Add exact count assert: messages.length === 6
- Add exact content asserts: preserved turn 1/2, post-boundary present
- Assert no stale, no system entries in final messages

MCP tools test additions:
- Direct test of connectSdkMcpServers() function
- Assert clients.length === 0 (in-process, no MCP connections)
- Assert tools.length === 1 with proper name/description
- Verify handler works via direct call (not via Tool.call which needs context)

* fix(sdk): published types complete, init errors fatal, permission session IDs

Three fixes for SDK production readiness:

1. HIGH: Published SDK types incomplete
   - Add coreTypes.generated.d.ts to package.json "files" array
   - sdk.d.ts re-exports from ./sdk/coreTypes.generated.js which was missing
   - TypeScript consumers would get module resolution errors

2. MEDIUM: query() swallows real init() failures
   - Add _engineWasInjected field to track pre-injected vs fresh engine
   - Check _engineWasInjected, not _engine !== null (always true after setEngine)
   - Auth/config/init errors now properly fatal for normal query() calls

3. MEDIUM: SDK permission events lose real session id
   - Pass sessionId to createExternalCanUseTool() in both query.ts and v2.ts
   - Permission_request/timeout messages now have correct session_id
   - Hosts can correlate permission callbacks to sessions

Test result: 225 pass, 0 fail

* fix(sdk): complete package types + dynamic permission session_id

Two fixes for SDK production readiness:

1. Published SDK types now include actual definitions
   - Replace 215-byte wrapper with 63KB coreTypes.generated.ts
   - TypeScript consumers get full type definitions (SDKMessage, etc.)
   - npm pack now includes real generated types

2. Permission event session_id dynamic for all query() paths
   - createExternalCanUseTool accepts string | (() => string | undefined)
   - query.ts passes () => queryImpl.sessionId getter
   - Fresh/fork/continue queries emit correct session_id at event time
   - V2 passes static sessionId (stable at creation/resume)
   - Add 4 tests: static sessionId, getter resolution, undefined fallback, timeout

Test result: 229 pass, 0 fail

* fix(sdk): fix sdk.d.ts for real TypeScript consumer compilation

Two issues prevented external consumers from compiling against packed SDK types:

1. SDKRateLimitError used constructor parameter properties (readonly resetsAt,
   readonly rateLimitType) which are invalid in .d.ts declarations — moved to
   class properties with separate constructor signature.

2. Re-exported SDKMessage/SDKUserMessage/SDKResultMessage were not imported
   into local scope — added import type alongside export type so TypeScript
   can resolve them for use in other declarations within the same file.

Added package-consumer-types.test.ts that compiles a real temp project against
the SDK types with skipLibCheck:false, catching both regressions.

* fix(sdk): eliminate React/Ink imports from SDK bundle

SDK bundle leaked React/Ink imports via tool UI modules, keybindings,
react-compiler-runtime, and spawnMultiAgent's static React import.

Changes:
- Stub root ink.js barrel, tool UI.js, keybindings/, react-compiler-runtime,
  It2SetupPrompt, and React hook files in SDK build
- Add local no-op stub for react/jsx-dev-runtime (jsxDEV returns null)
- Convert spawnMultiAgent's static React/It2SetupPrompt imports to dynamic
  await import() — spawnTeammate logic stays fully intact
- Add post-build leakage validation (fails on from "react"/"ink"/jsx-dev-runtime)
- Remove react/jsx-dev-runtime from SDK externals (now handled by build plugin)

* fix(sdk): wire disallowedTools through permission context

QueryOptions.disallowedTools was declared but never used. buildPermissionContext()
now passes it to alwaysDenyRules.cliArg so getTools() filters denied tools from
the model-visible list. Also added to V2 SDKSessionOptions for API consistency.

* fix(sdk): defer permission warning to execution time

createDefaultCanUseTool() warned at construction time even when the caller
provided canUseTool/onPermissionRequest. Move warning to first actual default
denial so valid SDK consumers never see false warnings. Add tests for
disallowedTools filtering, tool exclusion, and warning timing.

* refactor(sdk): extract transcript helpers + fix permission typing

- Extract shared transcript utilities to transcript.ts
  (parseJsonlEntries, findLastCompactBoundary, applyPreservedSegmentRelinks,
  buildConversationChain, stripExtraFields) deduplicating query.ts and v2.ts

- Add PermissionTarget interface to hide internal pendingPermissionPrompts
  map from createExternalCanUseTool, with deletePendingPermission and
  denyPendingPermission methods on QueryImpl and SDKSessionImpl

- Fix sessionId stability: preserve constructor UUID for fresh queries
  when continue:true finds no existing sessions, and when explicit
  sessionId does not resolve to a valid transcript file

- Add getMcpClients/setMcpClients to QueryEngine for SDK cleanup access

* fix(sdk): resolve remaining TypeScript errors in SDK modules

- Fix PermissionDecision type compatibility: import from types/permissions
  and cast PermissionResolveDecision to PermissionDecision properly

- Fix AsyncIterator/AsyncGenerator: async generators must return
  AsyncGenerator (which implements AsyncIterable), not AsyncIterator

- Fix Map method callable errors: cast additionalWorkingDirectories
  to Map<string, unknown> before calling .set() and .keys()

- Fix ApiKeySource type: map internal ApiKeySource to SDK's narrower
  type using conversion function, spread info before apiKeySource
  to avoid override

- Fix MCP config scope type: cast 'session' scope to ScopedMcpServerConfig
  for connectToServer compatibility

- Add PermissionMode import and cast for decisionReason.mode

- Deny pending permissions in interrupt(): resolve all pending promises
  with deny before clearing the map (both query.ts and v2.ts)

* fix(sdk): correct init skip logic and test mocks

- query.ts: skip init() entirely for injected engines (mocks, SDK host
  overrides) instead of calling init() and swallowing errors. Pass
  { injected: false } from query() factory to distinguish real engine
  from test mocks.
- mock-engine.ts: add getMcpClients() and setMcpClients() methods to
  match QueryEngine API added in this PR.
- permissions.test.ts: use filterToolsByDenyRules instead of getTools
  for disallowedTools tests, with proper base tool fixtures.

* fix: address code review feedback for exports and build script

package.json exports (Breaking Change Mitigation):
- Add "./package.json": "./package.json" for tool compatibility
- Add "./dist/cli.mjs": "./dist/cli.mjs" for CLI bundle access
- Keep ./sdk as sole library entrypoint
- Root import intentionally blocked (CLI-first package, no main field)

build.ts (Bug Fix):
- Add | undefined to result/sdkResult type declarations
- Add optional chaining: result?.success, sdkResult?.success
- Prevents TypeError masking actual build errors when Bun.build throws

tests/sdk/package-consumer-types.test.ts:
- Update simulated exports to match real package.json
- Add tests verifying exports map structure and file existence

---------

Co-authored-by: Ali Alakbarli <ali.alakbarli@users.noreply.github.com>
2026-05-04 20:56:30 +08:00