45 Commits
Author SHA1 Message Date
649c224f3c feat(web-search): add AI/ML API Toolbox with safe error reporting (#2465)
* feat(web-search): add AI/ML API Toolbox as an explicit search backend

Add an `aimlapi` web search provider that runs the AI/ML API Toolbox tool
`web/search` (POST https://tools.aimlapi.com/v1/tools/run). It reuses the
key the AI/ML API model provider already has (AIMLAPI_API_KEY, or the
route's OPENAI_API_KEY fallback while OPENAI_BASE_URL is the canonical
AI/ML API endpoint), so a user on that provider has nothing to paste.

Every run is billed to the user's AI/ML API balance, so the backend is
explicit-only: it is not in the auto chain, and an AIMLAPI_API_KEY alone
never routes searches through it. Each run carries max_cost_usd 0.025.

- Map output.results[] (url/title/snippet/published_at) onto search hits,
  prefixing the snippet with the publication date.
- Send allowed_domains as include_domains; apply both domain lists to the
  results like the other backends (the tool has no exclude field).
- Poll GET /v1/tools/runs/{id} after a 202 until the run ends, within the
  web search timeout; report is_error runs and HTTP error codes.
- Send the key and the X-AIMLAPI-Source / X-AIMLAPI-Partner-ID attribution
  headers only to tools.aimlapi.com, checked by exact hostname.
- /search aimlapi selects it only when the key is present and states the
  billing; /search key|remove-key aimlapi refuse, because saving
  AIMLAPI_API_KEY to the global env would also switch the model provider.
- Document the backend next to the other search providers.

* fix(web-search): never send a proxy profile's key to the AI/ML API Toolbox

An aimlapi provider profile that points at the user's own proxy exports
the proxy's credential as AIMLAPI_API_KEY, with
CLAUDE_CODE_PROVIDER_ROUTE_ID=aimlapi and OPENAI_BASE_URL set to the proxy.
The search provider returned AIMLAPI_API_KEY unconditionally, so
/search aimlapi would have sent that proxy credential to tools.aimlapi.com.

Resolve no key in that case: the backend reports itself as not
configured, /search aimlapi refuses the selection, and a search fails
with the not-configured error before any request is made.

* fix(web-search): link the published AI/ML API setup page from /search

The npm package does not ship docs/, so the /search key|remove-key aimlapi
message pointed at a path an installed user cannot open. Link the page on
GitHub instead and update the matching test expectation.

* fix(web-search): refuse redirects on Toolbox requests and pin the cost cap

Both Toolbox requests carry the key and the attribution headers, but the
hostname check only covered the first URL. Send them with redirect: 'error'
so no redirect hop can receive the headers, and assert that in the run and
poll tests plus a new test where a redirect fails after a single request.

Pin the documented $0.025 max_cost_usd in the request-body test instead of
comparing against the exported constant.

* fix(web-search): redact echoed AI/ML API credentials in search errors

* docs(web-search): describe the requested AI/ML API cost limit

* docs(web-search): clarify variable pricing and credential reuse

* fix(web-search): honor AI/ML API endpoint alias

* test(search): isolate provider credentials per test

* fix(web-search): reuse individual AI/ML API pool credentials

---------

Co-authored-by: Stan <stan@aimlapi.com>
Co-authored-by: Hugo <hugo@aimlapi.com>
2026-10-28 21:35:43 -07:00
Ilya Bogin e61e913b03 feat: add Keenable as an explicit web search backend (#2435)
* feat: add Keenable as an explicit web search backend

Adds a Keenable provider selectable with /search keenable or
WEB_SEARCH_PROVIDER=keenable. It works without a key through the keyless
endpoint and uses the keyed endpoint when KEENABLE_API_KEY is set. Keenable
stays out of the auto chain, so existing search order does not change.

* fix(web-search): fall back to the Keenable description when a result has no snippet

* test(web-search): type the description-fallback fetch mock like its neighbours

The zero-argument mock did not overlap Bun's fetch type and failed bun run typecheck:fast with TS2352. It now takes the typed request arguments the other mocks in the file use.
2026-10-28 06:57:01 -07:00
JATMN 1ce3c30dee fix: remove next 40 unused local findings (batch 5) (#2457) 2026-10-27 10:24:37 -07:00
JATMN 31274c145a fix: remove next 40 unused local findings (#2454) 2026-10-27 10:06:01 -07:00
JATMN 9876d04b3a fix: simplify redundant UI conditions (#2449) 2026-10-27 08:50:43 -07:00
JATMN 3adc887e85 fix: clean up existing ESLint warnings (#2429)
* fix: resolve all existing ESLint any warnings

* fix(sdk): preserve raw shapes with _input fields

* fix(sdk): preserve interface inputs and mutable raw arguments

* fix(sdk): enforce callback variance at tool creation

* fix(sdk): preserve raw input and JSON handler compatibility

* fix(sdk): preserve handlers for keyless JSON schemas

* fix: restore original SDK boundary for lint cleanup
2026-10-26 21:41:55 -07:00
GravireiandGravirei afa71563b6 chore(rebrand): update Gitlawb references to Twigpine (#2246) (#2248)
* chore(rebrand): update Gitlawb references to Twigpine (#2246)

The GitHub org has been renamed to Twigpine, so point repo URLs, badges,
CI repo-identity guards, and user-facing sponsor/partner labels at the new
name.

Scope is deliberately limited to references that are safe to change today:

- GitHub URLs (github.com, api.github.com, raw.githubusercontent.com),
  including the skills registry and release-please release notes
- The release-please release trigger, which gates on the repo slug and
  would otherwise stop publishing
- User-visible display strings: gateway label/name, AI/ML API partner
  name, sponsored-tip sponsor, and docs/site prose

Left on the old name because the rename has not landed for them:
- `@gitlawb/openclaude` npm scope (the package does not exist on npm)
- opengateway/ads/docs subdomains, which resolve only under gitlawb.com
- The `gitlawb-opengateway` preset id, tip ids, and gateway file names,
  which are persisted in user config and sent on the wire

* fix(aimlapi): stop matching topup intent on partnerName (#2246)

partnerName is display metadata that travels alongside the partner id, not
part of the intent identity. It is now excluded from INTENT_KEYS, so
matchesIntent no longer requires it to be unchanged.

Without this, renaming the partner on the Gitlawb -> Twigpine rebrand would
have stranded any in-flight topup: a receipt persisted under the old name
fails the strict equality check, so loadAimlapiTopupState returns null and
the user restarts checkout. partnerId remains the identity field, so
receipts are still scoped to the correct partner.

* fix(web): use current Twigpine artwork for the partner card (#2246)

The partner asset was a byte-identical rename, so the roster row rendered
the former GitLawb key/branch symbol under the Twigpine name while the
README pointed at the official tree mark. Vendors the current artwork from
twigpine.com/logo.png, resized to the roster's 256x256 convention, keeping
the self-hosting contract in web/src/data/partners.ts.

---------

Co-authored-by: Gravirei <gravirei@users.noreply.github.com>
2026-10-12 09:15:34 +08:00
Sakiandliuyang 5bc160fb8b feat: add AnySearch as an explicit web search backend (#2261)
* feat: add AnySearch web search backend

* fix(anysearch): preserve non-json error details

---------

Co-authored-by: liuyang <liuy.web@lingyu.work>
2026-10-12 08:39:56 +08:00
Kevin CodexandOpenClaude 790c009ae9 feat(search): make Exa the default web search backend and add /search (#2257)
* feat(search): make Exa the default web search backend and add /search

Exa now leads the auto chain: keyed Exa (EXA_API_KEY) runs first, other
keyed backends next, then a new keyless Exa free-tier adapter (Exa's hosted
MCP endpoint), with DuckDuckGo as the last-resort fallback. Built-in search
for Anthropic/Vertex/Foundry/Codex is unchanged. WEB_SEARCH_PROVIDER=exa
uses the keyed API when a key is set, otherwise the free tier.

New Exa options: EXA_SEARCH_TYPE, EXA_NUM_RESULTS, EXA_FREE_TIER (opt-out).
The free-tier adapter handles all three rate-limit reply shapes, spaces
calls to Exa's default QPS, backs off for five minutes after a limit, and
parses Exa's plain-text result blocks defensively.

New /search command: pick a backend, add or remove an API key via a masked
input, show status, run a test search. Choices are saved to the env block
of ~/.openclaude.json (0600) and applied immediately; settings.json env
entries that override them are reported instead of silently losing.

Security: isSensitive arg redaction now also covers local-jsx commands
(/ads, /search), which previously recorded raw inline args in the
transcript and model context.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(search): fall through when domain filters empty the Exa free-tier hits

The keyless Exa endpoint has no server-side domain filters, so when the
client-side filter removed every hit, runSearch returned a successful empty
result and never tried DuckDuckGo, which can honor the filter. Providers can
now set `fallbackInAuto` on their output; auto mode skips to the next
backend on that signal, and returns the filtered-empty result only if no
later backend does better. Genuine no-results replies and explicit `exa`
mode are unchanged.

Also adds doc comments to the touched functions that lacked them.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

---------

Co-authored-by: OpenClaude <openclaude@gitlawb.com>
2026-10-06 10:37:04 +08:00
JATMN 5fb5abb6f8 fix(ci): restore main validation (#2236) 2026-09-22 10:21:10 +08:00
JATMN 70d97fd137 feat(web-search): add Ollama backend (#2234)
* feat(web-search): add Ollama backend

* fix(web-search): harden Ollama routing

* fix(web-search): preserve Ollama profile identity

* fix(web-search): align Ollama diagnostics

* fix(web-search): align Ollama endpoint diagnostics

* fix(web-search): honor Ollama endpoint aliases

* fix(web-search): reject Ollama key placeholders

* fix(web-search): complete Ollama configuration guidance
2026-09-21 09:10:23 +08:00
6d09260291 chore(types): reduce any usage with safety tooling + 7-batch phase 1 (#2208)
* chore(types): add any-usage baseline + lint budget gate (Phase 0)

Phase 0 of a multi-phase plan to reduce explicit `any` usage safely.
No source files modified in this PR.

- Add eslint@9 + @typescript-eslint deps (lint only, no fix rules).
- New minimal flat eslint.config.js enabling only
  `@typescript-eslint/no-explicit-any` as warn. Stub plugins registered
  for legacy `custom-rules`, `eslint-plugin-n`, and `react-hooks`
  references in existing `// eslint-disable` comments so the comments
  stay valid (no rule name resolution errors).
- New `scripts/any-usage-report.ts`: walks src/, tests/, scripts/ and
  buckets `any` usage into `: any` annotations, `as any` casts,
  `[key: ...]: any` index signatures, `// @ts-ignore` directives, and
  generic `any` args. Emits `reports/any-usage.{json,md}`. `--check`
  mode exits non-zero if the count grows vs the baseline.
- Lock the baseline at 1,031 occurrences across 198 files.
- Add `lint`, `lint:any-budget`, `check:strict` scripts.
- Plan: docs/plans/imperative-hugging-sphinx.md

Verification:
- `bun run typecheck` clean
- `bun run lint` exit 0 (845 warnings, 0 errors)
- `bun run lint:any-budget` ok (current=1031 baseline=1031)

* chore(types): narrow any in 6 small src files (Phase 1, batch 1)

Phase 1 of the plan to reduce `any` usage. Targets only files with
1–2 lint hits where the concrete type is reachable in scope.

- src/services/api/withRetry.ts: isQuotaExhausted signature `any` → `unknown`,
  add local structural narrowing for `.message`/`.status` access.
- src/utils/validation.ts: assertFunction predicate signature `any[]`/`any` →
  `unknown[]`/`unknown`.
- src/utils/conversationArc.ts: extractTextFromContent — replace `(block: any)`
  in filter/map with explicit type predicate narrowing.
- src/utils/optionalRuntimeModule.ts: dynamic-import wrapper returns
  `Promise<unknown>` instead of `Promise<any>`.
- src/entrypoints/mcp.ts: tighten block mapping with the existing structural
  type already in scope; replace `(finalResult as any).isError` with a
  narrow cast.
- src/components/EffortPicker.tsx: drop redundant `: any` on useAppState
  selector — the function's generic infers `AppState` for free.

Verification:
- bun run typecheck clean
- bun run lint: 835 warnings (was 845)
- bun run lint:any-budget: 1022 / 1031 baseline (delta -9)
- bun test src/utils/optionalRuntimeModule.test.ts + conversationArc.test.ts:
  36 pass / 0 fail
- bun test src/services/api/withRetry.test.ts: 44 pass / 0 fail

* chore(types): narrow any in WebSearchTool + adapters (Phase 1, batch 2)

Phase 1 continues. These are all boundary code (HTTP responses from
external search APIs), so every `any` is at a module boundary per the
plan. Each adapter now types its response with `unknown` + structural
narrowing instead of `any`.

- providers/types.ts: firstMatch/normalizeHit accept `unknown` and narrow
  to `Record<string, unknown>` before index access.
- providers/timeout.ts: fetchJsonWithWebSearchTimeout returns
  Promise<unknown>.
- providers/{bing,brave,jina,linkup,mojeek,tavily,you,exa,custom}.ts: each
  adapter's response data is cast to a minimal shape (`{ results?: unknown }`
  etc.), then array elements are narrowed to `Record<string, unknown>` with
  `typeof` checks per field.
- WebSearchTool.ts: getCodexSources and extractCodexWebSearchFailure now
  take `Record<string, unknown>`; nested `action`/`error`/`result` are
  cast locally to `Record<string, unknown>` so `?.field` chains type-check.

Verification:
- bun run typecheck clean
- bun run lint: 805 warnings (was 835, -30)
- bun run lint:any-budget: 992 / 1022 baseline
- bun test src/tools/WebSearchTool: 138 pass / 0 fail across 9 files

* chore(types): narrow any in 6 mid-size src files (Phase 1, batch 3)

Phase 1 continues. Each replacement uses a precise structural cast
(`as { uuid?: string }` etc.) or `unknown` with typeof/Array.isArray
narrows at use sites.

- src/utils/settings/types.ts: zod preprocess callback `any` → `unknown`,
  copy via `Record<string, unknown>`.
- src/entrypoints/sdk/v2.ts: initialMessages param/var `any[]` → `Message[]`;
  local cast at the boundary with stripChainFields.
- src/entrypoints/sdk/query.ts: msg `.uuid` access uses
  `as { uuid?: string }`; `fileHistoryCanRestore` etc. accept `UUID` (from
  `crypto`); `mcpServerStatus` accesses `client.error` / `client.config`
  directly since the union members already have them.
- src/commands/cache-probe/cache-probe.ts: getField reduce `any` → `unknown`
  with typeof guard; catch (err: any) → `err: unknown` with
  `err instanceof Error`; mainMakeUsage/mainConvertChunkUsage take `unknown`
  and read fields via typeof checks.
- src/services/compact/snipProjection.ts: snipMetadata / uuid accessed via
  narrow structural casts, no longer `as any`.
- src/utils/messages.ts: tool_use block destructuring for extra_content uses
  `as { extra_content?: unknown; [k: string]: unknown }` instead of
  `as any`.

Public SDK surface (entrypoints/sdk.d.ts, entrypoints/sdk/index.ts
SdkMcpToolDefinition<Schema = any>) is intentionally permissive for
consumer ergonomics and is left untouched per the plan's do-not-touch
list.

Verification:
- bun run typecheck clean
- bun run lint: 782 warnings (was 805, -23)
- bun run lint:any-budget: 969 / 992 baseline
- bun test (sdk v2/factories + compact + messages): 91 pass / 0 fail

* fix(types): correct BetaContentBlock cast in messages.ts (Phase 1 fix)

Verifier caught that the `as { [k: string]: unknown; extra_content?: unknown }`
cast on a `BetaToolUseBlock` did not sufficiently overlap (BetaToolUseBlock
lacks an index signature), so the destructured spread of `restBlock`
yielded a malformed union member that failed to assign to BetaContentBlock[].

Fix: cast through `unknown` first, then assert the final return value as
`BetaContentBlock` (the declared element type of the outer `.map`). This
narrows from the prior `as any` while keeping the previous structural
intent (preserve all block fields, optionally inject `extra_content`).

Verification:
- bun run typecheck clean
- bun test src/utils/messages/: 79 pass / 0 fail

* chore(types): narrow any in 5 dense src + 1 script (Phase 1, batch 4)

Each replacement preserves runtime behavior while using concrete types
where reachable and `unknown` with structural narrowing where not.

- src/components/ClaudeMdExternalIncludesDialog.tsx: 4 helper updaters
  (acceptProject/User, declineProject/User) take `ProjectConfig` from
  `src/utils/config.ts` instead of `any` (matches the
  `saveCurrentProjectConfig` updater signature).
- src/components/ProviderManager.tsx: env-clearing objects in
  `activateGithubProvider` and the GitHub disable path used
  `undefined as any`. The intent is "delete this key on merge" but the
  zod schema coerces to string. Replaced with
  `undefined as unknown as string` (honest about the lie) and added a
  comment explaining intent.
- src/services/compact/snipCompact.ts: introduced a structural
  `SnipMessage` type for the public function params. Tests construct
  fake messages with `uuid: 'u4'` (not branded UUID), so a strict
  `Message[]` would reject them; the structural type accepts both
  production messages and test fixtures. `snipCompactIfNeeded` is
  generic `<T extends SnipMessage>` so callers get the same shape back.
  Internal content blocks use `unknown` + type-predicate narrowing.
- src/query.ts: yield site for the boundary message casts to `Message`
  since the value is constructed as a real `Message` in production.
- scripts/grpc-cli.ts: proto descriptor, stream, and message callback
  typed as `unknown` / `ClientDuplexStream<unknown, unknown>`. The
  dynamic `.proto` payload is cast once at the boundary into a
  structural shape so per-field access (`text_chunk.text` etc.) is
  tracked explicitly.
- scripts/generate-sdk-types.ts: the JSON-Schema-walking converter
  takes `unknown` everywhere and narrows via typeof / Array.isArray at
  each access. `Map<unknown, string>` for placeholder identity
  comparison, `Set<string | undefined>` no longer used here.

Verification:
- bun run typecheck clean
- bun run lint: 728 warnings (was 782, -54)
- bun run lint:any-budget: 916 / 969 baseline
- bun test src/services/compact: 91 pass / 0 fail

* chore(types): remove 3 eslint-disable no-explicit-any suppressions

Tighten three small call sites where the `any` was used as a lazy
escape hatch and a real type was reachable in scope.

- src/main.tsx: `(global as any).require('inspector')` becomes
  `(global as unknown as { require: (s: string) => { url: () => string } })`.
  The dynamic require is being phased out, but while it remains the
  narrow structural cast documents the shape we actually consume.
- src/ink/reconciler.ts: catch (error: any) → catch (error: unknown);
  `error.code === 'ERR_MODULE_NOT_FOUND'` accesses the code via
  `(error as { code?: unknown }).code`.
- src/components/wizard/WizardProvider.tsx:
  `createContext<WizardContextValue<any> | null>(null)` →
  `createContext<WizardContextValue | null>(null)`. The
  `WizardContextValue` generic already defaults to
  `Record<string, unknown>`, so the `<any>` was redundant; consumers
  that need a tighter type can supply it via
  `useContext<WizardContextValue<T>>`.

Note: most remaining lint warnings come from file-level
`/* eslint-disable @typescript-eslint/no-explicit-any */` blocks at the
top of do-not-touch files (src/types/{message,utils,tools}.ts,
src/constants/querySource.ts, src/entrypoints/sdk/controlTypes.ts) and
from public SDK surface (src/entrypoints/sdk{,.d.ts,v2.ts}). Both are
intentional per the plan.

Verification:
- bun run typecheck clean

* chore(types): refresh any-usage baseline

* fix(types): address PR review feedback on any-reduction tooling

- Pin eslint to resolved version (9.39.5) so lint warning totals stay
  stable across lockfile updates.
- Wire lint:any-budget into the regular `check` gate (was only in
  check:strict), so any-usage regressions block PRs.
- any-usage-report: use lstat and skip symlinks so a self-referencing
  link under src/tests/scripts can't stall the budget scan.
- any-usage-report --check: exit nonzero when the baseline file is
  missing or invalid so the gate can't be silently disabled.
- Remove dangling references to docs/plans/imperative-hugging-sphinx.md
  from eslint.config.js and scripts/any-usage-report.ts headers.
- mcp.ts: defensively skip non-object tool-output entries (primitives
  / null) instead of crashing on boundary data.
- messages.ts: realign the mis-indented tool_search-off branch so the
  blockExtra + return block reads at the right level.

No budget delta (913 -> 913). Typecheck, lint, budget, and the affected
unit tests all green.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(types): narrow any in src/grpc/server.ts (5 warnings)

- protoDescriptor cast: replace `as any` with a structural type that only
  declares the AgentService shape we actually use.
- handleChat: ServerDuplexStream<any, any> -> <unknown, unknown>; the
  clientMessage is narrowed locally where it's read.
- previousMessages: any[] -> Message[] (already imported, already used
  as the type of this.sessions).
- catch (err: any) -> catch (err); err.message is now gated behind an
  instanceof Error check.

Typecheck, lint, budget, and the 2 grpc unit tests all green.
Lint 728 -> 723 (-5). Budget 913 -> 909 (-4).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(types): narrow any in src/utils/knowledgeGraph.ts (18 warnings)

Add explicit LegacyEntity / LegacyRelation / LegacySummary / LegacyData
types so the migration path stops round-tripping through `any`.

- LegacySource.data, mergeLegacySources, normalizeLegacyData,
  readLegacySqliteStore, SqliteReadResult.data, and doMigration all take
  the new LegacyData shape.
- LegacyData.entities is typed `Record<string, LegacyEntity> | LegacyEntity[]`
  because older JSON dumps stored entities as an array. normalizeLegacyData
  converts the array form to the record form so downstream code can keep
  using Object.entries. (Old loop relied on the same `id ?? entryKey`
  fallback, so behavior is preserved; verified by all 20 unit tests.)
- SQLite row accessor now returns `Record<string, unknown>` (was `any[]`);
  per-row readers narrow `row.id`, `row.attributes`, `row.keywords`,
  `row.content` from `unknown` and the existing typeof guards preserve
  the previous fail-soft behavior on malformed rows.

Lint 723 -> 705 (-18). Budget 909 -> 894 (-15). All 20 knowledgeGraph
unit tests pass; typecheck clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(types): narrow SDK tool() generic and handler signature (12 warnings)

The public SDK MCP tool() helper previously defaulted its Schema generic
to `any` and typed handler args as `any`. Switch both to `unknown`:

- SdkMcpToolDefinition<Schema = unknown>
- tool<Schema = unknown> with handler (args: Schema, extra: unknown)
- annotations?: any -> ToolAnnotations (imported from MCP SDK types)

This is a strictly tighter API: callers who pass a concrete schema get
the same behavior; callers who relied on implicit any now have to
validate handler args (which they should have been doing anyway).

All 4 SDK MCP tool wiring tests pass; typecheck clean.
Lint 705 -> 695 (-10). Budget 894 -> 884 (-10).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(types): address PR feedback on SDK + grpc type hardening

Apply 4 of 5 CodeRabbit review items (skipped item 5: SDK resume flow
JSONL validation, deferred as larger refactor).

- sdk.d.ts: switch MCP type imports to the package's public
  `@modelcontextprotocol/sdk/types.js` subpath (was the internal
  dist/esm/types.js path).
- sdk.d.ts: handler return Promise<any> -> Promise<CallToolResult> in
  both SdkMcpToolDefinition and the tool() helper, matching the
  runtime contract and the sibling declarations in sdk/index.ts and
  sdk/v2.ts.
- grpc/server.ts: introduce a local ClientMessage interface (matching
  the openclaude.proto oneof payload) and type the duplex-stream
  request as ServerDuplexStream<ClientMessage, unknown>. The handler
  body was already reading these fields by name; the type now
  documents them.
- grpc/server.ts: ServiceDefinition generic was being passed a
  ServerDuplexStream, which isn't a valid T (T must be a record
  keyed by method name). Switch to ServiceDefinition<{ Chat: never }>
  so the call to addService accepts a real implementation-map key.

Typecheck, lint, budget, and the 6 affected unit tests all green.
Lint 695 -> 693 (-2). Budget 884 -> 882 (-2).

* fix(types): address 6 reviewer findings on PR #2208

Six findings (four P2, two P3) from the second review pass.

1. knowledgeGraph legacy array normalization: previously dropped entries
   whose `id` was neither string nor number, breaking recovery for
   idless array entries. Use the array index as the fallback key so
   mergeLegacySources' existing `id ?? entryKey` logic continues to
   recover the entry. All 20 unit tests still pass.

2. SDK tool() generic conflated schema descriptor with invocation
   value type, breaking the test fixture (and any external consumer)
   that declares the handler with a concrete value type. Restore
   `args: any` for the handler across the factory, SdkMcpToolDefinition,
   and the .d.ts declaration; keep `Schema` as the generic for
   inputSchema typing.

3. sdk.d.ts now declares ToolAnnotations and CallToolResult locally
   instead of importing from @modelcontextprotocol/sdk, preserving
   the optional peer contract. A consumer without MCP installed no
   longer gets TS2307 on a no-op SDKSession type import.

4. any-usage-report now strips comments, string literals, and
   template-literal contents before applying the annotation regex,
   so `// example: any`, `': any'`, and similar non-type occurrences
   no longer count. @ts-ignore / @ts-expect-error / @ts-nocheck
   directives are preserved for the ignore category. Budget went
   from 882 to 776.

5. generate-sdk-types: schemas module cast as Record<string, () => unknown>
   was wrong because HOOK_EVENTS is a non-callable; cast as
   Record<string, unknown> and narrow per-access. def.getter() and
   Reflect.get(schema, 'description') both narrowed to a callable /
   object shape before use. Generated output unchanged (diff empty).

6. any-usage-report walk(): readdir failures now propagate instead
   of being silently swallowed, so a partial scan cannot publish a
   too-low replacement baseline or pass --check.

Typecheck, lint, budget, and 26 affected unit tests all green.

* chore(types): refresh any-usage baseline after upstream rebase

Baseline 776 -> 779. The +3 comes from upstream #2196 (Command Code
gateway) test mocks merged to main after the baseline was recorded:
2x (payload: any) in ProviderManager.test.tsx, 1x 'as any' in
providerProfiles.test.ts. Symptom-level narrowing of another merged
PR's test fixtures is out of scope; re-baseline to the rebased tree.

---------

Co-authored-by: Gravirei <gravirei@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-21 09:08:34 +08:00
0xfandom 6c7a12b2a2 fix(websearch): reject non-positive WEB_CUSTOM env overrides (#2124)
The custom web-search provider read WEB_CUSTOM_TIMEOUT_SEC and
WEB_CUSTOM_MAX_BODY_KB with Number(x) || DEFAULT. That idiom only
rescues 0 and NaN — a negative or Infinity value is truthy and passes
straight through. A negative WEB_CUSTOM_MAX_BODY_KB makes the
"body exceeds N bytes" check reject every POST search, and a negative
WEB_CUSTOM_TIMEOUT_SEC drives an immediate abort on every request.

Route both through readPositiveEnvNumber, which falls back to the
default for missing, empty, non-finite, or non-positive input.
2026-08-14 10:13:27 +08:00
JATMN 46e80568be fix(provider): support custom Anthropic bearer auth (#1929)
* fix(provider): support custom Anthropic bearer auth

* feat(provider): add custom Anthropic profile flow

* fix(provider): restore custom Anthropic tokens on startup

* fix(provider): preserve custom Anthropic env setup

* fix(provider): clear stale custom Anthropic tokens

* fix(provider): preserve custom Anthropic API-key env setup

* fix(provider): cover custom Anthropic auth routing

* test(api): isolate custom Anthropic client routing

* test(api): cache-bust client provider imports

* feat(provider): clarify custom provider presets

* fix(provider): address custom Anthropic review feedback

* fix(provider): preserve custom Anthropic headers

* fix(provider): require custom Anthropic token

* fix(provider): isolate custom Anthropic credentials

* fix(provider): guard custom Anthropic setup

* fix(provider): complete custom Anthropic integration

* fix(provider): classify custom Anthropic proxies

* fix(provider): gate proxy cache extensions

* fix(provider): preserve custom Anthropic isolation

* fix(provider): retain direct proxy model option

* fix(provider): honor custom endpoint boundaries

* fix(provider): keep proxy credentials local

* fix(provider): disable proxy fast mode

* fix(provider): preserve first-party route identity

* fix(provider): isolate custom Anthropic endpoints

* fix(provider): gate remaining first-party features

* fix(provider): isolate custom Anthropic proxy features

* test(web-search): make Brave timeout mock abort-aware

* fix(provider): address custom Anthropic review feedback

* test(provider): cover first-party beta gates

* fix(provider): complete custom Anthropic isolation

* fix(provider): complete custom Anthropic routing

* fix(provider): address custom Anthropic review followups

* fix(provider): close custom Anthropic review gaps

* test(provider): keep custom Anthropic mock helpers isolated

* test(provider): isolate model options gateway mocks

* fix(provider): stabilize custom Anthropic model option display

* fix(provider): address remaining review threads

* fix(provider): synchronize active profile persistence

* fix(provider): preserve custom Anthropic API key auth

* fix(provider): avoid forwarding inherited Anthropic keys

* fix(provider): guard custom auth selection

* fix(provider): require first-party Anthropic port

* test(web-search): avoid duplicate shared lock

* fix(provider): resolve remaining review findings

* fix(provider): harden custom Anthropic routing

* fix(provider): simplify Anthropic thinking gate

* fix(provider): preserve custom proxy routing and secret permissions

* fix(mcp): isolate Claude.ai config cache by provider

* fix(model): keep custom endpoints out of first-party UX

* fix(provider): scope Opus off switch to Anthropic

* fix(provider): disable tool search for custom proxies

* fix(provider): close custom Anthropic review gaps

* fix(provider): reject Anthropic staging custom profiles

* fix(webfetch): classify custom Anthropic endpoints

* fix(provider): block bearer auth at Anthropic origin

* fix(provider): keep custom auth off staging OAuth

* test(provider): strengthen auth regression coverage
2026-07-16 08:14:22 +08:00
Xiang Han f961ae7432 fix: suspend footer work during ctrl-c feedback (#1963) 2026-07-14 22:07:31 +08:00
JATMN 6b2a1d8aef test(websearch): allow shared timeout test lock wait (#1959) 2026-07-14 14:39:00 +08:00
Calandjatmn 3b41cf3adb fix(command-semantics): cover remaining linter runner exits (#1700)
* fix(command-semantics): cover remaining linter runner exits

* fix(command-semantics): handle env prefixes and PowerShell chains

* fix(command-semantics): preserve runner and pipeline failures

* fix(command-semantics): narrow setup and runner failure guards

* fix(command-semantics): catch real setup failure stderr

* fix(command-semantics): preserve setup failures with output

* fix(command-semantics): parse inline env split strings

* fix(command-semantics): inspect Bash merged failure output

* fix(command-semantics): align PowerShell failure parsing

* test(web-search): stabilize Brave timeout assertion

* fix(command-semantics): cover package scripts and wrapper failures

* fix(command-semantics): handle script run forms and PS call operator

* fix(command-semantics): cover package prefixes and npm errors

* fix(command-semantics): stabilize brave timeout and ps flags

* test(websearch): assert brave timeout aborts fetch signal

* fix(command-semantics): handle tsc diagnostic exit 1

* fix(command-semantics): preserve script diagnostics

* fix(command-semantics): guard silent skipped diagnostics

* fix(command-semantics): tighten setup failure guards

---------

Co-authored-by: jatmn <the@jat.mn>
2026-07-08 11:43:50 +08:00
Bogdan e204d5ad36 feat(doctor): add WebSearch backend diagnostics (#1884)
* feat(doctor): add WebSearch backend diagnostics

* fix(doctor): tighten Firecrawl cloud URL diagnostics

* fix(firecrawl): align cloud URL detection

* test(websearch): stabilize Brave timeout assertion

* fix(firecrawl): handle bare cloud host casing

* fix(doctor): align WebSearch auto diagnostics with fallback

* fix(doctor): align custom preset diagnostics
2026-07-08 08:42:34 +08:00
0xfandom 230888181a fix(websearch): match allowed/blocked domains case-insensitively (#1872)
hostMatchesDomain compared the request host against the caller's
allowed_domains / blocked_domains entries with a raw ===/endsWith. The host is
always lowercased (WHATWG new URL().hostname via safeHostname), but the domain
entries come straight from tool input and are never normalized. A capitalized
entry therefore never matched:

- blocked_domains: ['Reddit.com'] silently failed to block reddit.com results.
- allowed_domains: ['GitHub.com'] dropped every legitimate github.com hit.

Hostnames are case-insensitive, so lowercase both operands before comparing.
Lookalike hosts (notexample.com) are still rejected.
2026-07-07 21:57:06 +08:00
Bogdan 8599560b82 fix(websearch): add built-in provider request timeouts (#1874)
* Add timeouts for web search providers

* Fix WebSearch timeout body-stall test

* Address WebSearch timeout review feedback
2026-07-07 21:54:11 +08:00
Kevin CodexandOpenClaude 9755550137 Typecheck/zero tsc errors (#1597)
* ci(typecheck): add error-count ratchet toward zero tsc errors

tsc --noEmit currently reports 697 pre-existing errors (issue #473), so
PRs cannot be gated on a clean typecheck yet. This adds
scripts/typecheck-ratchet.ts and a per-file baseline: CI fails when the
count rises above the baseline (listing exactly which files regressed),
passes at or below it, and --update lowers the baseline to lock in
gains. Wired into pr-checks as its own step; once the baseline reaches
zero the step becomes a plain `bun run typecheck`.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(typecheck): mechanical sweep — 697 → 624 tsc errors

Type-only fixes with no runtime behavior change, except the deliberate
NODE_ENV restorations:

- Restore process.env.NODE_ENV comparisons that the source snapshot had
  baked into the literal "production", making the conditions constant
  (AutoUpdater dev/test skip, useTypeahead, ink devtools injection,
  interactiveHelpers onboarding skip, TestingPermissionTool.isEnabled —
  the last now correctly enables under bun test, +3 tests run green)
- Type stream read helpers in openaiShim/codexShim as
  Bun.ReadableStreamDefaultReadResult<Uint8Array<ArrayBuffer>> and
  annotate throwClassifiedTransportError as never-returning, clearing
  the reader/response undefined cascades (29 errors)
- Delete 14 stale @ts-expect-error directives
- Widen useState/useRef/array generics inferred from null/[] literals
- as-const notification priority/color literals to match Priority
- Accept readonly Tool[] in checkLocalModelContextLoad/getCombinedTools

Baseline lowered via typecheck:ratchet --update; full suite green
(3690 tests), smoke + bundle guard green.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(typecheck): recreate missing modules — 624 → 415 tsc errors

The open snapshot never mirrored ~60 modules; the bundler noop-stubs
them at build time (() => null named exports), so every recreated
module here is runtime-inert by construction: no import-time side
effects, gated features stay off (isAssistantMode/isSkillSearchEnabled
→ false, tools isEnabled → false, dialogs render null), lookups return
empty, telemetry no-ops. Types are honest and derived from importer
usage — no any.

Highlights:
- sdk: runtimeTypes re-exports/aliases, sdkUtilityTypes
  (NonNullableUsage), settingsTypes.generated; coreTypes.generated
  usage fields regenerated as a self-contained structural type (the
  consumer package ships without sdkUtilityTypes/@anthropic-ai/sdk, so
  the generated file must stay dependency-free — generator override
  updated to match, package-consumer-types tests green)
- services: contextCollapse operations/persist/stats, compact
  cachedMicrocompact state/types + reactiveCompact, skillSearch (7
  modules), oauth/types, lsp/types, sessionTranscript
- cli/server/daemon: Transport interface, parseConnectUrl, server/*
  (7), daemon/*, bg/templateJobs/runners; assistant/* (KAIROS), ssh/*
- tools/components: WorkflowTool trio, ReviewArtifact pair,
  OverflowTest/TerminalCapture/VerifyPlanExecution/DiscoverSkills,
  WebBrowserPanel, task dialogs, message variants, ink events/cursor
- types: statusLine, fileSuggestion, notebook, messageQueueTypes;
  SerializedMessage rebuilt as distributed Omit-union so transcript
  guards narrow again; vitest-compat.d.ts mirrors Bun's runtime
  'vitest' → 'bun:test' aliasing
- TS2304 names: ant-model helpers imported from existing antModels.ts,
  inert Ultraplan/Gates/LogoV2 stubs, PromiseWithResolvers local type
- build.ts: ACCEPTABLE_RUNTIME_STUBS emptied — both grandfathered
  bundle-reaching stubs (MonitorMcpDetailDialog,
  VerifyPlanExecutionTool/constants) are now real typed modules, so
  the degrade-on-use debt the guard tracked is retired

Validation: full suite 3690 green, smoke + bundle guard green,
typecheck:type-tests green, sdk package-consumer tests green; baseline
lowered via typecheck:ratchet --update.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(typecheck): reconstruct Message discriminated union — 415 → 342 tsc errors

src/types/message.ts was a stub where all ~40 message type aliases were
'export type X = any'. Bare-any aliases break the one thing the union is
for: narrowing. Type predicates like isHookAttachmentMessage collapsed to
'never' in guard chains, cascading TS2339/TS2345 through utils/messages.ts,
messageFilters.ts, groupToolUses.ts, collapseReadSearch.ts, REPL.tsx,
compact.ts, stopHooks.ts and the message components.

Envelope design (permissive-body discriminated union):
- Each variant declares its literal discriminant(s) — message.type for the
  envelope union (user/assistant/attachment/progress/system), subtype for
  the 17-variant System family — plus the properties constructor functions
  in utils/messages.ts actually populate, with '[key: string]: any' as an
  escape hatch so unreconstructed properties never error.
- UserMessage<C> / AssistantMessage<T> are generic over content shape so
  NormalizedUserMessage / NormalizedAssistantMessage<T> reuse the envelope
  without Omit (Omit over an index-signature type collapses keyof to
  string and silently drops the discriminant, breaking narrowing).
- AssistantMessage.message is a structural AssistantMessageContent<T>, not
  the SDK's BetaMessage: synthetic constructors don't populate every
  SDK-required field (stop_details), and SDK-facing consumers need
  assignability to Record<string, unknown>-style bodies.
- AttachmentMessage<T = Attachment> / ProgressMessage<T = Progress> stay
  generic over their payloads (utils/attachments.ts and Tool.ts types).
- UI wrappers (GroupedToolUseMessage, CollapsedReadSearchGroup,
  CollapsibleMessage, RenderableMessage) and stream/control envelopes
  (StreamEvent over BetaRawMessageStreamEvent, RequestStartEvent,
  TombstoneMessage, ToolUseSummaryMessage) reconstructed from call sites.
- logs.ts SerializedMessage switched from the Omit<Message, never> trick
  (only sound against an any stub) to an Extract-based distributed union,
  keeping TranscriptMessage assignable to Message.

All other touched files are type-level-only adjustments (annotations on
evolving arrays that inferred never[], predicate types, casts in SDK wire
adapters and test fixtures) — no runtime logic changed anywhere; the full
bun test suite passes 3690/0 before and after.

Result: 415 → 342 tsc errors, every never-cascade in the message pipeline
resolved, no file above its per-file baseline (ratchet updated).

Part of issue #473.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(typecheck): narrow unknowns and fix signature drift — 342 → 94 tsc errors

Clears every remaining non-test error. Honest fixes dominate: evolving
array/let/useState/useRef annotations (the repo's noImplicitAny:false
disables evolving types), real type guards over unknown wire payloads,
hoisted react-compiler-style params annotated with their components'
real Props, and callee signature corrections (useRegisterOverlay
optional param, generic useVoiceState<T>, growthbook shim's accepted
refresh-interval param) that each cleared several call sites. Targeted
reason-commented casts only at SDK/stub/wire boundaries; no any, no
new suppressions.

Runtime deviations are confined to already-broken paths: benchmark.ts
imported a function name that never existed (module-load crash),
caches.ts called stub methods unguarded (TypeError for ant-gated
users), messageActions returned undefined from a string function;
CACHE_EDITING_BETA_HEADER is a best-effort reconstruction of a
squash-lost constant, reachable only behind feature-gated first-party
paths (flagged for review).

Also: ConnectorTextBlock gains its wire-proven optional signature
field; MCP server factory ambient types gain close(); ink
render-node-to-output's nodeType cast fixed (intersection was
collapsing the intended widening); upstreamproxy relay normalizes the
socket data union.

Validation: full suite 3690 green, smoke + bundle guard green;
remaining 94 errors are all in test files (PR 5). Baseline lowered via
typecheck:ratchet --update.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(typecheck): clean test typing, gate CI on zero tsc errors — 94 → 0

Closes the typecheck burn-down (issue #473): bun run typecheck now
exits 0 across the whole repo and CI fails on any new error.

Test typing: new src/test/typedMocks.ts centralizes the two bun:test
gaps (asMockFetch — Mock<T> lacks fetch.preconnect; callArgs —
argless-signature mocks collapse mock.calls to []). Beyond the
helpers, fixes are honest: discriminated-union narrowing before
member access, fixture typing with boundary casts, assertion-type
corrections, and two tests realigned to production signatures they
had drifted from (requestLogging logApiCallEnd args,
incrementalTokenCounter tokenBudget rename) with identical assert
outcomes. No assertion semantics changed; all touched suites pass.

CI: the ratchet served its purpose and is retired — pr-checks now
runs a plain `bun run typecheck` step; ratchet script and baseline
deleted.

Burn-down summary across the series: 697 → 624 (mechanical sweep) →
415 (recreate ~60 missing modules) → 342 (Message discriminated
union) → 94 (narrowing + signature drift) → 0 (this PR).

Validation: tsc --noEmit exit 0, full suite 3690 green, smoke +
bundle guard green.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(typecheck): reconcile with upstream parallel typecheck fixes

Upstream landed #1591/#1592/#1595 while this series was in flight,
fixing some of the same errors differently. Rebase resolutions prefer
upstream where it is authoritative: their CACHE_EDITING_BETA_HEADER
value ('cache-editing-2025-12-01', unconditional) replaces this
series' feature-gated reconstruction; their cachedMicrocompact stub
shapes (with their new test file) replace ours, with boundary casts in
claude.ts where the stub's unknown[] edits meet the local pinned
delete-edit shape; their reader/ReadResult stream typing in openaiShim
replaces ours. MessageWithoutProgress now matches its name
(Exclude<NormalizedMessage, ProgressMessage>), reconciling upstream's
RenderableMessage GroupingResult with this series' message union; the
@ts-expect-error upstream added for settingsTypes.generated is removed
since the module now exists.

tsc exit 0; full suite 3697 green (incl. upstream's new
cachedMicrocompact tests); smoke + bundle guard green.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

* fix(sdk): keep result usage counters required, fix assistant stub exports

Addresses jatmn's and chioarub's review on the typecheck PR:

1. SDK usage contract restored: the generated result types' usage now
   keeps input_tokens, output_tokens, cache_creation_input_tokens, and
   cache_read_input_tokens as REQUIRED numbers — result messages are
   populated from QueryEngine.totalUsage (initialized from
   EMPTY_USAGE), so they are always present at runtime and strict
   consumers may sum them without undefined guards. The richer nested
   metadata (cache_creation, server_tool_use, service_tier) is modeled
   explicitly instead of hiding behind the index signature; the nested
   objects carry no index signature so the SDK's interface types stay
   assignable. Generator override updated and artifacts regenerated; a
   new package-consumer type test sums the counters and reads the
   nested fields so this contract cannot silently regress. The
   sessionHistory test fixture now carries all four counters, matching
   runtime shape.

2. Assistant install wizard stub mismatch fixed: dialogLaunchers
   imported NewInstallWizard/computeDefaultInstallDir through a module
   shape cast, but the assistant stub only exported default — a
   guaranteed runtime crash if the gated path lit up. The stub now
   provides real typed exports: a wizard that cancels immediately (so
   the launcher resolves null/user-cancelled instead of hanging on an
   empty dialog) and an inert computeDefaultInstallDir; the unsafe
   cast in dialogLaunchers is gone.

Validation: tsc exit 0; full suite 3698 green (incl. the new consumer
counters test); smoke + bundle guard green.

Co-Authored-By: OpenClaude <openclaude@gitlawb.com>

---------

Co-authored-by: OpenClaude <openclaude@gitlawb.com>
2026-06-11 07:59:18 +08:00
JATMNandJATMN 492cde2619 Remediate audit findings, replace vulnerable Firecrawl SDK, and harden release validation (#1030)
* Harden release publish checks and remove vulnerable Firecrawl SDK

Add a post-publish npm verification step to the release workflow so GitHub releases fail if the npm latest tag does not resolve to the expected version within the retry window.

Update dependency pins to remediate the audit findings by moving axios to 1.16.0, upgrading the Anthropic SDK to 0.94.0, and bumping the Bedrock and Vertex wrapper packages so Bun installs dedupe onto the patched SDK.

Replace the @mendable/firecrawl-js dependency with a small in-repo fetch-based Firecrawl client used by WebFetchTool and the Firecrawl web-search provider. Preserve self-hosted support, add transient 502 retry/backoff behavior, and cover the new client with focused tests.

Validation:
- bun test src/tools/firecrawl/client.test.ts src/tools/WebSearchTool/providers/firecrawl.test.ts
- bun run build
- bun run smoke
- packed-install npm audit --omit dev --json returned 0 vulnerabilities

* Harden Bun test isolation for release validation

Fix shared-module test leaks that were breaking providerProfile in the full serialized Bun suite.

- preserve full module surfaces when mocking env/provider modules
- remove unnecessary env/envUtils mocks from user/install surface tests
- use a fresh providerProfile module import for the Codex OAuth cleanup regression
- relax the Windows-only permission assertion in providerProfile tests

Validation:
- bun install --frozen-lockfile
- bun test --max-concurrency=1
- bun run smoke
- bun run build
- npm pack

* Complete execa mock coverage in user test

Fix the remaining cross-file Bun mock leak reported in review by expanding the persisted execa mock in src/utils/user.test.ts to include execaSync.

This keeps later imports that touch secure-storage and exec helpers from failing or hanging when bun test runs files serially after user.test.ts.

Validation:
- bun test src/utils/user.test.ts src/utils/effort.codex.test.ts
- bun test --max-concurrency=1
- bun run build
- bun run smoke
- npm pack

* Preserve full module surfaces in user test mocks

Convert the auth, config, cwd, and execa mocks in src/utils/user.test.ts into pass-through mocks with targeted overrides.

This fixes the remaining Bun process-global mock leakage where later suites could fail or hang after user.test.ts because leaked partial mocks were missing exports such as auth/config helpers or execaSync.

Validation:
- bun test src/utils/user.test.ts src/utils/effort.codex.test.ts
- bun test src/utils/user.test.ts src/utils/openclaudeInstallSurfaces.test.ts
- bun test --max-concurrency=1

* Override ip-address to 10.2.0

Add a top-level override for ip-address and refresh bun.lock so the MCP SDK -> express-rate-limit path resolves to ip-address@10.2.0 instead of 10.1.0.

This keeps the branch's audit-remediation scope aligned with the remaining transitive advisory path without changing the direct MCP SDK pin.

Validation:
- bun pm why ip-address
- bun audit

* fix: use cleanup-safe Firecrawl timeouts

* Isolate attribution settings tests

* test: remove stale provider profile import

---------

Co-authored-by: JATMN <12479882+jatmn@users.noreply.github.com>
2026-06-09 06:19:44 +08:00
Cal 1c279577f9 feat: add .gitattributes to enforce LF line endings (#1550)
* feat: add .gitattributes to enforce LF line endings

Prevents Windows/NTFS contributors with core.autocrlf=true from
accidentally converting the entire codebase CRLF→LF, which produces
PRs with +600k/-600k diffs across 700+ files.

- * text=auto normalizes all text files to LF on checkin
- Explicit binary markers for images, fonts, archives, compiled code
- No code changes — config-only, zero risk

* fix: treat SVGs as text, not binary

SVGs are XML-based vector files and should be diffable/mergeable.
CodeRabbit review: https://github.com/Gitlawb/openclaude/pull/1550#discussion_r4632644506

* fix: renormalize CRLF TypeScript files to LF

The new *.ts text attribute exposed 3 TS files with CRLF in the index.
Renormalizing so a fresh Windows checkout stays clean. Fixes jatmn's P2 finding.
2026-06-06 08:19:05 +08:00
Nik b3b771476d fix(websearch): surface adapter failure when auto mode falls back to native (#994) (#1168)
* fix(websearch): surface adapter failure when auto mode falls back to native (#994)

When `WEB_SEARCH_PROVIDER=auto` and the configured adapter chain fails
on a recoverable error (DuckDuckGo "rate-limited from this network",
adapter timeout, 5xx, etc.), the tool falls through to the native
Anthropic / Codex web-search path silently. The only signal that the
adapter failed is a `console.error` line — it never reaches the tool
result the user sees. On rate-limit-prone networks (datacenter IPs,
VPNs) this manifests as "no results found" with no actionable hint,
exactly the symptom reported in #994.

This change captures the adapter error in `adapterFallthroughNotice`
inside the catch branch and prepends it to the eventual native / Codex
output via a small pure helper, `withAdapterFallthroughNotice`. The
hits-present and native-error paths are unchanged; the helper only
mutates a shallow copy when a notice is set, and is a no-op otherwise.

Result: users on a rate-limited adapter chain who get native results
also see *why* the adapter failed, and users whose native search also
returns nothing finally get the actionable diagnostic (configure
TAVILY_API_KEY / FIRECRAWL_API_KEY / etc.) instead of a silent empty.

Test coverage in WebSearchTool.test.ts asserts the pure-helper
contract: no-op when notice is undefined, prepend-not-mutate when a
notice is provided.

* fix(websearch): narrow #994 fix to the reachable adapter-failure surface

Address @techbrewboss feedback: the previous patch's
`adapterFallthroughNotice` machinery and the
`withAdapterFallthroughNotice` helper were unreachable under the
current provider selection.

`shouldUseAdapterProvider()` and `hasNativeSearchFallback()` are
mutually exclusive in auto mode — when a native path exists
(firstParty/vertex/foundry/Codex) the adapter is never tried, and when
the adapter IS tried (openai-shim providers) there is no native
fallback. So the assignment at `adapterFallthroughNotice = ...` and
both `withAdapterFallthroughNotice(...)` call sites could never fire.

Narrow the PR to the path that #994 actually hits today: an
openai-shim provider (moonshot/minimax/nvidia-nim/github copilot) where
the adapter fails transiently and there is no native fallback. The
existing throw at that branch already surfaces the underlying adapter
error verbatim; extract `buildAdapterUnavailableError(provider, errMsg)`
so it is directly testable and cannot regress, and replace the dead
notice helper + its tests with focused coverage of the reachable
message.

Drop the no-op shallow-copy `withAdapterFallthroughNotice` helper and
its two tests; keep the descriptive error throw as the single,
reachable surfacing path.
2026-05-17 05:35:28 +08:00
JATMN f12eb1c9e8 Harden test isolation for smoke stability (#1192)
* Fix flaky smoke build checks

Replace feature-flag build preprocessing with a Bun onLoad transform so smoke/build no longer rewrites tracked src files while tests may be reading them.

Keep telemetry stubs ahead of the feature transform in both CLI and SDK builds, preserve non-empty text token counts in hybrid context splitting, and make the corrupted Orama stress test assert against the actual project directory used by the test.

Verified with bun run smoke, bun test src/utils/hybridContextStrategy.test.ts, bun test src/utils/knowledgeGraph.stress.test.ts --rerun-each 3, and bun test --max-concurrency=1.

* Harden KnowledgeGraph smoke stress isolation

Give each KnowledgeGraph stress test its own temporary config directory and remove it during teardown so Orama, SQLite, and corrupted-file state cannot bleed between stress cases or later PR test runs.

Reviewed at least 20 open PRs and found the recurring smoke-and-tests failure cluster is the full unit suite, especially KnowledgeGraph corrupted Orama recovery. Verified with bun test src/utils/knowledgeGraph.stress.test.ts --rerun-each 5, bun test --max-concurrency=1, and bun run smoke.

* Harden smoke test isolation

Audit and harden broad smoke-adjacent test suites for process-global leaks, including env/config restoration, shared registry/module mock cleanup, fetch/axios/mock restoration, and global MACRO/platform/sandbox mutations.

Replace fragile render sleeps in interactive tests with output-driven waits, and isolate provider/model/profile tests behind the shared mutation lock so unrelated PRs do not inherit stale process state.

Make SQLite knowledge graph cleanup clear closed on-disk databases before best-effort file cleanup, with coverage for the stale database reset path.

Verified: bun run smoke; bun test --max-concurrency=1; python -m pytest -q python/tests; bun run security:pr-scan -- --base origin/main; bun run test:provider; npm run test:provider-recommendation.

* Harden test isolation across smoke suite

Guard process-global test mutations with the shared mutation lock across env, module mock, config cache, and storage tests.\n\nDeep-copy global config snapshots, restore transient globals precisely, and make plugin/LSP mocks expose compatible export surfaces so concurrent test loading does not poison unrelated suites.\n\nReplace fixed SDK cleanup sleeps with call polling to remove timing sensitivity.\n\nVerification:\n- bun test --max-concurrency=1\n- bun run smoke\n- python -m pytest -q python/tests\n- bun run test:provider\n- npm run test:provider-recommendation\n- bun run security:pr-scan -- --base origin/main\n- git diff --check

* Close remaining test global-state leaks

Guard remaining cache, plugin, console, and VS Code module-mock tests with the shared mutation lock.\n\nThis follow-up audit covers non-env process-global state that can leak across test files: tool schema cache, cache stats tracker state, plugin loader caches, console.error replacement, and VS Code mock.module usage.\n\nVerification:\n- leak-surface scans for env/global/mock.module/cache outliers\n- duplicate top-level mock collision cluster\n- affected tests cluster\n- bun test --max-concurrency=1\n- bun run smoke\n- git diff --check

* Guard remaining mock restore cleanup

Lock tests that call bun:test mock.restore without installing module mocks themselves.\n\nmock.restore is process-global, so these cleanup hooks can still tear down another test file's active module mocks when files run concurrently.\n\nVerification:\n- expanded leak scans for env, globals, module mocks, mock.restore, timers, argv, and caches\n- bun test src/components/useCodexOAuthFlow.test.tsx src/services/github/deviceFlow.test.ts\n- bun test --max-concurrency=1\n- bun run smoke\n- git diff --check

* Harden test isolation for smoke stability

Serialize tests that mutate process-global state behind the shared mutation lock, including process.env, transient globals, global config/cache state, storage mocks, and Bun module mocks.

Add isolated env mutex instances for SDK mutex tests so timeout coverage no longer manipulates the live process-global mutex.

Move top-level mock.module setup behind lock acquisition and restore mocks before releasing locks to prevent cross-file leakage under parallel smoke runs.

Verified with: bun test --max-concurrency=1; bun test; bun run smoke.
2026-05-16 15:18:42 +08:00
JATMN 94e8ff3941 chore: centralize Bun version and refresh CI tool pins (#1171)
* chore: centralize Bun version and refresh CI tool pins

- add .bun-version as the shared Bun source of truth for workflows and Docker builds
- update PR and release workflows to read Bun from bun-version-file
- refresh pinned GitHub Actions and Docker action SHAs to newer low-risk releases
- align contributor docs with Bun 1.3.13 guidance

* test: stabilize reset and provider profile persistence

Harden knowledge graph reset behavior across Windows file-lock scenarios by improving SQLite and JSON reset signaling, preserving a safe JSON source of truth when SQLite cannot be cleared, and adding direct storage regression coverage.

Also centralize deterministic config-home handling for tests, tighten provider profile persistence path resolution and cleanup semantics, isolate environment-sensitive suites with the env mutex, and remove flaky external npx dependency from the SDK consumer type test.

* test: fix Codex OAuth callback flake

Investigate the real provider smoke failure from GitHub Actions and fix the root cause instead of patching the symptom.

- make Codex OAuth callback host explicit and consistent across redirect URI generation and listener binding
- allow safe loopback host overrides for localhost, 127.0.0.1, and ::1
- harden Codex OAuth tests with env/fetch isolation so they do not poison neighboring provider suites
- pin the OAuth callback tests to 127.0.0.1 to avoid localhost IPv4/IPv6 family mismatch flakes in CI

Validated with bun test src/services/api/codexOAuth.test.ts, bun test src/services/api/providerConfig.codexSecureStorage.test.ts, and bun run test:provider.

* test: harden Codex OAuth callback tests

Investigate the recurring provider-smoke OAuth failures across multiple PR runs and fix the flaky callback test design at the root.

- remove the free-port reservation race from Codex OAuth tests
- add bounded callback retry only for loopback listener warm-up during the in-process OAuth test flow
- move ephemeral callback port support into an explicit CodexOAuthService test seam instead of widening production env parsing
- keep runtime callback-port semantics unchanged while adding regression coverage for callback host and port parsing

Validated with targeted Codex OAuth tests and repeated provider-bucket reruns to check for recurring flake.

* test: serialize provider shared-state suites

Fix the recurring provider smoke flake at the root cause by serializing test suites that mutate process.env or globalThis.fetch.

Add a shared test mutation lock and wire it into the provider bucket so Codex OAuth no longer races with unrelated provider/config/openai shim tests under Bun's parallel test execution. Cleanup now releases the lock in finally blocks, and the shared lock waits indefinitely by default to avoid timeout-based CI flakes.

* test: fix smoke root causes and noisy suites

Replace the Codex OAuth test's live loopback listener dependency with an injected listener seam, avoid module-mock leakage across provider suites, and clean up the auth-code listener test setup.

Also harden noisy storage and search tests by asserting expected log output, isolating SQLite masterpiece persistence per test cwd, and removing routine benchmark/stress logging from passing runs.

* build: harden Bun version install in Docker

Validate the repo-tracked .bun-version value before using it in the Docker build stage, strip line endings, and install Bun through a quoted semver-only variable instead of raw shell expansion.

* test: replace flaky conversation arc benchmark

Fix the recurring smoke failure caused by an absolute wall-clock assertion in the normal unit suite. Replace the CI-speed-sensitive conversation arc benchmark with deterministic regression coverage that verifies repeated fact extraction, expected entity shapes, bounded graph growth, and populated-summary behavior.

* test: isolate shared-state smoke suites

* test: restore codex credential mocks between suites

* test: fix shared-state and provider init-order flakes

* test: isolate remaining shared-state smoke suites

Serialize the remaining smoke-sensitive suites that mutate process env, CLAUDE_CONFIG_DIR, fetch, or SDK session globals.

Add shared lock coverage to discovery, agent/skills loading, platform storage, and SDK lifecycle/preserved-segment tests. Restore session and cwd state inside the lock boundary so parallel files cannot leak bootstrap state into knowledge graph and SDK isolation tests.

Validated with repeated smoke and full-suite passes:
- bun run smoke (2x)
- bun test
- bun test --max-concurrency=1
- bun run test:provider
- python -m pytest -q python/tests
- npm run test:provider-recommendation
2026-05-15 13:07:39 +08:00
chioarub 877b4dc886 fix: replace raw abort signal timeouts (#1123) 2026-05-13 11:10:01 +08:00
JATMNandJATMN 6655d470b3 Centralize OpenClaude display name (#1084)
* Centralize OpenClaude display name

Add PRODUCT_DISPLAY_NAME as the shared product display label and use it across prompt identity, onboarding, permission dialogs, hook/agent UI, and tool permission messages.

Update onboarding security copy to a single security note, remove the prompt injection note, and replace remaining user-facing Claude possessives with the product display name where appropriate.

* Restore plan author display name

* Restore onboarding prompt injection warning

Reinstate the two-item security notes layout that was collapsed during the product display name cleanup.

Keep the OpenClaude PRODUCT_DISPLAY_NAME branding while restoring a concise prompt-injection/trusted-code warning that does not depend on the external Claude Code docs link.

Verification: parsed src/components/Onboarding.tsx with TypeScript createSourceFile.

---------

Co-authored-by: JATMN <12479882+jatmn@users.noreply.github.com>
2026-05-11 07:57:21 +08:00
Dan Nakhla 402cd3dbe8 feat(websearch): add first-class Brave adapter; fix Google + Brave presets; restore Exa snippets (#1044)
This PR addresses three real bugs in WebSearch's provider layer plus adds Brave
as a first-class adapter so users with a BRAVE_API_KEY get auto-detection +
auto-chain inclusion (matching the ergonomics of TAVILY_API_KEY, EXA_API_KEY,
etc.).

## 1. New: Brave first-class adapter

`providers/brave.ts` — auto-detects `BRAVE_API_KEY`, slots into the auto chain
between Jina and Bing. Sends the bare token in `X-Subscription-Token` per
Brave's API contract. Mirrors the structure of `tavily.ts` / `bing.ts`.

Brave runs an independent web index (~30B pages), making it a useful
non-Google, non-Bing fallback. Bing's hosted API was sunsetted in Aug 2025
for new users, so Brave is a more practical default fallback in 2026.

## 2. Bug fix: Brave preset sent malformed auth header

The `WEB_PROVIDER=brave` preset in `custom.ts` declared
`authHeader: 'X-Subscription-Token'` but no `authScheme`, so the default
`'Bearer'` scheme prefix kicked in, producing:

    X-Subscription-Token: Bearer <token>      ← wrong, returns 401

Brave's API expects:

    X-Subscription-Token: <token>              ← bare token, no scheme

Fix: declare `authScheme: ''` on the preset; update
`buildAuthHeadersForPreset` to emit a bare token (no leading space) when the
scheme is empty.

## 3. Bug fix: Google preset never worked

`WEB_PROVIDER=google` was wired with `Authorization: Bearer <key>`, but the
Google Custom Search JSON API does not support Bearer auth. It requires:

  - `?key=<API_KEY>`  as a query param
  - `?cx=<ENGINE_ID>` as a query param (Programmable Search Engine ID)

The preset previously had no slot for the engine ID at all, so any user
trying `WEB_PROVIDER=google` hit a 400/401 immediately.

Fix: extend `ProviderPreset` with two minimal fields — `authQueryParam` (key
goes in URL, not header) and `envQueryParams` (additional URL params sourced
from env vars). Rewire the `google` preset to use them; reading
`GOOGLE_CSE_ID` for `cx`.

A clear error fires fast if either `WEB_KEY` or `GOOGLE_CSE_ID` is missing,
instead of silently producing a 400 from upstream.

> Note: Google has announced the Custom Search JSON API will be discontinued
> on 2027-01-01 and is closed to new customers. The fix unbreaks existing
> users for the remaining ~8 months; the README includes a sunset notice and
> recommends Brave/Tavily/Exa for new setups.

## 4. Bug fix: Exa results had empty descriptions

`providers/exa.ts` never passed `contents` in the request body. Per the Exa
docs (https://docs.exa.ai/reference/search-api-guide-for-coding-agents):

  > Use `highlights` for agent workflows. Highlights return 10x fewer
  > tokens with the most relevant excerpts.

Without `contents: { highlights: true }`, the Exa response includes only
`{title, url, id, ...}` — no `text`, no `highlights`, no `summary`. The
adapter was then mapping `r.snippet ?? r.text` (neither field exists in the
default response shape), so every Exa hit came back with
`description: undefined`. Tavily/Brave/DDG all return snippets — Exa was
silently degraded.

Fix: request `contents: { highlights: true }` in the body, and map
`results[].highlights[]` (an array of strings) into the description by
joining up to 3 excerpts with ` … `. Falls back to `text` when present, then
`undefined` if neither field is populated.

## Tests

105 / 105 pass in `src/tools/WebSearchTool/providers/`:

  - `providers/brave.test.ts`         (new) — 7 tests: auth header, mapping,
                                              domain filters, error paths
  - `providers/exa.test.ts`           (new) — 9 tests: contents request shape,
                                              highlights mapping, fallback
                                              chain, error paths
  - `providers/custom.test.ts`        (extended) — 6 new tests covering
                                              `authScheme: ''` (Brave preset),
                                              `authQueryParam` suppression
                                              (Google preset), GOOGLE_CSE_ID +
                                              WEB_KEY fail-fast errors, full
                                              request shape via mocked fetch

205 / 205 pass across `src/tools/`. tsc clean for changed files.

## Docs

  - `README_SEARCH_PROVIDERS.md` — promotes `BRAVE_API_KEY` to first-class,
    documents `GOOGLE_CSE_ID` + sunset notice, fixes the provider table,
    updates the auto-chain priority list and mode list
  - `.env.example` — adds `BRAVE_API_KEY` line, documents `GOOGLE_CSE_ID`
    requirement + sunset notice, updates auto-chain priority comment

## Migration / behavior changes

  - `WEB_PROVIDER=google` users must now set `GOOGLE_CSE_ID` (was previously
    unable to function at all, so this is a strict improvement).
  - `WEB_PROVIDER=brave` users with `WEB_AUTH_SCHEME=""` workarounds can
    drop the workaround — the preset now emits a bare token by default.
  - Brave joins the auto-chain priority order
    (firecrawl → tavily → exa → you → jina → **brave** → bing → mojeek →
    linkup → ddg).
2026-05-07 21:31:56 +08:00
0xfandom 1c746750f6 fix(web-search): surface diagnostic when adapter returns 0 hits and no native fallback (#1006)
For openai-shim providers (minimax, moonshot, nvidia-nim, github
copilot, etc.) hasNativeSearchFallback() is false, so when the
DuckDuckGo adapter returns 0 hits in auto mode the call() path
silently falls through to the native Anthropic web_search_20250305
tool. Those providers don't support that tool, producing a silent
"Did 0 searches" with no signal that the default DDG backend is
rate-limited or that no API-key backend is configured.

Convert the silent fallthrough into an actionable result that names
the active provider, the failing backend, and the env vars to set
(FIRECRAWL_API_KEY / TAVILY_API_KEY / EXA_API_KEY / JINA_API_KEY /
BING_API_KEY / MOJEEK_API_KEY / LINKUP_API_KEY / YOU_API_KEY) plus
the native-provider escape hatch (Anthropic / Vertex / Foundry).
Same root-cause family as the catch branch directly below — that
already throws an actionable error for the throw-on-failure case;
this matches it for the success-with-0-hits case.

Auto mode + 0 hits + native fallback available is unchanged: still
falls through so Anthropic/Vertex/Foundry/Codex can serve the
result. Explicit adapter mode is unchanged.

Adds bun:test coverage for the new helpers via the WebSearchTool
__test export.

Refs #614, #994
2026-05-04 20:57:06 +08:00
a133e7631a feat: support self-hosted Firecrawl via FIRECRAWL_API_URL (#949)
* feat: support self-hosted Firecrawl via FIRECRAWL_API_URL

Adds FIRECRAWL_API_URL env var to enable self-hosted Firecrawl
instances. Both WebFetchTool and firecrawl search provider now check
for either FIRECRAWL_API_KEY (cloud) or FIRECRAWL_API_URL (self-hosted).
The FirecrawlClient accepts apiUrl for custom endpoints.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove incorrect non-null assertion on FIRECRAWL_API_KEY

Passing undefined to FirecrawlClient.apiKey is correct when using
FIRECRAWL_API_URL without an API key. Also adds regression tests for
isConfigured() covering all four env combinations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: trigger CI

---------

Co-authored-by: Kunthawat Greethong <kunthawat@gmailcom>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-04 16:47:36 +08:00
Kevin CodexandOpenClaude 531e3f1059 feat(tools): resilient web search and fetch across all providers (#836)
- Add exponential backoff retry to DuckDuckGo adapter (3 attempts with
  jitter) to handle transient rate-limiting and connection errors.
- Add native fetch() fallback in WebFetch when axios hangs with custom
  DNS lookup in bundled contexts.
- Prevent broken native-path fallback for web search on OpenAI shim
  providers (minimax, moonshot, nvidia-nim, etc.) that do not support
  Anthropic's web_search_20250305 tool.
- Cherry-pick existing fixes:
  - a48bd56: cover codex/minimax/nvidia-nim in getSmallFastModel()
  - 31f0b68: 45s budget + raw-markdown fallback for secondary model
  - 446c1e8: sparse Codex /responses payload parsing
  - ae3f0b2: echo reasoning_content on assistant tool-call messages
- Fix domainCheck.test.ts mock modules to include isFirstPartyAnthropicBaseUrl
  and isGithubNativeAnthropicMode exports.

Co-authored-by: OpenClaude <openclaude@gitlawb.com>
2026-04-23 01:14:00 +08:00
KRATOSandClaude Opus 4.7 3c4d8435c4 fix: surface actionable error when DuckDuckGo web search is rate-limited (#834)
Non-Anthropic / non-codex providers (minimax, kimi, generic OpenAI-compatible)
fell through to the DDG adapter when no paid search key was configured. DDG's
scraper is blocked on most IPs, so web_search surfaced an opaque "anomaly in
the request" error. Catch that response in the DDG provider and rethrow with
the exact env vars that would unblock the tool, or the option to switch to a
native-search provider.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-23 00:58:20 +08:00
guanjiawei 2ff5710329 fix retry Codex and OpenAI fetches via proxy-aware helper (#720) 2026-04-16 21:42:14 +08:00
25ce2ca7bf fix: resolve 12 bugs across API, MCP, agent tools, web search, and context overflow (#674)
* fix: resolve 12 bugs across API, MCP, agent tools, web search, and context overflow

API fixes:
- Fix Gemini 400 error: delete 'store: false' field for Gemini endpoints
  (was globally injected, Gemini rejects unknown fields)
- Fix session timeout 500 errors after ~25min: add 120s idle timeout
  on SSE stream readers in openaiShim and codexShim to detect dead
  connections and trigger withRetry reconnection
- Fix context overflow 500 errors: add handler in errors.ts for 500
  responses caused by oversized conversation context (too many tokens),
  surfacing user-friendly message with recovery actions instead of raw
  'API Error: 500'

Agent loop fix:
- Fix premature task completion: detect continuation signals like
  'so now I have to do it' in assistant text without tool calls and
  inject a meta nudge to force the agent to continue

Web search improvements:
- Increase result counts: Bing/Tavily/Exa/Firecrawl from 10→15,
  Mojeek/You/Jina from default→10 (explicit), max_uses 8→15

MCP fixes:
- Reduce default tool timeout from ~27.8 hours to 5 minutes
  (tools no longer hang indefinitely on unresponsive servers)
- Add retry logic (3 attempts) for tools/list fetch failures
  (prevents all MCP tools from silently disappearing on timeout)
- Add abort signal check in URL elicitation retry loop
- Improve MCP error messages with server and tool name context

Agent tool fixes:
- Fix SendMessage race condition: double-check task status before
  auto-resuming stopped agents to prevent duplicate registration
- Fix auto-compact circuit breaker gap: when auto-compact fails 3+
  consecutive times, proactively block oversized context BEFORE the
  API call instead of letting it 500. Clear message with recovery
  instructions (/new, /compact, rewind).

Tests: 850 total, 0 failures (25 new bugfix tests)

* fix: address all 4 review blockers + 6 additional issues from PR #674

Blockers (from Vasanthdev2004 review):

1. Continuation nudge infinite loop — no loop guard
   Added continuationNudgeCount to State, capped at MAX_CONTINUATION_NUDGES (3).
   Counter increments on each nudge, resets on tool execution (next_turn).

2. Continuation signal regexes too broad — high false-positive rate
   Tightened all patterns to require explicit action verbs. Added completion
   marker check (done/finished/completed/summary). Broad patterns only fire
   on messages <80 chars.

3. BUGFIXES.md in repo root — scope contamination
   Removed. PR description already contains this info.

4. AgentTool dump state cleanup is comment-only, not a bug fix
   Wrapped clearInvokedSkillsForAgent and clearDumpState in individual
   try/catch blocks so one failure doesn't prevent the other.

Additional issues:

5+6. readWithTimeout ignores AbortSignal, timer leak on abort
   Added optional signal param to openaiStreamToAnthropic,
   codexStreamToAnthropic, collectCodexCompletedResponse, readSseEvents.
   Added abort listener that clears idle timer so AbortError surfaces
   cleanly instead of spurious idle timeout.

7. MCP error format change breaks consumers
   Reverted human-readable message to original errorDetails format.
   Moved server/tool context to telemetryMessage param only.

10. AgentTool test broken by comment change
   Updated test assertions to match new defensive cleanup text + try/catch.

12. Mojeek test regex dangerously broad
   Tightened to match searchParams.set('t', '10') specifically.

14. linkup.ts in providerCounts test — no result count field
   Removed from providers list (uses depth param, not result count).

15. Error message overlap between errors.ts and query.ts
   Prefixed errorDetails with 'Context overflow (500):' to distinguish.

Tests: 851 pass, 0 fail

---------

Co-authored-by: openclaude-bot <bot@openclaude.ai>
Co-authored-by: Fix Bot <fix@openclaude.dev>
2026-04-14 18:59:53 +08:00
euxaristiaandClaude Opus 4.6 a02c44143b fix(web-search): close SSRF bypasses in custom provider hostname guard (#610)
The previous `isPrivateHostname` used a list of regexes against
`URL.hostname`. Several literal-address forms slipped past it:

- IPv4-mapped IPv6 `[::ffff:127.0.0.1]` (WHATWG URL normalizes to
  `[::ffff:7f00:1]`, which no regex matched) — lets callers reach
  loopback and other private v4 via an IPv6 literal.
- ULA `fc00::/7` (e.g. `[fc00::1]`) — not covered.
- Link-local `fe80::/10` (e.g. `[fe80::1]`) — not covered.
- IPv4 `169.254.0.0/16` (cloud metadata, including 169.254.169.254),
  `100.64.0.0/10` (CGNAT), and the full `0.0.0.0/8` — not covered.
- The IPv6 regex `/^\[::1?\]$/` also required brackets, but `URL.hostname`
  returns bracketed form anyway, so this part happened to work.

WHATWG `new URL(...)` already normalizes short-form / numeric / hex /
octal IPv4 to dotted-quad before we see it, so those cases were in fact
handled — the remaining gaps were IPv6 and a few missing v4 ranges.

Replace the regex list with:
- a dotted-quad IPv4 parser + int range check covering 0/8, 10/8,
  100.64/10, 127/8, 169.254/16, 172.16/12, 192.168/16;
- a small IPv6 parser (handles `::` compression and embedded v4 suffix)
  + a byte-range check covering `::`, `::1`, IPv4-mapped (recursing
  into the v4 classifier), IPv4-compatible, `fc00::/7`, `fe80::/10`,
  and `fec0::/10`.

Export `isPrivateHostname` and add unit tests covering every bypass
listed above plus public-address negatives.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-12 21:09:46 +08:00
euxaristiaandClaude Opus 4.6 7817fe88bd fix(web-search): stop leaking abort listeners in custom provider retry (#611)
`fetchWithRetry` created a fresh `AbortController` per attempt and did:

    signal?.addEventListener('abort', () => controller.abort(), { once: true })

The listener was never removed. Consequences:

- On retry, a second listener was attached to the caller's signal,
  each closing over a different controller.
- After a successful fetch, the listener remained on the caller's
  signal indefinitely, referencing a controller whose work was done.
  For a long-lived caller signal this is a slow leak.
- The `{ once: true }` only helps if the signal actually fires — on
  non-aborted signals the listener stays attached forever.

Replace the manual controller + timer + listener dance with
`AbortSignal.any([signal, AbortSignal.timeout(ms)])`, which the
codebase already uses elsewhere (see src/services/mcp/xaa.ts). This:

- has no user-code listener to leak,
- gives each attempt a fresh independent timeout,
- cleanly distinguishes caller-initiated abort from timeout via
  `signal.aborted` vs `timeoutSignal.aborted` before rewriting the
  error as "Custom search timed out after Ns".

Also resets `lastStatus` per attempt so a 5xx on attempt 0 can't leak
into attempt 1's retry decision, and collapses the two redundant
retry branches (`lastStatus >= 500` and `lastStatus === undefined`)
into one.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-12 19:37:08 +08:00
91e4cfb15b fix: WebSearch providers + MCPTool bugs (#593)
* fix: WebSearch providers + MCPTool bugs

WebSearchTool:
- custom.ts: fix buildAuthHeadersForPreset WEB_AUTH_HEADER opt-out
- custom.ts: fix WEB_AUTH_SCHEME empty string handling
- custom.ts: fix walkJsonPath null safety for jsonPath parsing
- duckduckgo.ts: use SafeSearchType enum instead of raw 0
- mojeek.ts: always send Accept: application/json header
- README: fix timeout documentation (15s -> 120s to match code)
- custom.test.ts: add tests for auth header behavior

MCPTool:
- MCPTool.ts: fix outputSchema to accept ContentBlockParam[] (not just string)
- MCPTool.ts: fix isResultTruncated for array output (iterates text blocks)

* fix: address PR #593 review feedback

1. Export buildAuthHeadersForPreset and add direct tests for:
   - WEB_AUTH_HEADER="" explicit opt-out behavior
   - WEB_AUTH_SCHEME="" stripping scheme prefix
   - Preset defaults (authHeader + authScheme)
   - No WEB_KEY returns empty headers

2. Add duckduckgo.test.ts verifying SafeSearchType.STRICT === 0,
   confirming the enum change is semantically identical to the
   previous raw value.

Addresses review by @Vasanthdev2004 at
pullrequestreview-4093533095

---------

Co-authored-by: FluxLuFFy <flux@openclaude.dev>
Co-authored-by: Fix Bot <fix@openclaude.local>
2026-04-11 21:07:20 +08:00
FluxLuFFyandFluxLuFFy 32fbd0c7b4 fix: custom web search — WEB_URL_TEMPLATE not recognized, timeout too short, silent native fallback (#537)
* fix: custom web search — WEB_URL_TEMPLATE not recognized, timeout too short, silent native fallback

1. custom.ts: Add WEB_URL_TEMPLATE to isConfigured() so the custom provider
   is recognized when configured via URL template alone.

2. custom.ts: Bump DEFAULT_TIMEOUT_SECONDS from 15s to 120s.
   Self-hosted search APIs (SearXNG, internal) commonly need 30-90s.

3. WebSearchTool.ts: When an explicit adapter is selected via
   WEB_SEARCH_PROVIDER=custom, do not silently fall through to the
   native Anthropic path on adapter errors or 0-hit results.
   - 0 hits: return directly (no fallback)
   - Error: throw the real error (no fallback)
   - Auto mode: existing fallback behavior preserved

* fix: tighten auto-mode adapter fallback — only swallow transient errors

Address review feedback: in auto mode, only fall through to native on
transient errors (network failure, timeout, HTTP 5xx). Config and
guardrail errors (SSRF, HTTPS, bad URL, header allowlist, etc.) now
surface properly instead of being silently swallowed.

---------

Co-authored-by: FluxLuFFy <fluxluffy@users.noreply.github.com>
2026-04-09 20:41:58 +08:00
4ad6bc50c1 refactor: provider adapter system + 7 new search providers (bug-fixed) (#512)
* refactor: provider adapter system + 7 new search providers

Architecture:
- Each search backend is a small adapter implementing SearchProvider
- 12 providers: custom, tavily, exa, you, jina, bing, mojeek, linkup, firecrawl, duckduckgo + native
- WEB_SEARCH_PROVIDER controls selection: auto (fallback chain) or specific provider
- Auth always in headers, never in query strings

Bug fixes from review feedback:
- Fix applyDomainFilters catch block: keep hits with malformed URLs on blocked_domains
  (can't confirm blocked), drop on allowed_domains (can't confirm allowed)
- Add safeHostname() helper: safely extract hostname from URLs without throwing
- Replace unsafe new URL(r.url).hostname in 7 providers with safeHostname()
- Remove dead code: buildAllHeaders, buildAuthHeaders, parseExtraHeaders from types.ts
- Fix WEB_PARMS typo: consistently use WEB_QUERY_PARAM everywhere
- AbortSignal forwarded to fetch() in all 12 providers
- DuckDuckGo: wrap dynamic import in try/catch for graceful error
- Exa: remove double domain filtering (server-side already)
- runSearch(): aggregate all provider errors instead of throwing only the last one
- Retry logic: check numeric status code directly, retry 5xx/network, skip 4xx

Test coverage (44 tests, all passing):
- types.test.ts: safeHostname, normalizeHit, applyDomainFilters (20 tests)
- index.test.ts: getProviderMode, getProviderChain, getAvailableProviders (13 tests)
- custom.test.ts: extractHits flexible response parsing (11 tests)

Co-authored-by: FluxLuFFy <195792511+FluxLuFFy@users.noreply.github.com>

* security: add guardrails to custom search provider (Option B)

- HTTPS-only by default (opt-out: WEB_CUSTOM_ALLOW_HTTP=true)
- Private/localhost IPs blocked by default (opt-out: WEB_CUSTOM_ALLOW_PRIVATE=true)
- Header allowlist: only known-safe headers allowed unless WEB_CUSTOM_ALLOW_ARBITRARY_HEADERS=true
- Configurable timeout in seconds (WEB_CUSTOM_TIMEOUT_SEC, default 15)
- Configurable POST body limit (WEB_CUSTOM_MAX_BODY_KB, default 300)
- Removed max URL size restriction
- Audit log warning on first custom search call
- Updated .env.example and README_SEARCH_PROVIDERS.md with all new options

* fix: remove custom provider from auto chain (Option 1)

Remove customProvider from the auto fallback chain so it is only
available when WEB_SEARCH_PROVIDER=custom is explicitly selected.

Changes:
- Remove customProvider from ALL_PROVIDERS array in providers/index.ts
- Add 3 new tests verifying custom is excluded from auto chain
- Update README_SEARCH_PROVIDERS.md: auto priority, mode table, note
- Update .env.example: auto priority comment, custom mode annotation

All 47 tests pass (44 existing + 3 new).

Co-Authored-By: @Vasanthdev2004

* fix: address review blockers (routing, abort, config check, domain matching)

1. Native/Codex routing precedence in auto mode
   shouldUseAdapterProvider() now checks if native/first-party/vertex/foundry
   or Codex paths are available before falling back to adapter providers.
   Auto mode: native paths take precedence; adapter is fallback only.

2. AbortError stops provider chain immediately
   runSearch() now checks for AbortError/aborted signal before continuing
   the fallback chain. Cancelled searches don't create extra outbound requests.

3. Explicit provider mode fails fast on missing credentials
   runSearch() validates isConfigured() for explicit modes before attempting
   requests. Throws clear error: 'Search provider "X" is not configured.'

4. Domain filter exact-or-subdomain matching (fixes suffix collision)
   New hostMatchesDomain() helper: exact match or .subdomain match.
   badexample.com no longer matches example.com.

5. Tests: 56 pass (9 new) covering all 4 fixes

Co-Authored-By: @Vasanthdev2004

---------

Co-authored-by: Claude Fix <fix@openclaude.local>
Co-authored-by: FluxLuFFy <195792511+FluxLuFFy@users.noreply.github.com>
Co-authored-by: bot <bot@openclaw.ai>
2026-04-09 02:51:25 +08:00
Anandanandanandh8x 462a985d7e Remove embedded source map directives from tracked sources (#329)
Inline base64 source maps had been checked into tracked src files. This strips those comments from the repository without changing runtime behavior or adding ongoing guardrails, per the requested one-time cleanup scope.

Constraint: Keep this change limited to tracked source cleanup only
Rejected: Add CI/source verification guard | user requested one-time cleanup only
Confidence: high
Scope-risk: narrow
Reversibility: clean
Directive: If these directives reappear, fix the producing transform instead of reintroducing repo-side cleanup code
Tested: rg -n "sourceMappingURL" ., bun run smoke, bun run verify:privacy, bun run test:provider, npm run test:provider-recommendation
Not-tested: bun run typecheck (repository has many pre-existing unrelated failures)

Co-authored-by: anandh8x <test@example.com>
2026-04-04 21:19:27 +08:00
Meetpatel006 e5c9a6f629 Enable Free DDG WebSearch For Non-Claude Models (#234)
* added duck duck go for websearch tools that allowed free searching

* update readme

* Replace @phukon/duckduckgo-search with duck-duck-scrape and fix Firecrawl routing priority, and add DDG error handling

* refactor: streamline DuckDuckGo search fallback to use Firecrawl directly on rate limit

* docs: update README to clarify DuckDuckGo web search fallback and its limitations with TOS
2026-04-04 09:21:54 +08:00
Leonardo Grigorio ac4efae870 feat: add Firecrawl backend for WebSearch and WebFetch tools
WebSearch is currently disabled for all non-Anthropic providers (OpenAI
shim, DeepSeek, Ollama, etc.) because those providers have no native
search backend. This adds Firecrawl as a fallback that activates when
FIRECRAWL_API_KEY is set, unlocking web search for every model
openclaude supports.

WebFetch uses basic HTTP + Turndown for HTML-to-markdown conversion,
which fails silently on JS-rendered SPAs and bot-protected pages.
Firecrawl scrape replaces the fetch layer when FIRECRAWL_API_KEY is set,
returning clean markdown that handles dynamic content correctly.

Changes:
- WebSearchTool: add runFirecrawlSearch() using @mendable/firecrawl-js,
  respects allowed_domains (post-filter) and blocked_domains (-site: operators),
  includes result snippets alongside links. shouldUseFirecrawl() ensures
  firstParty/Vertex/Foundry/Codex providers keep their native backends.
- WebFetchTool: add scrapeWithFirecrawl(), drops into the existing
  applyPromptToMarkdown() pipeline so prompt processing is unchanged.
- Remove "Web search is only available in the US" restriction from
  prompt when Firecrawl is active (it works globally).
2026-04-02 12:18:20 -03:00
erdemozyol cec3629017 fix: support codex web tools and non-git agents 2026-04-02 14:08:22 +03:00
did:key:z6MkqDnb7Siv3Cwj7pGJq4T5EsUisECqR8KpnDLwcaZq5TPr d2542c9a62 asdf
Squash the current repository state back into one baseline commit while
preserving the README reframing and repository contents.

Constraint: User explicitly requested a single squashed commit with subject "asdf"
Confidence: high
Scope-risk: broad
Reversibility: clean
Directive: This commit intentionally rewrites published history; coordinate before future force-pushes
Tested: git status clean; local history rewritten to one commit; force-pushed main to origin and instructkr
Not-tested: Fresh clone verification after push
2026-03-31 03:34:03 -07:00