starred/openclaude
1
0
Fork 0
mirror of https://github.com/Gitlawb/openclaude.git synced 2026-11-06 20:58:14 -04:00
Code Issues Packages Projects Releases Wiki Activity
Files
v0.31.0
openclaude/docs/integrations/common-pitfalls.md
T
0xfandom 06e0ae6e0b feat(settings): per-model context_window and max_output_tokens overrides (#1234) 
* feat(settings): per-model context_window and max_output_tokens overrides

Adds a `modelLimits` settings.json map so users can declare context window
and max output tokens for OpenAI-compatible models that are not in the
built-in catalog. Resolution order is env var → settings → catalog, so the
existing CLAUDE_CODE_OPENAI_CONTEXT_WINDOWS / _MAX_OUTPUT_TOKENS env vars
keep priority. Keys are matched exactly, then by prefix, with optional
"<host>:<model>" host-qualified forms.

Example:
  "modelLimits": {
    "qwen3.6-plus": { "contextWindow": 1048576, "maxOutputTokens": 32768 }
  }

Closes #478

* fix(model-limits): apply settings overrides in resolveModelRuntimeLimits

The settings.json `modelLimits` fallback was only wired into the scalar
getOpenAIContextWindow / getOpenAIMaxOutputTokens helpers. The runtime
resolution path (resolveModelRuntimeLimits) instead consumes the *Matches
variants, which only returned env-var matches — so a configured modelLimits
override never reached the model whose limits are actually resolved at
runtime, contrary to the documented env → settings → catalog order.

Fold the settings lookup into getOpenAIContextWindowMatches /
getOpenAIMaxOutputTokenMatches as a dedicated `settings` field, and insert it
between the exact env override and the built-in catalog in
resolveModelRuntimeLimits. Declare `modelLimits` on GlobalConfig so the lookup
is typed. Add an integration test that drives resolveModelRuntimeLimits and
asserts settings resolve for exact, host-qualified and prefix keys, and that an
env override still wins.

* fix(model-limits): resolve modelLimits from settings.json, not global config

readSettingsLimits read getGlobalConfig().modelLimits, which is ~/.openclaude.json
— a different file from the settings.json the feature is documented and schema'd
for (SettingsSchema in settings/types.ts). A user adding modelLimits to
settings.json saw no effect. Read it from getInitialSettings() (the merged
settings snapshot, session-cached) instead, and drop the now-unused modelLimits
field from GlobalConfig so the override lives in one place. Rewire both test
suites to a gated-passthrough getInitialSettings mock.

* test(model-limits): capture real settings module at load, not in beforeEach

The settings modelLimits suites re-imported the real settings module with
a cold dynamic import inside beforeEach. That import sat right at Bun's
default 5s hook timeout and intermittently failed the first test. Capture
the genuine module once via a top-level query-string-busted static import
so the cost moves to module load (no hook timeout) and the gated
passthrough still bypasses other suites' settings.js mocks. Also correct
the integration-test comment to name the helpers the runtime path
actually calls (getOpenAIContextWindowMatches / getOpenAIMaxOutputTokenMatches).

* fix(model-limits): keep env-prefix override above settings in runtime resolver

resolveModelRuntimeLimits ordered the settings `modelLimits` value above
the env-prefix match, so a broad env-prefix override (e.g.
`{"my-custom":N}`) was silently overtaken by a more specific settings
entry (`my-custom-deployment`). The scalar getOpenAIContextWindow treats
env (exact ?? prefix) as strictly higher priority than settings; mirror
that in the runtime resolver: env.exact, then env.prefix, then settings,
then catalog/cache/descriptor. Regression covers both contextWindow and
maxOutputTokens.

* docs(model-limits): document the settings.json modelLimits override

Add a user-facing section near the env-var overrides covering the
modelLimits map, JSON shape, exact/prefix/host-qualified key matching, and
the env > settings > catalog > descriptor precedence.

* fix(model-limits): keep catalog above env-prefix in runtime precedence

The previous reorder put the env-prefix match above the catalog value,
which broke the existing invariant that a `:cloud` catalog variant takes
its known catalog limit rather than inheriting a broad base-model env
prefix (deepseek-v4-pro:cloud regressed to 262144). Correct order:
exact env -> catalog/cache -> env prefix -> settings -> descriptor. This
still keeps settings strictly below env-prefix (the original drift fix)
while preserving catalog precedence over prefix. Docs precedence updated
to match.

* docs(advanced-setup): document CLAUDE_CODE_OPENAI_MAX_OUTPUT_TOKENS env var

The modelLimits section referenced the max-output env var but the
Environment Variables table only listed the context-window one, leaving
output-only configuration undocumented. Add the matching table row.

Refs #478

* docs(model): align openaiContextWindows precedence comments with the resolver

The module header and the OpenALimitOverrideMatches.settings comment claimed a
resolution order of "env → settings → catalog", which contradicts
resolveModelRuntimeLimits (exact env → catalog/discovery cache → prefix env →
settings modelLimits → descriptor default). Since this module only produces the
override candidates and does not own the precedence, narrow the comments to say
so and point at runtimeMetadata.ts as the authoritative chain, so this
precedence-sensitive code isn't misread when touched again.

* fix(model): rank host-qualified modelLimits keys above bare model keys

lookupByModel grouped all exact matches (host-qualified and bare) ahead of all
prefix matches, so a bare exact key like `qwen3.6-plus` beat a host-qualified
prefix like `openrouter.ai:qwen3`. That defeated the advertised per-endpoint
disambiguation for versioned model families. A host-qualified key is strictly
more specific than a bare one, so rank both host-qualified forms (exact and
prefix) in the high-priority tier ahead of the bare exact match, leaving only
the bare prefix in the low-priority tier.

* fix(model): keep an exact modelLimits key ahead of any host-qualified prefix

The previous commit ranked host-qualified PREFIX matches above bare exact
matches, which broke the deliberate precedence in context.test.ts: an exact
`gpt-4o` limit was overridden by an unrelated `api.foo.com:gpt-4` prefix that
only matches a shorter, different model name.

Restore the tiering so an exact match (host-qualified or bare) always beats a
prefix, and a host-qualified key beats a bare key WITHIN the same match kind.
The supported way to set a different limit for the same model per endpoint is a
host-qualified EXACT key. Narrow the regression + comment to that behavior.

* docs(model): align modelLimits matching wording with exact-over-prefix rule

Narrow the advanced-setup wording so a host-qualified key only wins over a bare
key within the same match kind (a bare exact key still beats a host-qualified
prefix), matching lookupByModel's exact ?? prefix behavior; per-endpoint limits
for the same model need host-qualified exact keys. Also note modelLimits as part
of the documented user-override layer in the integration add-model and
common-pitfalls guides.

* docs(model): clarify modelLimits host-port key and catalog/cache precedence

Spell out that the host-qualified key uses new URL(baseUrl).host — including the
port when present (localhost:4000:my-model, not localhost:my-model) — and split
the precedence line so the built-in catalog is shown as checked before the
discovery-cache value, matching resolveModelRuntimeLimits.
2026-07-09 06:10:49 +08:00

6.6 KiB
Raw Permalink Blame History

Common Pitfalls

Purpose

This is the short pre-PR checklist for descriptor-era integration work.

Use it after reading the architecture note, the relevant how-to guide, and the reference samples.

Pitfall 1: Using the wrong descriptor type

Common mistake: Modeling a route as a gateway or vendor just because it feels close enough.

Safer rule:

  • use VendorDescriptor when the route is the canonical direct vendor API;
  • use GatewayDescriptor when the route hosts, proxies, or aggregates models behind its own endpoint contract;
  • use AnthropicProxyDescriptor when the route accepts Anthropic-native traffic through its own Anthropic-style env contract;
  • use ModelDescriptor for shared model metadata, not route availability.

Pitfall 2: Treating category as the routing contract

Common mistake: Using gateway category to decide runtime behavior.

Safer rule: transportConfig.kind is the routing contract. category is only optional grouping/display metadata.

Pitfall 3: Reintroducing removed gateway fields

Common mistake: Adding fields such as targetVendorId, isOpenAICompatible, or routing-style gateway classification.

Safer rule: Keep routing in transportConfig.kind. Use current descriptor fields from src/integrations/descriptors.ts, not legacy examples from older branches or notes.

Pitfall 4: Calling registry mutation helpers from descriptor files

Common mistake: Using registerGateway(...), registerVendor(...), or registerModel(...) inside contributor-authored descriptor files.

Safer rule: Use the define* helpers from src/integrations/define.ts and default-export the descriptor. Loader-owned registration stays in src/integrations/index.ts.

Pitfall 5: Putting route availability into shared model files

Common mistake: Treating src/integrations/models/*.ts as the main place to say where a model is available.

Safer rule: Shared model descriptors answer what a model is. Route-owned catalogs answer where it is offered.

Pitfall 6: Duplicating model defaults in catalog entries

Common mistake: Marking catalog entries with per-model default or recommended flags after the route already declares defaultModel.

Safer rule: Declare the route's default once with defaultModel. UI recommendation labels derive from that route default.

Pitfall 7: Forgetting providerModelMap boundaries

Common mistake: Assuming providerModelMap enables a route automatically.

Safer rule: Use providerModelMap only to record route-specific API names for the same conceptual model. The route catalog still decides whether that route exposes the model.

Pitfall 8: Omitting openaiShim.maxTokensField on strict routes

Common mistake: Assuming every OpenAI-compatible route accepts the same max-token field.

Safer rule:

  • use max_completion_tokens for newer hosted OpenAI-style contracts;
  • use max_tokens for local or legacy-shaped routes and other providers that reject the newer field;
  • keep the choice explicit in transportConfig.openaiShim.maxTokensField when the route is strict.

Pitfall 9: Flattening real protocol differences

Common mistake: Treating Bedrock, Vertex, Gemini, GitHub native Claude mode, or Mistral as if they were all just generic OpenAI-compatible routes.

Safer rule: If the external API contract is genuinely different, keep that difference explicit. Descriptor-first does not mean protocol differences should be hidden.

Pitfall 10: Overstating /usage support

Common mistake: Declaring usage support because the descriptor schema allows it, without checking the active runtime/UI path.

Safer rule:

  • use usage.supported: true only when the route has real current support;
  • delegate from a gateway to a vendor only when the vendor is the true source of usage data;
  • keep unsupported routes explicit with usage: { supported: false };
  • remember that the current resolver in src/commands/usage/index.ts is still vendor/gateway-focused, with the current settings UI still concrete for Anthropic, MiniMax, and Codex.

Pitfall 11: Hiding discovery complexity in the descriptor file

Common mistake: Packing a large hybrid catalog or complex discovery rules inline in gateways/<id>.ts.

Safer rule: Move large catalog or discovery-specific logic into a companion gateways/<id>.models.ts file and keep the descriptor file small.

Pitfall 12: Rebuilding the old OpenAI context table

Common mistake: Adding built-in context or output limits to src/utils/model/openaiContextWindows.ts.

Safer rule: Put built-in model metadata in src/integrations/models/. Keep openaiContextWindows.ts focused on documented user overrides — the CLAUDE_CODE_OPENAI_CONTEXT_WINDOWS / CLAUDE_CODE_OPENAI_MAX_OUTPUT_TOKENS env vars and the settings.json modelLimits map — not a built-in table.

Pitfall 13: Forgetting the compatibility layer

Common mistake: Changing descriptor metadata and assuming every public surface is already descriptor-native.

Safer rule: If the route should be user-facing in preset flows, add descriptor preset metadata and regenerate the artifacts:

  • bun run integrations:generate
  • src/integrations/generated/integrationArtifacts.generated.ts
  • env-facing flows that still preserve legacy names

Do not hand-edit:

  • src/integrations/compatibility.ts
  • src/integrations/profileResolver.ts
  • src/integrations/providerUiMetadata.ts
  • preset typing or preset ordering tables

Only touch remaining env-facing compatibility surfaces when the route truly needs them.

Pitfall 14: Using stale repo paths in docs

Common mistake: Pointing contributors at outdated files or command entrypoints.

Safer rule: Use the current repo surfaces:

  • src/commands/usage/index.ts for descriptor-backed /usage routing
  • src/components/Settings/Usage.tsx for the current usage UI boundary
  • src/integrations/routeMetadata.ts for route/default/label helpers
  • src/integrations/runtimeMetadata.ts for request-shaping metadata
  • src/integrations/discoveryCache.ts and src/integrations/discoveryService.ts for discovery caching and loading

Final check

Before opening or landing integration docs or descriptor changes:

  • confirm the descriptor type is correct;
  • confirm transportConfig.kind is doing the routing work;
  • confirm examples use define* helpers plus default exports;
  • confirm route catalogs own availability;
  • confirm route defaults are declared once through defaultModel;
  • confirm built-in model limits live in src/integrations/models/;
  • confirm strict OpenAI-compatible routes specify the correct max-token field;
  • confirm /usage docs match the actual current resolver/UI behavior;
  • confirm any illustrative sample is clearly marked as illustrative.
Reference in New Issue View Git Blame Copy Permalink