When model.maxTokens is 0 (unset/unknown), omit maxTokens from the
Bedrock ConverseStream inferenceConfig instead of sending 0.
Bedrock's inferenceConfig.maxTokens is optional — when omitted, the
model uses its own internal default. This is optimal for Bedrock's
token quota management:
- At request start, Bedrock reserves input_tokens + max_tokens from
your TPM quota
- For Claude 3.7+ models, output tokens have a 5x burndown rate
- Sending an unnecessarily high maxTokens wastes TPM capacity during
the reservation window, reducing concurrent request throughput
- Omitting it lets Bedrock use the model default (~4096 for Claude),
matching expected output sizes and maximizing quota utilization
Also applies the same conditional pattern to temperature — only
include it in inferenceConfig when explicitly set.
Changes:
- simple-options.ts: Use ?? instead of || so explicit 0 is preserved;
return undefined when model.maxTokens is 0 (unset)
- amazon-bedrock.ts: Spread maxTokens and temperature into
inferenceConfig only when defined
Fixes#3399
Add metadata?: Record<string, unknown> to StreamOptions so providers
can extract fields they understand. Anthropic provider extracts user_id
for abuse tracking and rate limiting. Other providers ignore it.
Based on #1384 by @7Sageer, reworked to use a generic type instead of
Anthropic-specific typing on the base interface.
When a provider (e.g., Google Gemini CLI) requests a retry delay longer
than maxDelayMs (default: 60s), the request fails immediately with an
informative error instead of waiting silently for hours.
The error is then handled by agent-level auto-retry, which shows the
delay to the user and allows aborting with Escape.
- Add maxRetryDelayMs to StreamOptions (packages/ai)
- Add maxRetryDelayMs to AgentOptions (packages/agent)
- Add retry.maxDelayMs to settings (packages/coding-agent)
- Update _isRetryableError to match 'retry delay' errors
fixes#1123
Armin Ronacher