The 2.5-flash-lite model was incorrectly matching the 2.5-flash
case, setting minimal budget to 128. Per Google's docs, flash-lite's
minimum thinking budget is 512, not 128. This caused the error:
The thinking budget 128 is invalid. Please choose a value between
512 and 24576.
Add a dedicated 2.5-flash-lite case before the 2.5-flash case so
the more specific match takes priority.
Fixes#2838
Co-authored-by: JasonOA888 <JasonOA888@users.noreply.github.com>
- Add NON_OVERFLOW_PATTERNS to explicitly exclude known non-overflow errors
- Consolidate Cerebras 400/413 no-body check into OVERFLOW_PATTERNS
- Format Bedrock errors as ${error.name}: ${error.message} for pattern matching
Google's promptTokenCount includes cachedContentTokenCount, so using it
directly as the input token count causes double-counting when calculateCost
multiplies input by the input rate AND cacheRead by the cacheRead rate.
The google-gemini-cli provider already handles this correctly (subtracting
cachedContentTokenCount from promptTokenCount), but google.ts and
google-vertex.ts were using the raw promptTokenCount.
This fix aligns both providers with the google-gemini-cli behavior.
Add an optional requestMetadata field to BedrockOptions that forwards
key-value pairs to the Bedrock Converse API ConverseStreamCommand. Tags
appear in AWS Cost Explorer split cost allocation data, enabling callers
to attribute inference costs to specific applications or contexts.
Changes:
- Add requestMetadata?: Record<string, string> to BedrockOptions with
JSDoc documenting AWS constraints (max 50 pairs, key 64 chars, value
256 chars, no aws: prefix)
- Pass requestMetadata to commandInput via conditional spread to avoid
sending undefined when omitted
- Export BedrockOptions from package root (consistent with other
provider option types)
- Add E2E tests: metadata forwarded to SDK payload, and omitted when
not provided
closes#2510
Add AWS_BEDROCK_FORCE_CACHE=1 environment variable support. When the
model ID doesn't contain a recognizable Claude model name, users can
set this variable to force cache point injection.
Map unknown finish_reason values (e.g. "end" from Ollama/LM Studio) to
"stop" instead of throwing, since assistant content is already produced.
fixes#2142
Add optional `client` field to AnthropicOptions so callers can pass
alternative SDK clients (e.g. AnthropicVertex) that
share the same messages.stream() API. When omitted, behavior is unchanged.
Signed-off-by: sallyom <somalley@redhat.com>
Some OpenAI-compatible providers (e.g., Moonshot/Kimi) return usage
data in chunk.choices[0].usage instead of the standard chunk.usage.
Extract usage parsing into a helper and check choice.usage as fallback.
closes#2017
The OpenAI Chat Completions API standard format for assistant message
content is a plain string. Sending it as an array of
{type:"text", text:"..."} objects causes some models (notably DeepSeek
V3.2 via NVIDIA NIM) to mirror the content-block structure literally
in their output. This produces recursive nesting where each turn wraps
the previous content blocks deeper:
[{'type':'text','text':'[{\'type\':\'text\',\'text\':...}]'}]
The fix unifies the assistant content serialization to always use a
joined string — the same approach already used for the github-copilot
provider — for all openai-completions backends.
Affected models observed: deepseek-ai/deepseek-v3.2 (nvidia provider).
Models like GLM-5, GPT-4, Claude were unaffected as they tolerate
array content, but sending a standard string is safer for all.
Co-authored-by: geraldoaax <geraldoaax@users.noreply.github.com>
Markus Ylisiurunen