Add an optional requestMetadata field to BedrockOptions that forwards
key-value pairs to the Bedrock Converse API ConverseStreamCommand. Tags
appear in AWS Cost Explorer split cost allocation data, enabling callers
to attribute inference costs to specific applications or contexts.
Changes:
- Add requestMetadata?: Record<string, string> to BedrockOptions with
JSDoc documenting AWS constraints (max 50 pairs, key 64 chars, value
256 chars, no aws: prefix)
- Pass requestMetadata to commandInput via conditional spread to avoid
sending undefined when omitted
- Export BedrockOptions from package root (consistent with other
provider option types)
- Add E2E tests: metadata forwarded to SDK payload, and omitted when
not provided
closes#2510
Add AWS_BEDROCK_FORCE_CACHE=1 environment variable support. When the
model ID doesn't contain a recognizable Claude model name, users can
set this variable to force cache point injection.
Map unknown finish_reason values (e.g. "end" from Ollama/LM Studio) to
"stop" instead of throwing, since assistant content is already produced.
fixes#2142
Add optional `client` field to AnthropicOptions so callers can pass
alternative SDK clients (e.g. AnthropicVertex) that
share the same messages.stream() API. When omitted, behavior is unchanged.
Signed-off-by: sallyom <somalley@redhat.com>
Some OpenAI-compatible providers (e.g., Moonshot/Kimi) return usage
data in chunk.choices[0].usage instead of the standard chunk.usage.
Extract usage parsing into a helper and check choice.usage as fallback.
closes#2017
The OpenAI Chat Completions API standard format for assistant message
content is a plain string. Sending it as an array of
{type:"text", text:"..."} objects causes some models (notably DeepSeek
V3.2 via NVIDIA NIM) to mirror the content-block structure literally
in their output. This produces recursive nesting where each turn wraps
the previous content blocks deeper:
[{'type':'text','text':'[{\'type\':\'text\',\'text\':...}]'}]
The fix unifies the assistant content serialization to always use a
joined string — the same approach already used for the github-copilot
provider — for all openai-completions backends.
Affected models observed: deepseek-ai/deepseek-v3.2 (nvidia provider).
Models like GLM-5, GPT-4, Claude were unaffected as they tolerate
array content, but sending a standard string is safer for all.
Co-authored-by: geraldoaax <geraldoaax@users.noreply.github.com>
- Map redacted_thinking to ThinkingContent with redacted: true instead of
adding a new content type. The opaque payload goes in thinkingSignature,
thinking text is set to "[Reasoning redacted]" so it renders naturally
everywhere. Cross-model transform drops redacted blocks.
- Skip interleaved-thinking-2025-05-14 beta header for Opus 4.6 / Sonnet 4.6
where adaptive thinking makes it deprecated/redundant.
- Do not send temperature when thinkingEnabled is true (incompatible with
both adaptive and budget-based thinking).
Based on #1665 by @tctev
Z.ai uses the same enable_thinking: boolean parameter as Qwen to control reasoning, not thinking: { type: "enabled" | "disabled" }.
The wrong parameter name means Z.ai ignores the disable request and always runs with thinking enabled, wasting tokens and adding latency.
Merge the Z.ai and Qwen branches since they use the same format.
PR by @okuyam2y
wjonaskr