feat(ai): sync model reads, explicit async refresh
Provider.getModels() is sync-only (last-known list; must not throw) with an optional refreshModels() where dynamic providers fetch. The sync-or-async union invited latent sync assumptions that would detonate on the first dynamic provider; async-only reads would force sync consumer surfaces (extension find/getAll) through Promises. Sync reads plus an explicit refresh verb keeps the contract single and the staleness visible. Models.getModels()/getModel() are sync best-effort reads; Models.refresh(provider?) rejects with ModelsError(model_source) for a single provider and is concurrent best-effort across all providers. createProvider() takes a models array plus an optional refreshModels fetcher (stored on success, in-flight calls deduped, list unchanged on rejection). forceRefresh options are gone. Also finishes the in-progress AuthStorage fallbackResolver removal (drops the now-unused includeFallback option from getApiKey).
This commit is contained in:
Mario Zechner
@@ -4,13 +4,13 @@
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- The root entrypoint (`@earendil-works/pi-ai`) is now core-only and side-effect free. The old global API moved to the temporary `@earendil-works/pi-ai/compat` entrypoint, a strict superset of the root: switching a file's import path is the only migration step. Moved symbols include `stream`/`complete`/`streamSimple`/`completeSimple`, `getModel`/`getModels`/`getProviders` (now deprecated aliases of `getBuiltinModel`/`getBuiltinModels`/`getBuiltinProviders` from `@earendil-works/pi-ai/providers/all`), `registerApiProvider`/`unregisterApiProviders`/`resetApiProviders`/`getApiProvider`, `getEnvApiKey`/`findEnvKeys`, `setBedrockProviderModule`, the per-API lazy stream wrappers (`anthropicMessagesApi`, ...), and the image-generation API. `/compat` will be removed with the coding-agent ModelManager migration; new code uses `createModels()` and the provider factories instead.
|
||||
- The root entrypoint (`@earendil-works/pi-ai`) is now core-only and side-effect free. The old global API moved to the temporary `@earendil-works/pi-ai/compat` entrypoint, a strict superset of the root: switching a file's import path is the only migration step. Moved symbols include `stream`/`complete`/`streamSimple`/`completeSimple`, `getModel`/`getModels`/`getProviders` (now deprecated aliases of `getBuiltinModel`/`getBuiltinModels`/`getBuiltinProviders` from `@earendil-works/pi-ai/providers/all`), `registerApiProvider`/`unregisterApiProviders`/`resetApiProviders`/`getApiProvider`, `getEnvApiKey`/`findEnvKeys`, `setBedrockProviderModule`, the per-API lazy stream wrappers (`anthropicMessagesApi`, ...), and the image-generation API.
|
||||
- Renamed the `Provider` type to `ProviderId`. `Provider` now names the runtime provider interface (id, name, auth, model listing, stream behavior).
|
||||
- API implementation modules moved from `src/providers/` to `@earendil-works/pi-ai/api/*`, renamed by API id (`anthropic` -> `api/anthropic-messages`, `google` -> `api/google-generative-ai`, `mistral` -> `api/mistral-conversations`, `amazon-bedrock` -> `api/bedrock-converse-stream`), each exporting exactly `stream` and `streamSimple`. The old per-impl export names (`streamAnthropic`, `streamSimpleAnthropic`, ...) are gone; the legacy package subpaths (`./anthropic`, `./google`, ...) keep working and point at the new modules.
|
||||
|
||||
### Added
|
||||
|
||||
- New `Models` runtime: `createModels()` builds an isolated provider collection with async model listing, auth resolution (`getAuth`), and `stream`/`complete`/`streamSimple`/`completeSimple` that resolve auth through the owning provider. `createProvider()` builds providers from parts (single API implementation or a map dispatched on `model.api`); `hasApi()` narrows dynamically listed models.
|
||||
- New `Models` runtime: `createModels()` builds an isolated provider collection with sync model reads (`getModels`/`getModel` return the last-known lists), an explicit async `refresh(provider?)` for dynamic providers, auth resolution (`getAuth`), and `stream`/`complete`/`streamSimple`/`completeSimple` that resolve auth through the owning provider. `createProvider()` builds providers from parts (single API implementation or a map dispatched on `model.api`; static `models` array plus an optional `refreshModels` fetcher with in-flight dedupe); `hasApi()` narrows dynamically listed models.
|
||||
- Provider auth substrate: `ProviderAuth` (`{ apiKey?, oauth? }`), one type-tagged credential per provider, `CredentialStore` (`read`/`modify`/`delete` with serialized writes; in-memory default), `envApiKeyAuth()`, `lazyOAuth()`, and injectable `AuthContext`. OAuth refresh runs under the store lock with double-checked expiry; a stored credential owns its provider (no silent env fallback).
|
||||
- One provider factory per built-in provider under `@earendil-works/pi-ai/providers/*` (e.g. `anthropicProvider()`, `openrouterProvider()`), plus `@earendil-works/pi-ai/providers/all` with `builtinProviders()`/`builtinModels()` and typed `getBuiltin*` catalog reads. Generated catalogs are split per provider, so importing one provider pulls one catalog; `sideEffects` metadata makes the package tree-shakeable.
|
||||
- OAuth flows (Anthropic, OpenAI Codex, GitHub Copilot) gained `OAuthAuth` adapters (`login`/`refresh`/`toAuth`) on unified `prompt()`/`notify()` login callbacks; Copilot's per-credential base URL is derived in `toAuth()`.
|
||||
|
||||
+83
-35
@@ -64,10 +64,21 @@ export interface Provider<TApi extends Api = Api> {
|
||||
readonly auth: ProviderAuth;
|
||||
|
||||
/**
|
||||
* List models. Async and side-effect-free discovery only; provider-specific
|
||||
* model lifecycle (load/unload) belongs in app commands.
|
||||
* Current known models, sync. Static providers return their catalog;
|
||||
* dynamic providers return the list as of the last `refreshModels()`
|
||||
* (empty before the first). Must not throw; `Models` treats a throwing
|
||||
* implementation as having no models.
|
||||
*/
|
||||
getModels(options?: { forceRefresh?: boolean }): Promise<readonly Model<TApi>[]> | readonly Model<TApi>[];
|
||||
getModels(): readonly Model<TApi>[];
|
||||
|
||||
/**
|
||||
* Dynamic providers only: fetch and update the model list. Side-effect-free
|
||||
* discovery (no loading/downloading); provider-specific model lifecycle
|
||||
* belongs in app commands. Concurrent calls share one in-flight fetch.
|
||||
* May reject (network); on rejection the model list stays at its last-known
|
||||
* state and a later call retries.
|
||||
*/
|
||||
refreshModels?(): Promise<void>;
|
||||
|
||||
stream<T extends TApi>(
|
||||
model: Model<T>,
|
||||
@@ -88,19 +99,24 @@ export interface Models {
|
||||
getProvider(id: string): Provider | undefined;
|
||||
|
||||
/**
|
||||
* List models from one provider or all providers. Best-effort aggregation:
|
||||
* provider source failures yield the models that did list (empty for a
|
||||
* single failing provider). Apps that need the failure call
|
||||
* `getProvider(id).getModels()` directly.
|
||||
* Sync read of last-known models from one provider or all providers.
|
||||
* Best-effort: a provider whose `getModels()` throws yields no models.
|
||||
*/
|
||||
getModels(options?: { forceRefresh?: boolean }): Promise<readonly Model<Api>[]>;
|
||||
getModels(provider?: string, options?: { forceRefresh?: boolean }): Promise<readonly Model<Api>[]>;
|
||||
getModels(provider?: string): readonly Model<Api>[];
|
||||
|
||||
/**
|
||||
* Runtime model lookup. Dynamic model lists are typed as `Model<Api>`;
|
||||
* narrow with the `hasApi()` type guard.
|
||||
* Sync runtime model lookup against last-known lists. Dynamic model lists
|
||||
* are typed as `Model<Api>`; narrow with the `hasApi()` type guard.
|
||||
*/
|
||||
getModel(provider: string, id: string, options?: { forceRefresh?: boolean }): Promise<Model<Api> | undefined>;
|
||||
getModel(provider: string, id: string): Model<Api> | undefined;
|
||||
|
||||
/**
|
||||
* Ask dynamic providers to re-fetch their model lists. With a provider id,
|
||||
* rejects with `ModelsError` ("model_source") on that provider's fetch
|
||||
* failure; without one, refreshes all providers concurrently best-effort.
|
||||
* Static providers (no `refreshModels`) are no-ops.
|
||||
*/
|
||||
refresh(provider?: string): Promise<void>;
|
||||
|
||||
/**
|
||||
* Resolve request auth for a model. Includes a source label for status UI.
|
||||
@@ -171,37 +187,48 @@ class ModelsImpl implements MutableModels {
|
||||
return this.providers.get(id);
|
||||
}
|
||||
|
||||
async getModels(
|
||||
providerOrOptions?: string | { forceRefresh?: boolean },
|
||||
maybeOptions?: { forceRefresh?: boolean },
|
||||
): Promise<readonly Model<Api>[]> {
|
||||
const provider = typeof providerOrOptions === "string" ? providerOrOptions : undefined;
|
||||
const options = typeof providerOrOptions === "string" ? maybeOptions : providerOrOptions;
|
||||
|
||||
getModels(provider?: string): readonly Model<Api>[] {
|
||||
if (provider !== undefined) {
|
||||
const entry = this.providers.get(provider);
|
||||
if (!entry) return [];
|
||||
try {
|
||||
return await entry.getModels(options);
|
||||
return entry.getModels();
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
// Async wrapper turns sync throws from ill-behaved providers into rejections.
|
||||
const results = await Promise.allSettled(
|
||||
Array.from(this.providers.values(), async (entry) => entry.getModels(options)),
|
||||
);
|
||||
const models: Model<Api>[] = [];
|
||||
for (const result of results) {
|
||||
if (result.status === "fulfilled") models.push(...result.value);
|
||||
for (const entry of this.providers.values()) {
|
||||
try {
|
||||
models.push(...entry.getModels());
|
||||
} catch {
|
||||
// Best-effort: ill-behaved providers yield no models.
|
||||
}
|
||||
}
|
||||
return models;
|
||||
}
|
||||
|
||||
async getModel(provider: string, id: string, options?: { forceRefresh?: boolean }): Promise<Model<Api> | undefined> {
|
||||
const models = await this.getModels(provider, options);
|
||||
return models.find((model) => model.id === id);
|
||||
getModel(provider: string, id: string): Model<Api> | undefined {
|
||||
return this.getModels(provider).find((model) => model.id === id);
|
||||
}
|
||||
|
||||
async refresh(provider?: string): Promise<void> {
|
||||
if (provider !== undefined) {
|
||||
const entry = this.providers.get(provider);
|
||||
if (!entry?.refreshModels) return;
|
||||
try {
|
||||
await entry.refreshModels();
|
||||
} catch (error) {
|
||||
if (error instanceof ModelsError) throw error;
|
||||
throw new ModelsError("model_source", `Model refresh failed for ${provider}`, { cause: error });
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Cannot reject: the async mapper turns even sync throws from ill-behaved
|
||||
// providers into rejections, and allSettled captures all of them.
|
||||
await Promise.allSettled(Array.from(this.providers.values(), async (entry) => entry.refreshModels?.()));
|
||||
}
|
||||
|
||||
async getAuth(model: Model<Api>): Promise<AuthResult | undefined> {
|
||||
@@ -358,9 +385,16 @@ export interface CreateProviderOptions<TApi extends Api = Api> {
|
||||
headers?: Record<string, string>;
|
||||
/** Required — every provider has auth semantics, even ambient/keyless ones. */
|
||||
auth: ProviderAuth;
|
||||
models:
|
||||
| readonly Model<TApi>[]
|
||||
| ((options?: { forceRefresh?: boolean }) => Promise<readonly Model<TApi>[]> | readonly Model<TApi>[]);
|
||||
/** Initial model list (empty for purely dynamic providers). */
|
||||
models: readonly Model<TApi>[];
|
||||
/**
|
||||
* Dynamic providers: fetch the current list. Stored on success; concurrent
|
||||
* calls share one in-flight fetch. May reject: the stored list then stays
|
||||
* at its last-known state, the rejection propagates to the caller of
|
||||
* `refreshModels()` (wrapped as ModelsError "model_source" by
|
||||
* `Models.refresh(provider)`), and a later call retries.
|
||||
*/
|
||||
refreshModels?: () => Promise<readonly Model<TApi>[]>;
|
||||
/** Single implementation, or map keyed by `model.api` for mixed-API providers. */
|
||||
api: ProviderStreams | Partial<Record<TApi, ProviderStreams>>;
|
||||
}
|
||||
@@ -372,7 +406,9 @@ export interface CreateProviderOptions<TApi extends Api = Api> {
|
||||
* produces a stream error.
|
||||
*/
|
||||
export function createProvider<TApi extends Api = Api>(input: CreateProviderOptions<TApi>): Provider<TApi> {
|
||||
const { models } = input;
|
||||
let models = input.models;
|
||||
let inflightRefresh: Promise<void> | undefined;
|
||||
const refreshModels = input.refreshModels;
|
||||
const single =
|
||||
typeof (input.api as ProviderStreams).stream === "function" ? (input.api as ProviderStreams) : undefined;
|
||||
const byApi = single ? undefined : (input.api as Partial<Record<string, ProviderStreams>>);
|
||||
@@ -398,7 +434,19 @@ export function createProvider<TApi extends Api = Api>(input: CreateProviderOpti
|
||||
baseUrl: input.baseUrl,
|
||||
headers: input.headers,
|
||||
auth: input.auth,
|
||||
getModels: typeof models === "function" ? (options) => models(options) : () => models,
|
||||
getModels: () => models,
|
||||
refreshModels: refreshModels
|
||||
? () => {
|
||||
inflightRefresh ??= (async () => {
|
||||
try {
|
||||
models = await refreshModels();
|
||||
} finally {
|
||||
inflightRefresh = undefined;
|
||||
}
|
||||
})();
|
||||
return inflightRefresh;
|
||||
}
|
||||
: undefined,
|
||||
stream: (model, context, options) => dispatch(model, (streams) => streams.stream(model, context, options)),
|
||||
streamSimple: (model, context, options) =>
|
||||
dispatch(model, (streams) => streams.streamSimple(model, context, options)),
|
||||
@@ -409,7 +457,7 @@ export function createProvider<TApi extends Api = Api>(input: CreateProviderOpti
|
||||
* Runtime-checked narrowing for dynamically looked-up models:
|
||||
*
|
||||
* ```ts
|
||||
* const model = await models.getModel("anthropic", "claude-opus-4-7");
|
||||
* const model = models.getModel("anthropic", "claude-opus-4-7");
|
||||
* if (model && hasApi(model, "anthropic-messages")) {
|
||||
* // model: Model<"anthropic-messages">, stream options fully typed
|
||||
* }
|
||||
|
||||
@@ -72,7 +72,7 @@ describe("lazy provider module loading", () => {
|
||||
const result = runProbe(`
|
||||
const all = await import(${JSON.stringify(providersAllUrl)});
|
||||
const models = all.builtinModels();
|
||||
await models.getModels();
|
||||
models.getModels();
|
||||
`);
|
||||
expect(result.loadedSpecifiers).toEqual([]);
|
||||
});
|
||||
|
||||
@@ -55,7 +55,8 @@ function testProvider(input: {
|
||||
id: string;
|
||||
models?: Model<Api>[];
|
||||
auth?: ProviderAuth;
|
||||
getModels?: () => Promise<readonly Model<Api>[]>;
|
||||
getModels?: () => readonly Model<Api>[];
|
||||
refreshModels?: () => Promise<void>;
|
||||
calls?: ProviderCall[];
|
||||
}): Provider {
|
||||
const models = input.models ?? [testModel(input.id, "model-a")];
|
||||
@@ -72,7 +73,8 @@ function testProvider(input: {
|
||||
id: input.id,
|
||||
name: input.id,
|
||||
auth: input.auth ?? { apiKey: ambientAuth },
|
||||
getModels: input.getModels ?? (async () => models),
|
||||
getModels: input.getModels ?? (() => models),
|
||||
refreshModels: input.refreshModels,
|
||||
stream: (model, _context, options) => respond(model, options as StreamOptions | undefined),
|
||||
streamSimple: (model, _context, options) => respond(model, options as SimpleStreamOptions | undefined),
|
||||
};
|
||||
@@ -127,14 +129,14 @@ describe("Models runtime", () => {
|
||||
models.setProvider(testProvider({ id: "p1", models: [testModel("p1", "m1"), testModel("p1", "m2")] }));
|
||||
models.setProvider(testProvider({ id: "p2", models: [testModel("p2", "m3")] }));
|
||||
|
||||
expect((await models.getModels()).map((m) => m.id)).toEqual(["m1", "m2", "m3"]);
|
||||
expect((await models.getModels("p1")).map((m) => m.id)).toEqual(["m1", "m2"]);
|
||||
expect((await models.getModels("nope")).length).toBe(0);
|
||||
expect((await models.getModel("p2", "m3"))?.id).toBe("m3");
|
||||
expect(await models.getModel("p2", "missing")).toBeUndefined();
|
||||
expect(models.getModels().map((m) => m.id)).toEqual(["m1", "m2", "m3"]);
|
||||
expect(models.getModels("p1").map((m) => m.id)).toEqual(["m1", "m2"]);
|
||||
expect(models.getModels("nope").length).toBe(0);
|
||||
expect(models.getModel("p2", "m3")?.id).toBe("m3");
|
||||
expect(models.getModel("p2", "missing")).toBeUndefined();
|
||||
|
||||
// hasApi() narrows dynamically looked-up models with a runtime check
|
||||
const found = await models.getModel("p2", "m3");
|
||||
const found = models.getModel("p2", "m3");
|
||||
expect(found && hasApi(found, "openai-completions")).toBe(false);
|
||||
expect(found && hasApi(found, "test-api")).toBe(true);
|
||||
if (found && hasApi(found, "test-api")) {
|
||||
@@ -143,48 +145,63 @@ describe("Models runtime", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("swallows provider source failures for both all-provider and single-provider listing", async () => {
|
||||
it("swallows provider source failures for both all-provider and single-provider listing", () => {
|
||||
const models = createModels();
|
||||
models.setProvider(
|
||||
testProvider({
|
||||
id: "broken",
|
||||
getModels: async () => {
|
||||
getModels: () => {
|
||||
throw new Error("boom");
|
||||
},
|
||||
}),
|
||||
);
|
||||
models.setProvider(testProvider({ id: "ok", models: [testModel("ok", "m1")] }));
|
||||
|
||||
expect((await models.getModels()).map((m) => m.id)).toEqual(["m1"]);
|
||||
expect(await models.getModels("broken")).toEqual([]);
|
||||
expect(models.getModels().map((m) => m.id)).toEqual(["m1"]);
|
||||
expect(models.getModels("broken")).toEqual([]);
|
||||
// precise failures come from the provider directly
|
||||
await expect(models.getProvider("broken")?.getModels()).rejects.toThrow("boom");
|
||||
|
||||
// even sync-throwing (non-async) provider implementations are isolated
|
||||
models.setProvider({
|
||||
...testProvider({ id: "sync-broken" }),
|
||||
getModels: () => {
|
||||
throw new Error("sync boom");
|
||||
},
|
||||
});
|
||||
expect((await models.getModels()).map((m) => m.id)).toEqual(["m1"]);
|
||||
expect(() => models.getProvider("broken")?.getModels()).toThrow("boom");
|
||||
});
|
||||
|
||||
it("supports getModels(options) without a provider id", async () => {
|
||||
const seen: ({ forceRefresh?: boolean } | undefined)[] = [];
|
||||
it("refresh() updates dynamic providers; single-provider refresh failures reject", async () => {
|
||||
let list = [testModel("dyn", "before")];
|
||||
let refreshes = 0;
|
||||
const models = createModels();
|
||||
models.setProvider(testProvider({ id: "p1", models: [testModel("p1", "m1")] }));
|
||||
models.setProvider({
|
||||
...testProvider({ id: "p2" }),
|
||||
getModels: async (options) => {
|
||||
seen.push(options);
|
||||
return [testModel("p2", "m2")];
|
||||
},
|
||||
});
|
||||
models.setProvider(
|
||||
testProvider({
|
||||
id: "dyn",
|
||||
getModels: () => list,
|
||||
refreshModels: async () => {
|
||||
refreshes++;
|
||||
list = [testModel("dyn", "after")];
|
||||
},
|
||||
}),
|
||||
);
|
||||
models.setProvider(testProvider({ id: "static", models: [testModel("static", "s1")] }));
|
||||
|
||||
const all = await models.getModels({ forceRefresh: true });
|
||||
expect(all.map((m) => m.id)).toEqual(["m1", "m2"]);
|
||||
expect(seen).toEqual([{ forceRefresh: true }]);
|
||||
expect(models.getModel("dyn", "before")).toBeDefined();
|
||||
await models.refresh("dyn");
|
||||
expect(refreshes).toBe(1);
|
||||
expect(models.getModel("dyn", "after")).toBeDefined();
|
||||
expect(models.getModel("dyn", "before")).toBeUndefined();
|
||||
|
||||
// static providers are no-ops; refresh-all is best-effort
|
||||
await models.refresh("static");
|
||||
await models.refresh();
|
||||
expect(refreshes).toBe(2);
|
||||
|
||||
// single-provider refresh failures reject with ModelsError
|
||||
models.setProvider(
|
||||
testProvider({
|
||||
id: "flaky",
|
||||
refreshModels: async () => {
|
||||
throw new Error("fetch failed");
|
||||
},
|
||||
}),
|
||||
);
|
||||
await expect(models.refresh("flaky")).rejects.toMatchObject({ code: "model_source" });
|
||||
// refresh-all swallows the same failure
|
||||
await expect(models.refresh()).resolves.toBeUndefined();
|
||||
});
|
||||
|
||||
it("resolves auth: stored credential owns the provider, ambient only when nothing stored", async () => {
|
||||
|
||||
@@ -99,7 +99,7 @@ describe("OAuth through Models.getAuth (lazy load chain)", () => {
|
||||
const models = createModels({ credentials });
|
||||
models.setProvider(anthropicProvider());
|
||||
|
||||
const model = (await models.getModels("anthropic"))[0];
|
||||
const model = models.getModels("anthropic")[0];
|
||||
const result = await models.getAuth(model);
|
||||
expect(result?.auth.apiKey).toBe("oauth-access-token");
|
||||
expect(result?.source).toBe("OAuth");
|
||||
@@ -117,7 +117,7 @@ describe("OAuth through Models.getAuth (lazy load chain)", () => {
|
||||
const models = createModels({ credentials });
|
||||
models.setProvider(githubCopilotProvider());
|
||||
|
||||
const model = (await models.getModels("github-copilot"))[0];
|
||||
const model = models.getModels("github-copilot")[0];
|
||||
const result = await models.getAuth(model);
|
||||
expect(result?.auth.apiKey).toBe(access);
|
||||
expect(result?.auth.baseUrl).toBe("https://api.business.githubcopilot.com");
|
||||
|
||||
@@ -26,15 +26,15 @@ describe("builtin providers", () => {
|
||||
expect(providers.length).toBe(builtinProviders().length);
|
||||
expect(providers.map((p) => p.id)).toContain("anthropic");
|
||||
|
||||
const anthropic = await models.getModel("anthropic", "claude-haiku-4-5");
|
||||
const anthropic = models.getModel("anthropic", "claude-haiku-4-5");
|
||||
expect(anthropic?.api).toBe("anthropic-messages");
|
||||
|
||||
const all = await models.getModels();
|
||||
const all = models.getModels();
|
||||
expect(all.length).toBeGreaterThan(500);
|
||||
|
||||
// every provider lists at least one model and owns its models
|
||||
for (const provider of providers) {
|
||||
const list = await models.getModels(provider.id);
|
||||
const list = models.getModels(provider.id);
|
||||
expect(list.length).toBeGreaterThan(0);
|
||||
expect(list.every((m) => m.provider === provider.id)).toBe(true);
|
||||
}
|
||||
@@ -45,7 +45,7 @@ describe("builtin providers", () => {
|
||||
authContext: fakeAuthContext({ ANTHROPIC_API_KEY: "key", ANTHROPIC_OAUTH_TOKEN: "oauth-token" }),
|
||||
});
|
||||
models.setProvider(anthropicProvider());
|
||||
const model = (await models.getModel("anthropic", "claude-haiku-4-5"))!;
|
||||
const model = models.getModel("anthropic", "claude-haiku-4-5")!;
|
||||
|
||||
const result = await models.getAuth(model);
|
||||
expect(result?.auth.apiKey).toBe("oauth-token");
|
||||
@@ -55,7 +55,7 @@ describe("builtin providers", () => {
|
||||
it("reports bedrock as configured from ambient AWS credentials without an api key", async () => {
|
||||
const models = createModels({ authContext: fakeAuthContext({ AWS_PROFILE: "dev" }) });
|
||||
models.setProvider(amazonBedrockProvider());
|
||||
const model = (await models.getModels("amazon-bedrock"))[0];
|
||||
const model = models.getModels("amazon-bedrock")[0];
|
||||
|
||||
const result = await models.getAuth(model);
|
||||
expect(result?.auth).toEqual({});
|
||||
@@ -72,7 +72,7 @@ describe("builtin providers", () => {
|
||||
authContext: fakeAuthContext({ GOOGLE_CLOUD_PROJECT: "proj", GOOGLE_CLOUD_LOCATION: "us-central1" }, [adc]),
|
||||
});
|
||||
configured.setProvider(googleVertexProvider());
|
||||
const model = (await configured.getModels("google-vertex"))[0];
|
||||
const model = configured.getModels("google-vertex")[0];
|
||||
|
||||
const result = await configured.getAuth(model);
|
||||
expect(result?.auth).toEqual({});
|
||||
@@ -180,15 +180,28 @@ describe("createProvider", () => {
|
||||
expect(result.errorMessage).toContain("no API implementation");
|
||||
});
|
||||
|
||||
it("supports async model listers", async () => {
|
||||
it("supports dynamic providers: empty until refreshed, in-flight refreshes deduped", async () => {
|
||||
let fetches = 0;
|
||||
const provider = createProvider({
|
||||
id: "dynamic",
|
||||
auth: { apiKey: { name: "Test", resolve: async () => ({ auth: {} }) } },
|
||||
models: async () => [testModel("api-a", "listed")],
|
||||
models: [],
|
||||
refreshModels: async () => {
|
||||
fetches++;
|
||||
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||
return [testModel("api-a", "listed")];
|
||||
},
|
||||
api: recordingStreams("a", []),
|
||||
});
|
||||
const models = await provider.getModels();
|
||||
expect(models.map((m) => m.id)).toEqual(["listed"]);
|
||||
|
||||
expect(provider.getModels()).toEqual([]);
|
||||
await Promise.all([provider.refreshModels?.(), provider.refreshModels?.()]);
|
||||
expect(fetches).toBe(1);
|
||||
expect(provider.getModels().map((m) => m.id)).toEqual(["listed"]);
|
||||
|
||||
// a later refresh fetches again
|
||||
await provider.refreshModels?.();
|
||||
expect(fetches).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -199,7 +212,7 @@ describe("fauxProvider", () => {
|
||||
models.setProvider(faux.provider);
|
||||
faux.setResponses([fauxAssistantMessage("hello from faux")]);
|
||||
|
||||
const model = (await models.getModels(faux.provider.id))[0];
|
||||
const model = models.getModels(faux.provider.id)[0];
|
||||
const result = await models.completeSimple(model, context);
|
||||
expect(result.stopReason).toBe("stop");
|
||||
expect(result.content).toEqual([{ type: "text", text: "hello from faux" }]);
|
||||
|
||||
@@ -18,7 +18,7 @@ models.setProvider(anthropicProvider());
|
||||
// 2. Look up a model and check auth.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const model = await models.getModel("anthropic", "claude-haiku-4-5");
|
||||
const model = models.getModel("anthropic", "claude-haiku-4-5");
|
||||
if (!model) throw new Error("model not found");
|
||||
|
||||
const auth = await models.getAuth(model);
|
||||
|
||||
Reference in New Issue
Block a user