Providers

A provider is an LlmClient — the only thing the agent loop needs to talk to a model. Agentoria ships two adapters plus a mock, and anything else that satisfies the interface works.

Anthropic

import { createAnthropicProvider } from '@agentoria/runtime';

const provider = createAnthropicProvider({
  apiKey: process.env.ANTHROPIC_API_KEY!,
  model: 'claude-sonnet-4-6',
  cacheSystem: true, // opt-in prompt caching for the stable system prompt
});

OpenAI-compatible

The OpenAI adapter covers OpenAI and any OpenAI-compatible endpoint — Groq, Together, DeepSeek, Ollama, and Gemini's OpenAI-compat API — via baseURL:

import { createOpenAiProvider } from '@agentoria/runtime';

const openai = createOpenAiProvider({ apiKey: process.env.OPENAI_API_KEY!, model: 'gpt-4o' });

const deepseek = createOpenAiProvider({
  apiKey: process.env.DEEPSEEK_API_KEY!,
  baseURL: 'https://api.deepseek.com/v1',
  model: 'deepseek-chat',
});

const ollama = createOpenAiProvider({ apiKey: 'ollama', baseURL: 'http://localhost:11434/v1', model: 'llama3.1' });

Both adapters retry transient failures (429 / 5xx / network) with exponential backoff and Retry-After support; tune with maxRetries and retryBaseMs.

Testing with a mock

MockLlmClient replays a scripted stream — deterministic tests with no network:

import { MockLlmClient } from '@agentoria/runtime';

const provider = new MockLlmClient([
  { type: 'text', delta: 'Hello ' },
  { type: 'text', delta: 'world' },
]);

Fallback

withFallback returns an LlmClient that tries each provider in order until one starts streaming — useful for provider outages. See Resilience.

import { withFallback } from '@agentoria/runtime';

const provider = withFallback(anthropic, openai); // primary, then backup

Model cascade

withCascade is a peer of withFallback, but the trigger is quality, not a start failure: it tries the cheapest tier first and escalates to a stronger model only when the turn isn't good enough — so simple tasks stay cheap and only the hard ones pay for the strong model.

import { withCascade, judgeAccept } from '@agentoria/runtime';

const client = withCascade([mini, gpt4o, opus], {
  // default gate: escalate on an error or an empty turn.
  // or grade the cheap answer with a cheap model and escalate when it's inadequate:
  accept: judgeAccept({ by: mini }),
});
createAgent({ client, tools, system }); // nothing else changes

The gate is accept(turn, ctx) => boolean — return false to escalate. A tool-deciding turn counts as progress (kept), and the last tier always runs (nothing better to escalate to). Compose it with withFallback (cascade for cost, fallback for availability).

Streaming tradeoff: to judge a cheaper tier before committing, stream() buffers that tier's whole turn, then replays it (accepted) or escalates (rejected) — trading first-token latency for the quality gate. Set bufferStream: false to stream the first tier live and cascade only on complete().

From the environment

cascadeFromEnv reads per-tier models on one adapter — zero extra config is a single model (no cascade, no overhead); set the tiers to turn escalation on:

import { cascadeFromEnv } from '@agentoria/runtime';

// AGENT_MODEL_SMALL=gpt-4o-mini  AGENT_MODEL=gpt-4o  AGENT_MODEL_STRONG=o1
const { provider, models } = cascadeFromEnv(env); // ['gpt-4o-mini','gpt-4o','o1'] → cascade
// with only OPENAI_MODEL set → one client, no cascade