AI Router icon

AI Router

Automatically routes AI tasks to the most appropriate and cost-effective model

AI Router

AI-generated

Summary

Routes an AI prompt to the most suitable model among configured providers based on cost, speed, task affinity, and context size.

Inputs

  • prompt (required) — The user message to send to the AI model.
  • systemPrompt — Optional system-level instruction prepended before the user prompt (e.g., to set persona or output format).
  • temperature — Sampling temperature (0 = deterministic, 2 = creative). Ignored by reasoning models (o3, o4-mini, Claude Opus with extended thinking).
  • mode — Prioritisation strategy: auto (balanced), cost (cheapest viable), local (Ollama only), quality (best available), or speed (fastest).
  • taskHint — Override automatic task detection. Leave as Auto-Detect for heuristic classification.
  • allowedProviders — Which AI providers are eligible for routing. Providers with no API key in credentials are automatically skipped.
  • maxCostPer1k — Hard budget cap (USD per 1K tokens). Models with a blended cost above this value are excluded. Set to 0 for no limit.
  • ollamaModel — The Ollama model name to use. Must be installed locally via 'ollama pull '.
  • ollamaBaseUrl — Base URL of your local Ollama instance.
  • maxTokens — Maximum tokens to generate. Set to 0 for the provider default.
  • fallbackEnabled — Whether to automatically retry with the next-best model when the primary model returns a 429, 5xx, or network error (up to 3 attempts).
  • dryRun — Skip the API call and return only the routing decision (selected model, score, detected task). No tokens are spent.
  • maxItemsPerExecution — Maximum input items to process per run. Set to 0 to process all items.
  • outputModelUsed — Whether to add modelUsed, providerUsed, attemptsTaken, and token counts to the output JSON.
  • outputDetectedTask — Whether to add detectedTask and detectedTaskConfidence to the output.
  • outputScoreBreakdown — Whether to add scoreBreakdown to the output — the top-3 ranked candidate models with their scores and sub-scores.
  • outputEstimatedCost — Whether to add estimatedCostUSD to the output, calculated from token counts and registry pricing.

Output shape

A single record per input item with the model's response text (and optional thinking block).

Includes optional metadata fields (modelUsed, providerUsed, detectedTask, scoreBreakdown, estimatedCostUSD, inputTokens, outputTokens) based on configuration. Estimated cost is calculated from token counts and registry pricing.

Examples

Example 1: Send a coding prompt and automatically route to the best coding model (e.g., GPT-4.1 or Devstral) with cost estimation.

mode=quality, outputEstimatedCost=true, outputModelUsed=true

Example 2: Test routing logic without spending tokens.

dryRun=true, outputDetectedTask=true, outputScoreBreakdown=true

Example 3: Run a local-only task using Ollama to ensure zero cost and privacy.

mode=local, allowedProviders=[ollama]

Links

Discussion