What This Builds

This recipe builds a central model hub for a personal or team agent ecosystem:

• One OpenAI-compatible /v1/chat/completions endpoint hosted on Vercel.
• Provider keys stored only in Vercel environment variables, not on every server, laptop, or agent.
• Stable aliases such as gateway/default, gateway/free-code, gateway/nvidia-kimi, and gateway/mistral-large.
• A verified fallback chain, so clients can ask for a capability instead of hard-coding one vendor.
• An admin-only skip header for testing fallbacks without editing code.
• Optional controlled routes for subscription-backed CLI credits, kept out of the public default chain.

Use it when you have several agents running in different places and want a single model-control point:

This is not the paid Vercel AI Gateway credit product. It is a Vercel-hosted gateway that uses your own free-tier provider keys. Vercel provides the serverless runtime; the model capacity comes from provider free tiers, free credits, or already-owned subscription entitlements.

Strict-Free Profile

For a recipe that stays free to run, only put routes in the default chain when they are either:

• free-tier API routes with real generation capacity,
• free-credit routes that do not immediately require a card for basic use,
• no-key free routes used only as a last-resort reserve,
• or same-account free quotas such as Workers AI free allocation.

Keep low-cap daily routers and subscription-backed CLI routes outside gateway/default.

OpenRouter free routes are intentionally low priority unless the account has a funded balance, because unfunded free request caps are too low for agent loops.

CommandCode Go and OpenCode Go routes are useful when you already have those subscriptions, but they are not part of the strict-free default. Put them behind admin headers or forced aliases so general traffic does not burn subscription credits.

Architecture

Agents / scripts / servers
        |
        | OpenAI-compatible request
        v
Vercel Function: /api/v1/chat/completions
        |
        | alias resolution + fallback loop
        v
Provider adapters
        |
        +-- NVIDIA NIM
        +-- Mistral AI
        +-- Cloudflare Workers AI
        +-- Ollama Cloud
        +-- Kilo free route
        +-- optional subscription CLI adapters

The gateway accepts the normal Chat Completions shape:

{
  "model": "gateway/default",
  "messages": [{ "role": "user", "content": "Reply OK only." }],
  "max_tokens": 64,
  "temperature": 0
}

The response stays OpenAI-compatible and adds gateway metadata:

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "OK"
      }
    }
  ],
  "gateway": {
    "requestedModel": "gateway/default",
    "provider": "nvidia-kimi",
    "upstreamModel": "moonshotai/kimi-k2.6",
    "attempts": []
  }
}

Setup Script

Create a minimal Vercel project and push secrets without printing them:

#!/usr/bin/env bash
set -euo pipefail

PROJECT_NAME="${PROJECT_NAME:-agent-ai-gateway}"
KEY_FILE="${KEY_FILE:-./gateway.keys.env}"

if [[ ! -f "$KEY_FILE" ]]; then
  echo "Create $KEY_FILE with provider keys first." >&2
  exit 1
fi

pnpm dlx vercel@48.10.0 link --project "$PROJECT_NAME" --yes

required_env=(
  GATEWAY_ADMIN_TOKEN
  NVIDIA_API_KEY
  MISTRAL_API_KEY
  CLOUDFLARE_API_TOKEN
  CLOUDFLARE_ACCOUNT_ID
  OLLAMA_API_KEY
)

set -a
source "$KEY_FILE"
set +a

for name in "${required_env[@]}"; do
  value="${!name:-}"
  if [[ -z "$value" ]]; then
    echo "skip $name: not set"
    continue
  fi
  printf "%s" "$value" | pnpm dlx vercel@48.10.0 env add "$name" production --sensitive --force
done

pnpm dlx vercel@48.10.0 --prod --yes

Use this shape for gateway.keys.env:

GATEWAY_ADMIN_TOKEN="replace-with-long-random-token"
NVIDIA_API_KEY="..."
MISTRAL_API_KEY="..."
CLOUDFLARE_API_TOKEN="..."
CLOUDFLARE_ACCOUNT_ID="..."
OLLAMA_API_KEY="..."

Do not commit gateway.keys.env.

Client Configuration

Any OpenAI-compatible client can call the gateway:

curl -sS "https://<your-vercel-project>.vercel.app/api/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gateway/default",
    "messages": [{"role":"user","content":"Reply OK only."}],
    "max_tokens": 64,
    "temperature": 0
  }'

For forced routes:

curl -sS "https://<your-vercel-project>.vercel.app/api/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gateway/nvidia-kimi",
    "messages": [{"role":"user","content":"Reply OK only."}],
    "max_tokens": 64
  }'

For fallback testing, use the admin skip header:

curl -sS "https://<your-vercel-project>.vercel.app/api/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-gateway-admin-token: $GATEWAY_ADMIN_TOKEN" \
  -H "x-gateway-skip: nvidia-kimi,nvidia-deepseek" \
  -d '{
    "model": "gateway/default",
    "messages": [{"role":"user","content":"Reply OK only."}],
    "max_tokens": 64
  }'

Skill Included

Pair this recipe with an agent skill that knows how to:

1. read provider keys from the local key store,
2. sync only approved variables to Vercel,
3. smoke-test direct provider routes,
4. smoke-test the public gateway aliases,
5. keep weak, tiny-quota, billing-blocked, or subscription routes out of gateway/default.

The skill should expose three commands:

./scripts/sync-vercel-env.sh
./scripts/smoke-gateway.sh
./scripts/smoke-commandcode-go.sh

Use the skill for operations; use this recipe for product intent, architecture, and setup.

Product Use Cases

Fit And Limits

This fits:

• short model calls from many clients,
• personal infrastructure,
• small teams,
• hobby/free-tier deployments,
• a central control plane for free LLM capacity.

This does not fit:

• high-throughput public APIs,
• workloads that require guaranteed SLOs from free providers,
• large context routing without model-specific budget controls,
• sensitive prompts sent to no-key public routes,
• long-running autonomous agents that need persistent filesystem state.

Operational rules:

• Put rate limits in front of public clients.
• Keep gateway/default boring and reliable.
• Keep experimental models behind forced aliases.
• Re-run smoke tests weekly because free tiers and catalogs drift.
• Record every route as live, valid_key_gated_plan, valid_key_billing_blocked, missing_key, or no_default_chain.

Sources

• Vercel Functions limits: https://vercel.com/docs/functions/limitations
• Vercel Environment Variables: https://vercel.com/docs/environment-variables
• NVIDIA model catalog: https://build.nvidia.com/models
• Mistral usage tiers: https://docs.mistral.ai/admin/user-management-finops/tier
• Cloudflare Workers AI pricing: https://developers.cloudflare.com/workers-ai/platform/pricing/