Migrate from Vercel AI Gateway

Keep your Vercel AI SDK code, add response caching, detailed analytics, and smart routing. One provider for all models.

Let your AI agent do the migration

Copy this prompt into Claude Code, Cursor, or any coding agent — it reads our docs and handles the migration from Vercel AI Gateway for you.

Quick Migration

Swap your provider imports—your AI SDK code stays the same:

- import { openai } from "@ai-sdk/openai";
- import { anthropic } from "@ai-sdk/anthropic";
+ import { generateText } from "ai";
+ import { createOpenAI } from "@ai-sdk/openai";

+ const offrail = createOpenAI({
+   apiKey: process.env.OFFRAIL_API_KEY,
+   baseURL: "https://api.offrail.ai/v1"
+ });

const { text } = await generateText({
-   model: openai("gpt-5.2"),
+   model: offrail("gpt-5.2"),
  prompt: "Hello!"
});

The key difference: one provider, one API key, all models—with caching and analytics built in.

Zero-diff alternative: repoint the base URL

If your app passes bare model strings (model: "anthropic/claude-sonnet-5"), it resolves them through @ai-sdk/gateway — the AI SDK's default provider. OffRail implements that protocol, so you can keep every line of application code and repoint the provider instead:

import { createGateway } from "@ai-sdk/gateway";

globalThis.AI_SDK_DEFAULT_PROVIDER = createGateway({
	baseURL: "https://api.offrail.ai/v4/ai",
	apiKey: process.env.OFFRAIL_API_KEY,
});

No import changes, no model-string changes, and provider-native web search keeps returning source-url parts. See AI SDK Gateway protocol.

Prefer the explicit @ai-sdk/openai migration below when you want to address models by the gateway's own IDs and keep the provider instance under your control.

Migration Steps

Get Your OffRail API Key

Install the OffRail AI SDK Provider

Install the native OffRail provider for the Vercel AI SDK:

pnpm add @ai-sdk/openai

This package provides full compatibility with the Vercel AI SDK and supports all OffRail features.

Update Your Code

Basic Text Generation

// Before (Vercel AI Gateway with native providers)
import { openai } from "@ai-sdk/openai";
import { anthropic } from "@ai-sdk/anthropic";
import { generateText } from "ai";

const { text: openaiText } = await generateText({
	model: openai("gpt-4o"),
	prompt: "Hello!",
});

const { text: claudeText } = await generateText({
	model: anthropic("claude-3-5-sonnet-20241022"),
	prompt: "Hello!",
});

// After (OffRail - single provider for all models)
import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const offrail = createOpenAI({
	apiKey: process.env.OFFRAIL_API_KEY,
	baseURL: "https://api.offrail.ai/v1",
});

const { text: openaiText } = await generateText({
	model: offrail("openai/gpt-4o"),
	prompt: "Hello!",
});

const { text: claudeText } = await generateText({
	model: offrail("anthropic/claude-3-5-sonnet-20241022"),
	prompt: "Hello!",
});

Streaming Responses

import { createOpenAI } from "@ai-sdk/openai";
import { streamText } from "ai";

const offrail = createOpenAI({
	apiKey: process.env.OFFRAIL_API_KEY,
	baseURL: "https://api.offrail.ai/v1",
});

const { textStream } = await streamText({
	model: offrail("anthropic/claude-3-5-sonnet-20241022"),
	prompt: "Write a poem about coding",
});

for await (const text of textStream) {
	process.stdout.write(text);
}

Using in Next.js API Routes

// app/api/chat/route.ts
import { createOpenAI } from "@ai-sdk/openai";
import { streamText } from "ai";

const offrail = createOpenAI({
	apiKey: process.env.OFFRAIL_API_KEY,
	baseURL: "https://api.offrail.ai/v1",
});

export async function POST(req: Request) {
	const { messages } = await req.json();

	const result = await streamText({
		model: offrail("openai/gpt-4o"),
		messages,
	});

	return result.toDataStreamResponse();
}

Alternative: Using OpenAI SDK Adapter

If you prefer not to install a new package, you can use @ai-sdk/openai with a custom base URL:

import { createOpenAI } from "@ai-sdk/openai";
import { generateText } from "ai";

const offrail = createOpenAI({
	baseURL: "https://api.offrail.ai/v1",
	apiKey: process.env.OFFRAIL_API_KEY,
});

const { text } = await generateText({
	model: offrail("openai/gpt-4o"),
	prompt: "Hello!",
});

Update Environment Variables

# Remove individual provider keys (optional - can keep as backup)
# OPENAI_API_KEY=sk-...
# ANTHROPIC_API_KEY=sk-ant-...

# Add OffRail key
export OFFRAIL_API_KEY=orl_your_key_here

Model Name Format

OffRail supports two model ID formats:

Canonical Model IDs (without provider prefix) - Uses smart routing to automatically select the best provider based on uptime, throughput, price, and latency:

gpt-4o
claude-3-5-sonnet-20241022
gemini-1.5-pro

Provider-Prefixed Model IDs - Routes to a specific provider with automatic failover if uptime drops below 90%:

openai/gpt-4o
anthropic/claude-3-5-sonnet-20241022
google-ai-studio/gemini-1.5-pro

For more details on routing behavior, see the routing documentation.

Model Mapping Examples

Vercel AI SDK	OffRail
`openai("gpt-4o")`	`offrail("gpt-4o")` or `offrail("openai/gpt-4o")`
`anthropic("claude-3-5-sonnet-20241022")`	`offrail("claude-3-5-sonnet-20241022")` or `offrail("anthropic/claude-3-5-sonnet-20241022")`
`google("gemini-1.5-pro")`	`offrail("gemini-1.5-pro")` or `offrail("google-ai-studio/gemini-1.5-pro")`

Check the models page for the full list of available models.

Tool Calling

OffRail supports tool calling through the AI SDK:

import { createOpenAI } from "@ai-sdk/openai";
import { generateText, tool } from "ai";
import { z } from "zod";

const offrail = createOpenAI({
	apiKey: process.env.OFFRAIL_API_KEY,
	baseURL: "https://api.offrail.ai/v1",
});

const { text, toolResults } = await generateText({
	model: offrail("openai/gpt-4o"),
	tools: {
		weather: tool({
			description: "Get the weather for a location",
			parameters: z.object({
				location: z.string(),
			}),
			execute: async ({ location }) => {
				return { temperature: 72, condition: "sunny" };
			},
		}),
	},
	prompt: "What's the weather in San Francisco?",
});

This gives you the same managed experience with full control over your infrastructure.

Migrate from Vercel AI Gateway

On this page