Anthropic API Compatibility

OffRail provides a native Anthropic-compatible endpoint at /v1/messages that allows you to use any model in our catalog while maintaining the familiar Anthropic API format This is especially useful for applications designed for Claude that you want to extend to use other models.

Overview

The Anthropic endpoint transforms requests from Anthropic's message format to the OpenAI-compatible format used by OffRail, then transforms the responses back to Anthropic's format. This means you can:

• Use any model available in OffRail with Anthropic's API format
• Maintain existing code that uses Anthropic's SDK or API format
• Access models from OpenAI, Google, Cohere, and other providers through the Anthropic interface
• Leverage OffRail's routing, caching, and cost optimization features

Basic Usage

Configuration for Claude Code

This endpoint is perfect for configuring Claude Code to use any model available in OffRail:

export ANTHROPIC_BASE_URL=https://api.offrail.ai
export ANTHROPIC_AUTH_TOKEN=orl_your_api_key_here
# optional: specify a model, otherwise it uses the default Claude model
export ANTHROPIC_MODEL=gpt-5  # or any model from our catalog

# now run claude!
claude

Choosing Models

You can use any model from the models page. Popular options for Claude Code include:

# Use OpenAI's latest model
export ANTHROPIC_MODEL=gpt-5

# Use a cost-effective alternative
export ANTHROPIC_MODEL=gpt-5-mini

# Use Google's Gemini
export ANTHROPIC_MODEL=gemini-3.1-pro-preview

# Use Anthropic's actual Claude models
export ANTHROPIC_MODEL=claude-3-5-sonnet-20241022

Environment Variables

When configuring Claude Code or other Anthropic-compatible applications, you can use these environment variables:

ANTHROPIC_MODEL

Specifies the main model to use for primary requests.

• Default: claude-sonnet-4-20250514
• Example: export ANTHROPIC_MODEL=gpt-5

ANTHROPIC_SMALL_FAST_MODEL

Specifies a smaller, faster model used for background functionality and internal operations.

• Default: claude-3-5-haiku-20241022
• Example: export ANTHROPIC_SMALL_FAST_MODEL=gpt-5-nano

# Example configuration
export ANTHROPIC_BASE_URL=https://api.offrail.ai
export ANTHROPIC_AUTH_TOKEN=orl_your_api_key_here
export ANTHROPIC_MODEL=gpt-5
export ANTHROPIC_SMALL_FAST_MODEL=gpt-5-nano

Advanced Features

Making a manual request

curl -X POST "https://api.offrail.ai/v1/messages" \
  -H "Authorization: Bearer $OFFRAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "max_tokens": 100
  }'

Response Format

The endpoint returns responses in Anthropic's message format:

{
	"id": "msg_abc123",
	"type": "message",
	"role": "assistant",
	"model": "gpt-5",
	"content": [
		{
			"type": "text",
			"text": "Hello! I'm doing well, thank you for asking. How can I help you today?"
		}
	],
	"stop_reason": "end_turn",
	"stop_sequence": null,
	"usage": {
		"input_tokens": 13,
		"output_tokens": 20
	}
}

Request Format

/v1/messages expects Anthropic Messages requests and always answers in Anthropic's format. Because the two formats share model and messages, an OpenAI Chat Completions body can reach this endpoint by accident — and since unknown parameters are ignored rather than rejected, the request succeeds and returns an Anthropic response body that OpenAI SDKs cannot read. If your client reports an empty completion here, check that it is pointed at /v1/chat/completions.

Unknown parameters are deliberately ignored rather than rejected, so a valid Anthropic request is never denied for carrying an extra field. As a consequence, OpenAI-only parameters (response_format, stream_options, max_completion_tokens, n, stop, seed, frequency_penalty, and similar) have no effect here — the model will not honour them. Use /v1/chat/completions if you need them.

A body that is structurally OpenAI is rejected by the schema, as it always has been — OpenAI-shaped tools ({"type": "function", "function": {…}}), OpenAI content parts such as image_url, or assistant turns with content: null. Those rejections now name the mismatch and point at the right endpoint instead of returning an opaque validation error:

{
	"type": "error",
	"error": {
		"type": "invalid_request_error",
		"message": "This endpoint implements Anthropic's Messages API, and the request body uses OpenAI Chat Completions structures (tools[0].function) that Anthropic's format has no equivalent for. Send OpenAI-format requests to /v1/chat/completions instead, or convert the body to Anthropic's Messages format."
	}
}

Rejected requests are recorded in your logs with a client_error finish reason and zero cost, so a malformed client is visible in the activity feed rather than failing silently.

Prompt Caching

For Claude models, cache_control markers on system and message content blocks are forwarded to the provider unchanged, including the optional ttl (5m or 1h):

{
	"model": "claude-sonnet-4-6",
	"max_tokens": 100,
	"system": [
		{
			"type": "text",
			"text": "<several thousand tokens of stable instructions...>",
			"cache_control": { "type": "ephemeral" }
		}
	],
	"messages": [{ "role": "user", "content": "Hello!" }]
}

Cache usage comes back in Anthropic's native fields: usage.cache_creation_input_tokens (tokens written to the cache this request, billed at the write premium), usage.cache_read_input_tokens (tokens served from cache at the discounted rate), and usage.cache_creation (the per-TTL write breakdown).

Web Search

Anthropic's server-side web search tool works on this endpoint. Pass it as usual and the response carries server_tool_use and web_search_tool_result blocks before the text that cites them, so Anthropic SDK clients surface sources:

{
	"model": "claude-haiku-4-5",
	"max_tokens": 400,
	"messages": [{ "role": "user", "content": "What shipped in Node 24?" }],
	"tools": [{ "type": "web_search_20250305", "name": "web_search" }]
}

Replaying the assistant turn verbatim on the next request is supported: the server_tool_use and web_search_tool_result blocks are accepted and dropped, since the provider re-runs the search rather than reusing the previous results.

Tool Search

Anthropic's server-side tool search works on this endpoint. Pass a tool_search_tool_* tool alongside your catalog and mark the tools that should load on demand with defer_loading: true:

{
	"model": "claude-sonnet-4-6",
	"max_tokens": 1024,
	"messages": [{ "role": "user", "content": "What is the weather in Paris?" }],
	"tools": [
		{
			"type": "tool_search_tool_regex_20251119",
			"name": "tool_search_tool_regex"
		},
		{
			"name": "get_weather",
			"description": "Get the weather at a specific location",
			"input_schema": { "type": "object" },
			"defer_loading": true
		}
	]
}

Deferred tools stay out of the rendered tools section, so adding one does not invalidate an existing prompt cache. The response carries the server_tool_use and tool_search_tool_result blocks; replay them verbatim on the next request and Anthropic keeps expanding the tool_reference entries they carry, so Claude reuses a discovered tool instead of searching again. tool_reference blocks returned from your own client-side search inside a tool_result are forwarded unchanged too.

Send every tool definition on every request, including the deferred ones — Anthropic needs them server-side to run the search. At least one tool must stay non-deferred (normally the tool search tool itself), and a tool cannot carry both defer_loading: true and cache_control.

Where it works. Tool search reaches the provider on the Anthropic API and on Anthropic models served through Google Cloud, and it needs a Claude 4.5-generation model or newer — older Claude models reject it upstream. On every other provider, including Anthropic models on AWS Bedrock, the tool search tool and defer_loading are dropped and all tools are sent eagerly. The request still succeeds, it just loses the cache and token savings, so pin the provider (anthropic/claude-sonnet-4-6) when those savings matter.

Gateway Response Cache

If gateway caching is enabled on the project, a byte-identical request is replayed from cache instead of being sent upstream. Because the replayed body is identical (same id, same usage), the x-offrail-cache: HIT response header is the marker to check. Send x-no-cache: true to bypass the cache for a single request.