Inference API

/inference is a unified chat-completions surface over the four model-inference routes — OpenAI, Anthropic, Grok, and Hugging Face. One OpenAI Chat Completions-shaped contract, JSON and SSE alike, addressed by a canonical model id instead of four different native shapes. It is a translation layer, not a fifth upstream: no new upstream key, no separate catalog, no separate capacity pool — it shares the same kill switches, pilot allowlists, BYOK keys, rate limiter, and settlement as the four routes it fronts.

POST /inference/v1/chat/completions   → normalized chat completion (JSON or SSE)
GET  /inference/v1/models             → every model your account can currently reach
GET  /inference/v1/models/<id>        → one model's catalog entry

Canonical model IDs

model is mandatory on every chat request — there is no default and no fallback. Routing from a valid canonical id to a provider and underlying model is deterministic, never inferred:

Only the first / after the provider prefix is significant — everything after it, slashes included, is the underlying provider’s own model id. That’s why the Hugging Face form can itself contain a / (the HF repo’s owner/model) plus an optional :provider suffix, exactly as on the Hugging Face route itself.

An id that doesn’t parse, doesn’t resolve in the underlying provider’s catalog, or names a chat-incapable OpenAI model (embeddings/moderation/realtime-only entries are excluded) is indistinguishable from an id whose provider is currently disabled — all of it is “not available,” fail-closed.

Drop-in for the OpenAI SDK

Because the request and response shapes are OpenAI’s own, official OpenAI client libraries work by pointing base_url at this route and using a canonical id as model:

1
from openai import OpenAI
2
3
client = OpenAI(
4
    api_key="YOUR_API_KEY",
5
    base_url="https://api.nativeport.ai/inference/v1",
6
)
7
8
response = client.chat.completions.create(
9
    model="anthropic/claude-sonnet-5",
10
    messages=[{"role": "user", "content": "Summarize this page in one sentence."}],
11
)
12
print(response.choices[0].message.content)

Swapping model to openai/gpt-5.4-mini, xai/grok-4.3, or huggingface/openai/gpt-oss-120b requires no other code change — the same request body works against any admitted model.

Endpoints

Request fields — the intersection, not the union

The point of a unified endpoint is that one request schema works across any admitted model: the accepted fields are the intersection every backing provider can express — deliberately narrower than any single provider’s own field policy on its native route. A field outside this set is rejected with a fail-closed 400 unsupported_parameter before any upstream call, never silently dropped:

model, messages (text/image_url content parts), max_tokens, max_completion_tokens, temperature, top_p, stop, stream, stream_options (include_usage only), tools/tool_choice (function-type only), parallel_tool_calls.

This exact list is also what GET /inference/v1/models reports per model as supported_parameters — uniform across every listed model, and guaranteed never to drift from what the request validator actually accepts.

Response shape

Regardless of which of the four providers actually served the request, the response is always OpenAI’s chat.completion (or streaming chat.completion.chunk) shape:

• model is rewritten to the canonical id that served the request — never the provider’s own native model identifier.
• usage (prompt_tokens/completion_tokens/total_tokens, plus prompt_tokens_details.cached_tokens when the resolved model reports cache-read tokens) is reported in this one normalized shape even for providers whose native usage field names differ.
• tool_calls are normalized to OpenAI’s {id, type: "function", function: {name, arguments}} shape, arguments always a JSON-encoded string — including for Anthropic, whose native tool_use content blocks are translated on the way out.
• finish_reason is one of stop, length, tool_calls, content_filter — Anthropic’s stop_reason values (end_turn, max_tokens, tool_use, refusal, …) are mapped onto this same set.

Streaming

Set stream: true for text/event-stream instead of a single JSON object: a sequence of data: <chat.completion.chunk JSON> lines separated by blank lines, terminated by a literal data: [DONE] line. Set stream_options.include_usage: true to receive one extra usage-only chunk (empty choices) immediately before [DONE].

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1769434200,"model":"openai/gpt-5.4-mini","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1769434200,"model":"openai/gpt-5.4-mini","choices":[{"index":0,"delta":{"content":"This"},"finish_reason":null}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1769434200,"model":"openai/gpt-5.4-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]

This holds for every provider, including Anthropic — whose native message_start/content_block_*/message_delta/message_stop SSE sequence is translated event-by-event into this same chunk shape, tool-call argument deltas included.

Diagnostic headers

Every response from this endpoint — success or error, GET /inference/v1/models[/{id}] included — carries:

Anthropic’s max_tokens mapping

Claude’s native Messages API requires max_tokens; OpenAI’s Chat Completions API doesn’t. Calling /anthropic/v1/messages directly, an omitted max_tokens is left out and Anthropic’s own 400 stands — a client talking to Claude directly is expected to know the field is mandatory. This endpoint’s contract is different: a request written against one canonical model is supposed to work unchanged against any other, and OpenAI/xAI/Hugging Face chat completions all auto-cap output when a client sends neither max_tokens nor max_completion_tokens. Leaving Anthropic as the one silent exception would be an accidental per-provider asymmetry, so for a request resolving to an anthropic/* model:

• max_tokens or max_completion_tokens, if sent, maps directly onto Anthropic’s required max_tokens.
• If neither is sent, the gateway injects Anthropic’s configured default deterministically.
• A value above that model’s output ceiling is rejected (400 gateway_output_cap) — never silently clamped.

Errors

Every error this endpoint itself produces — validation, model resolution, provider dispatch — uses the same OpenAI-shaped envelope regardless of which provider was involved:

1
{
2
  "error": {
3
    "message": "...",
4
    "type": "invalid_request_error",
5
    "param": "model",
6
    "code": "model_not_available"
7
  }
8
}

Security, BYOK, billing, and the limiter

• Security: the same bearer token as every other route (see Authentication). No separate credential exists for this endpoint.
• Kill switch / pilot allowlist: reused per-provider, unchanged — resolving to a model on a provider that’s off or outside your allowlist is 400 model_not_available here (not the route-wide 404 the native routes give), and the model is simply absent from the catalog.
• BYOK: if your account has registered its own key for the resolved provider (documented on that provider’s page), a request that resolves to that provider bills to your own key instead of the shared gateway balance — the same BYOK resolution the native route uses, since dispatch runs through that provider’s real pipeline either way.
• Billing: metered per token from the normalized usage object, at the resolved model’s cataloged rate (GET /inference/v1/models → pricing). For an anthropic/* model, metering reads Anthropic’s raw response — cloned before any request/response translation — so a translation bug can never mis-bill.
• Limiter: one capacity pool per provider, shared between this endpoint and that provider’s native /openai/v1/*, /anthropic/v1/*, /grok/v1/*, or /huggingface/v1/* route — alternating between the two draws from the same lease, not two separate ones.

Model catalog fields

Each entry in GET /inference/v1/models’s data array (and the single-model form at GET /inference/v1/models/{id}):

Examples

$
curl -X POST "$GATEWAY/inference/v1/chat/completions" \
>
  -H "Authorization: Bearer $TOKEN" \
>
  -H "Content-Type: application/json" \
>
  -d '{
>
    "model": "anthropic/claude-sonnet-5",
>
    "messages": [{"role": "user", "content": "Summarize this page in one sentence."}]
>
  }'

Need more than text-only chat?

This endpoint is deliberately narrow today — no embeddings, moderation, audio, files, batches, or fine-tuning, and image input only for the models whose catalog entry marks them vision-capable. For that broader surface, call the provider directly: