> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nativeport.ai/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nativeport.ai/_mcp/server. # Inference API `/inference` is a unified chat-completions surface over the four model-inference routes — [OpenAI](/openai), [Anthropic](/anthropic), [Grok](/grok), and [Hugging Face](/huggingface). One OpenAI Chat Completions-shaped contract, JSON and SSE alike, addressed by a **canonical model id** instead of four different native shapes. It is a translation layer, not a fifth upstream: no new upstream key, no separate catalog, no separate capacity pool — it shares the same kill switches, pilot allowlists, BYOK keys, rate limiter, and settlement as the four routes it fronts. ``` POST /inference/v1/chat/completions → normalized chat completion (JSON or SSE) GET /inference/v1/models → every model your account can currently reach GET /inference/v1/models/ → one model's catalog entry ``` This route ships fully mounted — unlike the four routes it fronts, it never 404s wholesale behind a kill switch. But a canonical id naming a model whose own provider is currently disabled, or outside your account's pilot allowlist, still isn't usable: it answers `400 model_not_available` on the chat endpoint rather than a route-wide `404`, and it's simply absent from `GET /inference/v1/models`. **That catalog call is the source of truth for what your account can reach right now** — a model existing in one of the four providers' own catalogs doesn't mean it's enabled for you today. ## Canonical model IDs `model` is **mandatory** on every chat request — there is no default and no fallback. Routing from a valid canonical id to a provider and underlying model is deterministic, never inferred: | Form | Example | | ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `openai/` | `openai/gpt-5.4-mini` | | `anthropic/` | `anthropic/claude-sonnet-5` | | `xai/` | `xai/grok-4.3` | | `huggingface//[:provider]` | `huggingface/openai/gpt-oss-120b`, or `huggingface/openai/gpt-oss-120b:together` pinned to an explicit serving provider | Only the **first** `/` after the provider prefix is significant — everything after it, slashes included, is the underlying provider's own model id. That's why the Hugging Face form can itself contain a `/` (the HF repo's `owner/model`) plus an optional `:provider` suffix, exactly as on the [Hugging Face route](/huggingface) itself. An id that doesn't parse, doesn't resolve in the underlying provider's catalog, or names a chat-incapable OpenAI model (embeddings/moderation/realtime-only entries are excluded) is indistinguishable from an id whose provider is currently disabled — all of it is "not available," fail-closed. ## Drop-in for the OpenAI SDK Because the request and response shapes are OpenAI's own, official OpenAI client libraries work by pointing `base_url` at this route and using a canonical id as `model`: ```python from openai import OpenAI client = OpenAI( api_key="YOUR_API_KEY", base_url="https://api.nativeport.ai/inference/v1", ) response = client.chat.completions.create( model="anthropic/claude-sonnet-5", messages=[{"role": "user", "content": "Summarize this page in one sentence."}], ) print(response.choices[0].message.content) ``` Swapping `model` to `openai/gpt-5.4-mini`, `xai/grok-4.3`, or `huggingface/openai/gpt-oss-120b` requires no other code change — the same request body works against any admitted model. ## Endpoints | Endpoint | Notes | | -------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `POST v1/chat/completions` | The only admitted `POST`. Accepts `stream: true` for Server-Sent Events. | | `GET v1/models` | Every canonical id your account can currently call, each with `capabilities`, `supported_parameters`, and USD pricing. | | `GET v1/models/{id}` | One model's catalog entry — same object shape as an entry in the list above. | ## Request fields — the intersection, not the union The point of a *unified* endpoint is that one request **schema** works across any admitted model: the accepted fields are the **intersection** every backing provider can express — deliberately narrower than any single provider's own field policy on its native route. A field outside this set is rejected with a fail-closed `400 unsupported_parameter` before any upstream call, never silently dropped: `model`, `messages` (`text`/`image_url` content parts), `max_tokens`, `max_completion_tokens`, `temperature`, `top_p`, `stop`, `stream`, `stream_options` (`include_usage` only), `tools`/`tool_choice` (function-type only), `parallel_tool_calls`. This exact list is also what `GET /inference/v1/models` reports per model as `supported_parameters` — uniform across every listed model, and guaranteed never to drift from what the request validator actually accepts. That uniformity is a schema guarantee, not a value guarantee for every field. `image_url` content validates structurally the same way for every model, but is only actually **honored** for one whose catalog entry reports `capabilities.vision: true`. Sent to a model that isn't marked vision-capable, the whole request is rejected preflight with `400 unsupported_parameter` — never forwarded for that provider to fail unpredictably on, or silently dropped. ## Response shape Regardless of which of the four providers actually served the request, the response is always OpenAI's `chat.completion` (or streaming `chat.completion.chunk`) shape: * **`model` is rewritten** to the canonical id that served the request — never the provider's own native model identifier. * **`usage`** (`prompt_tokens`/`completion_tokens`/`total_tokens`, plus `prompt_tokens_details.cached_tokens` when the resolved model reports cache-read tokens) is reported in this one normalized shape even for providers whose native usage field names differ. * **`tool_calls`** are normalized to OpenAI's `{id, type: "function", function: {name, arguments}}` shape, arguments always a JSON-encoded string — including for Anthropic, whose native `tool_use` content blocks are translated on the way out. * **`finish_reason`** is one of `stop`, `length`, `tool_calls`, `content_filter` — Anthropic's `stop_reason` values (`end_turn`, `max_tokens`, `tool_use`, `refusal`, …) are mapped onto this same set. ## Streaming Set `stream: true` for `text/event-stream` instead of a single JSON object: a sequence of `data: ` lines separated by blank lines, terminated by a literal `data: [DONE]` line. Set `stream_options.include_usage: true` to receive one extra usage-only chunk (empty `choices`) immediately before `[DONE]`. ``` data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1769434200,"model":"openai/gpt-5.4-mini","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]} data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1769434200,"model":"openai/gpt-5.4-mini","choices":[{"index":0,"delta":{"content":"This"},"finish_reason":null}]} data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1769434200,"model":"openai/gpt-5.4-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} data: [DONE] ``` This holds for every provider, including Anthropic — whose native `message_start`/`content_block_*`/`message_delta`/`message_stop` SSE sequence is translated event-by-event into this same chunk shape, tool-call argument deltas included. ## Diagnostic headers Every response from this endpoint — success or error, `GET /inference/v1/models[/{id}]` included — carries: | Header | Present | Meaning | | ------------------------- | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `x-nativeport-request-id` | Always | A gateway-assigned id for this call, for support/troubleshooting. | | `x-nativeport-provider` | Wherever a single provider applies | `openai`, `anthropic`, `xai`, or `huggingface`. Omitted only on the bare `GET /inference/v1/models` list, which spans every provider at once. | | `x-nativeport-model` | Wherever a single model applies | The canonical id — same value as the response body's `model`/`id` field. | ## Anthropic's `max_tokens` mapping Claude's native Messages API **requires** `max_tokens`; OpenAI's Chat Completions API doesn't. Calling [`/anthropic/v1/messages`](/anthropic) directly, an omitted `max_tokens` is left out and Anthropic's own `400` stands — a client talking to Claude directly is expected to know the field is mandatory. This endpoint's contract is different: a request written against one canonical model is supposed to work unchanged against any other, and OpenAI/xAI/Hugging Face chat completions all auto-cap output when a client sends neither `max_tokens` nor `max_completion_tokens`. Leaving Anthropic as the one silent exception would be an accidental per-provider asymmetry, so for a request resolving to an `anthropic/*` model: * `max_tokens` or `max_completion_tokens`, if sent, maps directly onto Anthropic's required `max_tokens`. * If **neither** is sent, the gateway injects Anthropic's configured default deterministically. * A value above that model's output ceiling is **rejected** (`400 gateway_output_cap`) — never silently clamped. ## Errors Every error this endpoint itself produces — validation, model resolution, provider dispatch — uses the same OpenAI-shaped envelope regardless of which provider was involved: ```json { "error": { "message": "...", "type": "invalid_request_error", "param": "model", "code": "model_not_available" } } ``` | Status | `code` | When | | ------ | ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `400` | `unsupported_parameter` | A field, message shape, or tool shape outside the common set — or `image_url` content aimed at a non-vision model. | | `400` | `invalid_request_error` | A field present but malformed (wrong type, missing sub-field, unparseable JSON tool arguments). | | `400` | `model_not_available` | `model` doesn't resolve, or its provider is currently disabled/outside your pilot allowlist. | | `400` | `gateway_output_cap` | An explicit `max_tokens`/`max_completion_tokens` above the resolved model's provider-configured output ceiling — every one of the four backing providers enforces its own ceiling here, same as on its native route. | | `404` | `model_not_found` | `GET /inference/v1/models/{id}` called with an id that doesn't resolve or isn't currently enabled — indistinguishable from "never existed." | | `429` | `gateway_rate_limited` | The per-account limiter refused the request (shared capacity with that provider's native route — see below). Carries a `retry-after` header. | | `502` | `gateway_upstream_unavailable` | The resolved provider's own upstream was unreachable (DNS/connection/TLS/timeout), rather than answering with an error of its own. | The account-wide bearer-token and credit gate — shared by **every** route in this gateway, checked before a request ever reaches `/inference` routing — still answers with the plain `{"error": "..."}` string shape documented on the [API reference overview](/api-reference#common-responses): `401` for a missing/unknown/revoked token, `402` for a zero-or-negative balance, `403` for a suspended account. Only errors produced *by* the inference endpoints themselves — past that gate — use the nested OpenAI-shaped envelope above. A `402` can also occur past the gate, from a per-request cost-estimate preflight inside the resolved provider's own pipeline; that one is OpenAI-shaped (`code: gateway_insufficient_credits`). ## Security, BYOK, billing, and the limiter * **Security**: the same bearer token as every other route (see [Authentication](/authentication)). No separate credential exists for this endpoint. * **Kill switch / pilot allowlist**: reused per-provider, unchanged — resolving to a model on a provider that's off or outside your allowlist is `400 model_not_available` here (not the route-wide `404` the native routes give), and the model is simply absent from the catalog. * **BYOK**: if your account has registered its own key for the resolved provider (documented on that provider's page), a request that resolves to that provider bills to your own key instead of the shared gateway balance — the same BYOK resolution the native route uses, since dispatch runs through that provider's real pipeline either way. * **Billing**: metered per token from the normalized `usage` object, at the resolved model's cataloged rate (`GET /inference/v1/models` → `pricing`). For an `anthropic/*` model, metering reads Anthropic's **raw** response — cloned before any request/response translation — so a translation bug can never mis-bill. * **Limiter**: one capacity pool per provider, shared between this endpoint and that provider's native `/openai/v1/*`, `/anthropic/v1/*`, `/grok/v1/*`, or `/huggingface/v1/*` route — alternating between the two draws from the same lease, not two separate ones. ## Model catalog fields Each entry in `GET /inference/v1/models`'s `data` array (and the single-model form at `GET /inference/v1/models/{id}`): | Field | Meaning | | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `id` | The canonical model id. | | `owned_by` | For a Hugging Face entry, the underlying HF repo's owner segment — not necessarily the same string as `provider`. | | `provider` | `openai`, `anthropic`, `xai`, or `huggingface`. | | `capabilities.streaming` / `.tools` / `.tool_choice` | Always `true` for every listed model today. | | `capabilities.vision` | Truthful per model — whether it's known to accept `image_url` content, not a blanket value. | | `supported_parameters` | The optional request fields this gateway accepts, identical across every listed model (see *Request fields* above). | | `pricing` | Per-1M-token USD rates: `input_per_million`, `output_per_million`, and where the provider has a cache-discount surface, `cached_input_per_million` (OpenAI's cached-input rate or Anthropic's cache-read rate), plus Anthropic-only `cache_write_5m_per_million` / `cache_write_1h_per_million`. | ## Examples #### Non-streaming (cURL) ```bash curl -X POST "$GATEWAY/inference/v1/chat/completions" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic/claude-sonnet-5", "messages": [{"role": "user", "content": "Summarize this page in one sentence."}] }' ``` #### Streaming (cURL) ```bash curl -N -X POST "$GATEWAY/inference/v1/chat/completions" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.4-mini", "stream": true, "stream_options": {"include_usage": true}, "messages": [{"role": "user", "content": "Summarize this page in one sentence."}] }' ``` #### Python (OpenAI SDK) ```python from openai import OpenAI client = OpenAI(api_key="YOUR_API_KEY", base_url="https://api.nativeport.ai/inference/v1") response = client.chat.completions.create( model="xai/grok-4.3", messages=[{"role": "user", "content": "Summarize this page in one sentence."}], ) print(response.model, response.choices[0].message.content) ``` #### Tools ```bash curl -X POST "$GATEWAY/inference/v1/chat/completions" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-5.4-mini", "messages": [{"role": "user", "content": "What is the weather in Boston?"}], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Look up the current weather for a city.", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]} } }], "tool_choice": "auto" }' ``` #### List the catalog ```bash curl "$GATEWAY/inference/v1/models" \ -H "Authorization: Bearer $TOKEN" ``` #### Look up one model ```bash curl "$GATEWAY/inference/v1/models/huggingface/openai/gpt-oss-120b:together" \ -H "Authorization: Bearer $TOKEN" ``` #### Error shape ```json { "error": { "message": "The model 'openai/gpt-5.9-preview' is not available through this gateway. List the available models via GET /inference/v1/models.", "type": "invalid_request_error", "param": "model", "code": "model_not_available" } } ``` ## Need more than text-only chat? This endpoint is deliberately narrow today — no embeddings, moderation, audio, files, batches, or fine-tuning, and image input only for the models whose catalog entry marks them vision-capable. For that broader surface, call the provider directly: #### [OpenAI](/openai) Responses, embeddings, moderation, files, batches, fine-tuning #### [Anthropic](/anthropic) Extended thinking, server tools, files, message batches #### [Grok](/grok) Responses API, tokenize, files, batches #### [Hugging Face](/huggingface) The full Inference Providers router, provider-pinned pricing