Tools

OpenAI

Responses, Chat Completions, embeddings and moderation — policed, not transparent

View as Markdown

Fronts OpenAI’s API. Unlike the search and scraping upstreams, this route is not a plain pass-through: every request is checked against a model allowlist and a request-field policy before it reaches OpenAI, because a request body here can reference stored resources, change the price, or create state on the shared account.

GET|POST|DELETE /openai/<api path>   → https://api.openai.com/<api path>
POST            /openai/webhooks     → OpenAI's own job-completion callback (not client-facing)

Only need stateless chat completion, and want the same request body to also work against Anthropic, Grok, or Hugging Face models? /inference is a narrower, OpenAI-shaped surface over all four. Use this route directly for Responses, embeddings, moderation, Realtime, or the stateful files/vector-stores/batches/fine-tuning surface.

Endpoints

Only a fixed set of endpoints is admitted; anything else under /openai is refused with the gateway’s own 404 and never reaches OpenAI.

EndpointNotes
POST v1/responsesStateless by default; store: true is the explicit opt-in to server-side state
POST v1/chat/completionsThe OpenAI-ecosystem-compatible endpoint
POST v1/embeddings
POST v1/moderationsFree — settled at zero cost, still rate-limited
POST v1/realtime/client_secretsMints a short-lived Realtime credential; the WebSocket session itself runs directly between your client and OpenAI
GET v1/models, GET v1/models/{id}Answered locally from the gateway’s own catalog — OpenAI’s upstream model list is never proxied
v1/files, v1/vector_stores, v1/batches, v1/fine_tuning/jobs (and their {id} sub-paths)Create/read/delete, ownership-tracked per account (see below)

The model in every request body must be on the gateway’s own catalog — a model not listed there is rejected before the call reaches OpenAI, regardless of whether OpenAI itself would accept it.

Request policy

A closed set of body fields is enforced on v1/responses and v1/chat/completions:

  • Forced, regardless of what you send: store pins to false unless you explicitly ask for v1/responses state; a per-tenant identifier replaces any safety_identifier/prompt_cache_key you supply, so prompt-cache partitioning and abuse attribution land on your account specifically; the output-token cap is injected at the gateway’s configured ceiling.
  • Denied outright (a 400 in OpenAI’s own error shape, never a silent strip): referencing a prior response by id, conversation, a literal prompt, store: true without the explicit opt-in path, background: true, hosted tools beyond function/custom, a non-default service_tier, and a few other fields — see the endpoint’s response for the exact rejected field when this happens.
  • Passed through unchanged: everything not explicitly forced or denied, including the stateless multi-turn include: ["reasoning.encrypted_content"] channel.

Stateful resources

Files, vector stores, batches, and fine-tuning jobs you create through this route are tracked per account. An id that belongs to a different account is never reachable through yours — it answers the same 404 an id that never existed would, so nothing about another customer’s resources is revealed. GET v1/files and GET v1/vector_stores (the bare list forms) are answered locally from this ownership record rather than proxied, since the real upstream lists would enumerate the whole shared account.

Batches and fine-tuning jobs settle their real cost when OpenAI’s own completion webhook (POST /openai/webhooks) reports them done — that endpoint is called by OpenAI, authenticated by a webhook signature, and has nothing to do with your bearer token.

Billing

Metered per token from each response’s own usage (including the streaming terminal event), at the gateway’s per-model rate table. Moderation is free. An account can register its own OpenAI key; while that’s active, calls bill to that key directly instead of metering through the gateway balance.

GPT-5.6 family rates

gpt-5.6 is an alias for gpt-5.6-sol — OpenAI’s model page states that “the gpt-5.6 alias routes requests to GPT-5.6 Sol.” All three GPT-5.6 models split into a short-context and a long-context rate depending on the request’s input token count, with the boundary at 272,000 input tokens. All rates are USD per 1M tokens, Standard tier:

ModelContextInputCached inputCache writeOutput
gpt-5.6-sol (gpt-5.6)≤ 272,000 input tokens$5.00$0.50$6.25$30.00
gpt-5.6-sol (gpt-5.6)> 272,000 input tokens$10.00$1.00$12.50$45.00
gpt-5.6-terra≤ 272,000 input tokens$2.00$0.20$2.50$12.00
gpt-5.6-terra> 272,000 input tokens$4.00$0.40$5.00$18.00
gpt-5.6-luna≤ 272,000 input tokens$0.20$0.02$0.25$1.20
gpt-5.6-luna> 272,000 input tokens$0.40$0.04$0.50$1.80

cache write tokens are a separate, mutually exclusive input category, not an add-on to input — a token is billed as input, cached input, or cache write, never more than one. OpenAI reports the count as cache_write_tokens inside usage.input_tokens_details (Responses API) or usage.prompt_tokens_details (Chat Completions API), alongside the existing cached_tokens count.

The async v1/batches endpoint bills at exactly 50% of every Standard rate above, across both context tiers.

Source: OpenAI’s pricing and prompt caching documentation.

Examples

curl -X POST "$GATEWAY/openai/v1/chat/completions" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.6-luna",
    "messages": [{"role": "user", "content": "Summarize this page in one sentence."}]
  }'

Responses arrive as OpenAI’s JSON, untouched once a request clears the checks above. Field-level request and response detail lives in OpenAI’s documentation.