Streaming and debug metrics

Streaming

Set "stream": true and the gateway relays the model's output as server-sent events: one data: line per chunk, each an OpenAI chat.completion.chunk, then data: [DONE].

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"A token"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" is…"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Every OpenAI SDK handles this for you (stream=True in Python, stream: true in Node, iterate the result). With cURL, pass -N so output is not buffered.

Debug metrics

Send X-K2I-Debug-Metrics: true on a streaming request and the gateway appends one more event after data: [DONE]:

data: [DONE]

data: {"k2i_metrics":{"ttft_ms":112,"tok_s":48.7,"cost_micro_pkr":45120}}

Stock OpenAI clients stop reading at [DONE] and discard the extra event, so it is safe to send the header everywhere; to read the figures you need a small SSE reader of your own. With cURL they are simply the last line.

curl -N https://api.intelligentinference.ai/v1/chat/completions \
  -H "Authorization: Bearer $I2_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-K2I-Debug-Metrics: true" \
  -d '{"model":"IntelligentInference/gpt-oss-120b(Global)","stream":true,"messages":[{"role":"user","content":"Hello"}]}'

The same three figures, for every request, are in the dashboard's request log and analytics; the header is for when you want them inside your own code path.

Timings, defined

• Time to first token is measured at the gateway: request in, first token out. It includes the model's prefill and the gateway's own overhead (about two milliseconds), not your network.
• Throughput is output tokens over generation time, after the first token.
• For non-streaming requests the first-token figure does not exist; the log shows it as not measured rather than zero.