Skip to main content
Durable internal serving at /deployments, separate from TTL-based Playground sessions. Every deployment has a stable endpoint, immutable revisions and its own revocable tokens. Suspending releases the GPUs and keeps the identity.
TunePlane model deployments

Managed deployments, their revisions and endpoints.

​
Create a deployment

The wizard has four steps: identity, model source, runtime settings, review. Then vLLM or SGLang, and the settings that shape it: GPU count, dtype, quantization, context length, concurrency, memory utilisation, and the model name callers send. Engine-specific fields appear once you pick the engine. Settings are a typed allowlist. There is no free-form shell-argument field, so nothing you enter here can become an extra flag on the engine process.
trust_remote_code is disabled by default and is available only when an administrator permits it, because it executes code from the model repository.

​
Status and recovery

The normal progression is Deploying, Warming, then Ready. Failures use Degraded or Failed; a paused deployment is Suspended. Ready means the runtime is alive, health checks pass, /v1/models contains the expected alias, and a minimal generation warm-up succeeds. The controller recreates a lost or repeatedly unhealthy single-replica runtime with bounded backoff. Suspend stops the runtime and releases GPUs; resume recreates it from the current revision.

​
Call the stable endpoint

The initial token is shown once after creation. Create, revoke, or rotate additional credentials under Settings → Deployment tokens.
The detail page’s Playground tab uses the signed-in web session and never reads or exposes deployment tokens. Client traffic must use a deployment token; a web access token is not a substitute.

​
Responses API

A Ready deployment guarantees Responses creation and SSE streaming. The stable gateway also accepts GET, POST, and DELETE, so retrieval, cancellation, deletion, input-item listing, and related endpoints pass through when the selected engine implements them. The vLLM baseline guarantees create, retrieve, and cancel; newer SGLang releases expose a broader stateful surface.
Response and conversation state lives in the single runtime replica. Old response_id and conversation_id values may be lost after recovery, suspension, or a revision switch. Applications that need state across runtime replacement should retain complete input items and use store: false.
OpenAI-hosted built-ins such as web search, file search, and code interpreter are not automatically available on local engines. Function tools, reasoning, and structured outputs depend on the model, engine version, and configured parser.

​
Updates and rollback

A model or configuration change creates a new immutable revision. With spare GPU capacity, TunePlane warms the candidate before moving the stable endpoint. If it fails, the old revision stays active. Without spare capacity, the user must explicitly accept a maintenance window; the platform never introduces silent downtime. Use Revisions to inspect configuration snapshots and roll back. The detail page also exposes metrics, logs and settings.
Deleting a deployment stops its runtime and revokes every token immediately. It cannot be undone. Suspend instead when you may want it back.

​
Next

Deploying a model version · Reflow · Model registry · Inference API