> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a model version

> Put a model behind a stable internal address, with revisions you can promote and roll back.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -X POST https://tuneplane.your-company.com/api/model-deployments \
  -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "support-agent",
    "model_source": {"kind": "model", "model": "alice/support-agent"},
    "config": {"engine": "vllm", "gpus": 2, "max_model_len": 8192},
    "fleet_id": "fleet-gpu-a",
    "serve_mode": "proxied"
  }'
```

You get a Model Deployment: a stable address that application traffic can call, serving whatever
revision is currently promoted.

Most people create one from the console's [Deployments page](/en/console/deployments) instead.
The API is here because a deployment is the thing you most often want in a script.

## A Deployment is not a Playground Session

|                    | Playground Session           | Model Deployment                    |
| ------------------ | ---------------------------- | ----------------------------------- |
| For                | A person trying a checkpoint | Application traffic                 |
| Lifetime           | An idle TTL, then it stops   | Until you suspend or delete it      |
| Address            | Issued per session           | Stable, and outlives every revision |
| Auth               | Your own login               | A revocable Deployment Token        |
| Changing the model | Start another session        | Create a revision and promote it    |

## Where the model comes from

`model_source.kind` is one of:

| Kind          | Fields                                            | Use it for                                                       |
| ------------- | ------------------------------------------------- | ---------------------------------------------------------------- |
| `model`       | `model` as `<owner>/<name>`, optional `version`   | The normal case. Omit the version to follow whatever is promoted |
| `artifact`    | `run_id`, optional `step`                         | One run's export directly                                        |
| `huggingface` | `repository`, `revision`, `use_linked_credential` | A hub model                                                      |
| `modelscope`  | `repository`, `revision`                          | A public ModelScope repository                                   |
| `shared_path` | a path                                            | Weights already on shared storage                                |

`model` with no version is what makes `tp model promote` the only step needed to change what
is being served. See [the model registry](/en/guides/model-registry).

Both are pinned on the way in. An `artifact` source records which exported step it took, and a
`model` source with no version resolves production once and stores the number it got. A revision
that loads different weights after a restart is not a revision, and re-following a moving
pointer on every launch is how that happened.

## Serving configuration

```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
{
  "engine": "vllm",
  "gpus": 4,
  "dtype": "bfloat16",
  "quantization": "none",
  "max_model_len": 8192,
  "gpu_memory_utilization": 0.9,
  "served_model_name": "support-agent",
  "tensor_parallel_size": 2,
  "data_parallel_size": 2
}
```

| Field                    | Default        | Notes                                                |
| ------------------------ | -------------- | ---------------------------------------------------- |
| `engine`                 | `vllm`         | Or `sglang`. Different engines, not a backend choice |
| `gpus`                   | `1`            | How many cards the deployment holds                  |
| `dtype`                  | `auto`         | `float16`, `bfloat16`                                |
| `quantization`           | `none`         | `awq`, `gptq`, `bitsandbytes`, `fp8`                 |
| `max_model_len`          | engine default | Context length                                       |
| `gpu_memory_utilization` | `0.9`          |                                                      |
| `max_concurrency`        | unset          |                                                      |
| `served_model_name`      | `model`        | The `model` field callers send                       |
| `tensor_parallel_size`   | all cards      |                                                      |
| `pipeline_parallel_size` | `1`            | For a model that does not shard cleanly              |
| `data_parallel_size`     | `1`            | For replicas of a small model across cards           |

`tensor_parallel_size × pipeline_parallel_size × data_parallel_size` must equal `gpus`. Anything
else is a container that either cannot start or quietly uses fewer cards than it was charged
for, and the second is worse because nothing looks wrong.

For a knob the schema does not model, `extra_args` is passed to the engine verbatim after the
platform's own flags, and `env` sets the engine process environment — `VLLM_*`, `NCCL_*`,
`HF_HOME`. Flags the platform has to be right about, such as `--port` and `--model`, are
reserved and refused.

Credentials do not belong in `env`. A private model source takes a governed Model Credential,
which is never echoed back on a read.

## Proxied or direct

`serve_mode` decides whether the console is in the data path.

| Mode                | Traffic goes                      | Reflow capture |
| ------------------- | --------------------------------- | -------------- |
| `proxied` (default) | Through the console to the engine | Yes            |
| `direct`            | Straight to the engine            | **No**         |

A `direct` deployment captures nothing, so there is no [Reflow](/en/guides/reflow) buffer to mine
later. Pick it for latency, knowing that.

## Which Fleet serves it

`fleet_id` is yours to choose and the platform never picks for you. A deployment is long-lived
and its address outlives the decision, and Fleets differ on things the platform cannot rank:
which department owns the machines, which data may touch them.

## Call it

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -X POST https://tuneplane.your-company.com/api/model-deployments/<id>/tokens \
  -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  -d '{"name": "support-service"}'
```

That returns the token once. Then application traffic uses the OpenAI-compatible endpoint:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl https://tuneplane.your-company.com/inference/<id>/v1/chat/completions \
  -H "Authorization: Bearer <deployment token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "support-agent", "messages": [{"role": "user", "content": "Hello"}]}'
```

A Deployment Token authorises traffic to one deployment and is revocable on its own, independent
of anyone's login session. Revoke one with
`DELETE /api/model-deployments/<id>/tokens/<token_id>`.

## Change what it serves

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# a new revision, promoted when it reports ready
curl -X POST .../api/model-deployments/<id>/revisions \
  -d '{"model_source": {...}, "config": {...}, "allow_downtime": false}'

# back to a known-good one
curl -X POST .../api/model-deployments/<id>/rollback \
  -d '{"revision_id": "rev-7", "allow_downtime": false}'
```

A revision is an immutable snapshot of the model source and the serving configuration together.
Promotion switches the endpoint to a revision that is already ready, so the address never points
at something still starting. `allow_downtime: true` skips waiting, for a deployment whose cards
cannot hold both revisions at once.

`suspend` releases the serving capacity and keeps the identity, the revisions, the address and
the tokens. It is not deletion.

## Check it worked

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
curl -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \
  https://tuneplane.your-company.com/api/model-deployments/<id>
```

`status` reads ready and `current_revision.internal_endpoint` is filled. The console's
Deployments page shows the same thing, plus `logs` and `metrics` for the engine process.

`desired_state` is what you asked for and `status` is what is observed. They differ while a
revision starts, and a difference that persists has its reason in `last_error`.

## Next

[Model registry](/en/guides/model-registry) · [Reflow](/en/guides/reflow) ·
[The Deployments page](/en/console/deployments) · [Inference API](/en/api-reference/inference)