Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
Start, list, extend, and stop Playground inference services
tp serve start <MODEL> [options]
| Argument / Option | Description |
|---|---|
MODEL (required) | HF id / absolute path on the shared disk / run:<run_id> (uses that run’s hf_export) |
-g, --gpus | GPU count = tensor parallel size (default 1) |
--ttl-hours | Idle TTL in hours; defaults to the server-side configuration |
tp serve ls # My services: status, endpoint, remaining TTL
tp serve extend <SERVE_ID> --ttl-hours 4
tp serve stop <SERVE_ID>
extend, and stop as soon as you’re done.tp serve ls can be configured directly into any OpenAI SDK client.
tp servetp serve <subcommand>
| Command | Description |
|---|---|
tp serve extend | Extend an inference service’s TTL |
tp serve ls | List my inference services |
tp serve start | Start an inference service |
tp serve stop | Stop an inference service |
tp serve extendtp serve extend <RUN_ID> [options]
| Argument | Description |
|---|---|
RUN_ID | required |
| Option | Description |
|---|---|
--hours <float> | How much longer, in hours (default 1.0) |
tp serve lstp serve ls
--help.
tp serve starttp serve start <MODEL> [options]
| Argument | Description |
|---|---|
MODEL | An HF id / an absolute path on shared storage / run:<run_id> (required) |
| Option | Description |
|---|---|
--gpus, -g <int> | How many cards (the tensor-parallel size) (default 1) |
--ttl-hours <float> | Idle TTL in hours (default: the server setting) |
tp serve stoptp serve stop <RUN_ID>
| Argument | Description |
|---|---|
RUN_ID | required |
--help.