> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tasksets and Agent evaluations

> Publish independent agent tasks and prepare a reproducible Harbor evaluation.

**Tasksets** live under Assets. **Agent evaluations** live under Evaluation.
A taskset defines the work and its train/eval split. An environment defines where
an agent acts. Harbor coordinates the agent, sandbox and verifier for each trial.
You can reuse a taskset across models and execution environments.

## Publish an immutable taskset

Place Harbor task directories beneath a common root. Each directory must contain
`task.toml`; task IDs equal directory names and must be unique.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tp taskset build ./tasks --split eval
tp taskset push ./tasks --name agent-tasks --version v1
```

`build` records file hashes in `taskset.json`. To mix training and evaluation tasks,
edit individual `split` fields before publishing. Modified task content requires
updated hashes and a new version; existing manifests are never overwritten by
`build`. Symlinks, overlapping task directories and path traversal are rejected.

The console displays sealed versions, split counts and the content digest. It
exposes no task text or reference-answer download. Private versions are visible
only to authorized users. Jobs reference `owner/name@version`; floating versions
are rejected. Authorized local-container jobs receive a read-only taskset mount
through `TUNEPLANE_JOB_TASKSET_DIR`. Other delivery backends are not supported yet.

Split labels select the evaluation cohort; they do not hide files from arbitrary
code inside an authorized job. Publish held-out tasks as a separate taskset when
you need separate access permissions.

## Prepare a Harbor evaluation

Open **Agent evaluations**, or follow the link from a taskset version. Choose:

* A sealed taskset version; only its evaluation split runs.
* A hosted model endpoint and immutable model revision.
* Docker or E2B as the sandbox backend available in your runtime.
* Runtime and harness SHA-256 identities, repetitions, concurrency and trial timeout.

The page validates the request and downloads `harbor-evaluation.json`. It does
not launch a job. Use the `harbor-eval` experiment in `tuneplane-examples`, place
this file at the project root, and follow that experiment's README. Its
`--check-plan` mode validates local task content without importing Harbor or
creating sandboxes. A platform submission uses `tp submit --taskset` with a
configured local profile and pinned runtime image.

## Capacity and results

A request supports up to 10,000 planned trials and 256 concurrent trials. Without a selected
[sandbox pool](/en/console/sandbox-pools), this is only a per-job limit. Actual capacity depends on the
provider and runtime. Docker access or E2B credentials must be provisioned in the
runtime; selecting a backend in the console does not provision it.

Infrastructure failures stop new trials. Cooperative cancellation waits for
started trials to clean up and saves partial results. Force-kill cleanup requires
provider TTLs and reconciliation. Every planned outcome contributes to coverage;
missing and unscored results do not become successful evaluations. Text capture
is disabled by default. Taskset content, verifier content, runtime, harness and
actual execution settings accompany results for comparison.

Runtime and harness hashes are declarations, not certification of the installed
backend. Shared pool capacity and leases are available through the Sandbox pools page. A
console launch/history workflow and native provider certification remain separate work.