> ## Documentation Index > Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt > Use this file to discover all available pages before exploring further. # Start from an engineering solution > A recipe, benchmark packs, a toolchain and the acceptance protocol that says what counts as better. ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} tp solution list # what this installation has tp solution show rtl-repair # what it references, and what counts as better tp solution init rtl-repair # copy its starting point into your project ``` Every piece a team needs to improve a model on one task already exists on this platform, and that is the problem. A recipe, an environment, two benchmark packs, a rubric, a toolchain and a data format are seven objects in six places, each correct on its own, and assembling them into "here is how we do RTL repair" is work every team does again from scratch. What they cannot copy from each other is the part that took the longest: the **acceptance protocol** — which baseline, which held-out split, how many tasks, how much better counts as better. A Solution is a declaration that names the seven and writes down the eighth. ## A Solution runs nothing It contains no reward function, no harness and no training code. Every field in it is a *name*, resolved on the deployment that installs it, and the platform never executes anything inside the pack. That boundary is the same one the rest of the platform keeps: the control plane parses, validates, stores and schedules, and what runs inside a job belongs to your project or to a plugin. A Solution that embedded a rubric's text would be a second copy of it, drifting away from the one the training run actually scores against. A name that does not resolve is reported as missing, which is a fact an operator can act on; an embedded copy that has drifted is not detectable at all. ```yaml solution.yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} schema: tuneplane/solution/v1 name: rtl-repair title: RTL repair and generation summary: > Train a model to write and fix Verilog against a specification, and judge it by compiling the result and running the official testbench. recipe: trl/sft benchmarks: [verilogeval-v2, rtllm-v2, ifeval] toolchain: eda-oss data_format: spec_rtl keywords: [rtl, verilog, hardware, repair] acceptance: baseline: prompt metrics: - key: pass@1 min_delta: 0.05 - key: syntax_pass min_delta: 0.0 - key: inst_level_strict_acc min_delta: 0.0 sample_min: 120 split_by: design human_review: true ``` | Field | What it names | | ------------- | ---------------------------------------------------------------------------------- | | `schema` | `tuneplane/solution/v1`. Anything else is refused with the schema this build reads | | `name` | The leaf name, and it must equal the directory the manifest sits in | | `title` | Required. What a list row says | | `summary` | Optional prose | | `recipe` | `/` of the recipe a run starts from | | `environment` | `/@` of the Environment, when the task needs one | | `benchmarks` | Benchmark pack ids. Several, because one number is never the whole answer | | `rubric` | `/` of a Rubric, for the part no tool can check | | `toolchain` | The Toolchain Profile the verification stages need | | `data_format` | One of the [nine row shapes](/en/guides/data-formats) | | `template` | The subdirectory `tp solution init` copies. Defaults to `template` | | `keywords` | Free tags | A manifest that names **neither** a recipe nor an environment is refused: there would be nothing for a user to start from. A `data_format` that is not one of the nine is refused with the list. ## Every first-party solution compares against a prompt baseline `acceptance.baseline` takes a model reference, or the string `prompt` — a prompt-only baseline on the same base model. All four shipped solutions use `prompt`, deliberately. **If careful prompting on the base model already meets the need, the right answer is to ship that and measure it, not to run a training job.** A protocol with no prompt baseline never surfaces that case, and the training run gets built anyway. The rest of the protocol is decided the same way: cheap to make honestly in advance, impossible to make honestly afterwards. | Field | What it decides | | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `baseline` | A model reference, or `prompt`. A baseline chosen after seeing the candidate's scores is not a baseline | | `metrics[].key` | The metric the candidate has to move | | `metrics[].min_delta` | How much better, as a **change** rather than a floor. A floor is a claim about the task's difficulty nobody can make before measuring. `0.0` is legitimate and means "must not regress" | | `metrics[].direction` | `higher` (default) or `lower`. Latency and area are the cases that are not higher | | `sample_min` | The fewest evaluation tasks the comparison may rest on. A benchmark of forty problems moves 2.5% when one flips, so a delta below the resolution of the sample is noise with a decimal point | | `split_by` | The metadata field that decides the train/eval split, the same thing `spec.data.quality.group_key` names | | `human_review` | Whether a person has to look before release. A field rather than an assumption: a functional pass on a testbench is evidence, an EDA script that "looks right" is not | `min_delta` cannot be negative. A metric where less is better says so with `direction: lower` rather than by writing a negative delta. Each solution declares at least one guardrail metric at `min_delta: 0.0`. A model that gains RTL pass rates while quietly losing instruction following has not improved; it has traded one capability for another without saying so. ## The four that ship ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} tp solution list ``` | Solution | Starts from | Benchmarks | Data format | Toolchain | | ---------------------- | ----------- | -------------------------------------- | ---------------- | --------- | | `rtl-repair` | `trl/sft` | `verilogeval-v2`, `rtllm-v2`, `ifeval` | `spec_rtl` | `eda-oss` | | `testbench-generation` | `trl/sft` | `verilogeval-v2-completion` | `testbench` | `eda-oss` | | `eda-script-assistant` | `trl/sft` | `humaneval`, `ifeval` | `script` | `eda-oss` | | `eda-log-triage` | `trl/sft` | `ifeval` | `log_root_cause` | — | **`rtl-repair`** has the shortest path from "the model answered" to "a tool says it works", which is why it is the first one to try on a new deployment. `pass@1` must gain 0.05 over the prompt baseline; `syntax_pass` and `inst_level_strict_acc` must not regress; at least 120 tasks; split by `design`. **`testbench-generation`** trains the model to write the bench rather than the design. Its two metrics pull in opposite directions on purpose — a bench has to accept a correct design *and* reject a broken one, and `assert(1)` scores perfectly on the first alone. **`eda-script-assistant`** is the one with the least public data and the most private data. Its two packs are guardrails, not the measurement: what it is actually judged on is a private taskset of the team's own scripts, which cannot ship in a first-party solution and has to be declared by the deployment that installs it. `human_review` is load-bearing here — a script that runs is not a script that is right, since it can silently constrain the wrong clock. **`eda-log-triage`** involves no simulator and declares no toolchain, so it is the one workflow here a deployment with no EDA toolchain can still run. It splits by `case` rather than by `design`: the same failure reported twice is one fact. ## Read the protocol before you train ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} tp solution show rtl-repair ``` `show` prints what the solution references, its data format, and then the protocol in the order somebody has to decide it: ```text theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} baseline prompt pass@1 at least +0.05 syntax_pass must not regress inst_level_strict_acc must not regress minimum tasks 120 split by design human review required ``` Freeze that before the first run. The whole value of the field is that it was written down first. ## Scaffold it into your project ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} tp solution init rtl-repair --into ./experiments tp solution init rtl-repair --force # overwrite an existing directory ``` `init` copies the solution's `template/` directory to `/` and stops. It does not submit, it does not configure the deployment, and it writes nothing the platform then owns. The files land in your repository and are yours to edit; nothing is uploaded, and the template runs in your container at the trust level of anything else you wrote. An existing target directory is refused rather than merged — a half-overwritten scaffold is worse than either outcome — so `--force` is how you say you meant it. **None of the four shipped solutions carries a `template/` directory.** `tp solution init` on any of them prints `solution '' ships no template` and exits 1. Until one ships a template, `list` and `show` are what the built-ins are for, and `init` is for solutions your own deployment publishes. ## Whether this deployment can run one A name is only worth publishing if somebody checks that it resolves, and that check is the one thing the SDK cannot do — it knows what the manifest says and nothing about which of those exist here. Two read-only endpoints do it: ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} curl -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \ https://tuneplane.your-company.com/api/solutions curl -H "Authorization: Bearer $TUNEPLANE_CLIENT_TOKEN" \ https://tuneplane.your-company.com/api/solutions/rtl-repair ``` The list gives `ready` and the references that did not resolve, named rather than counted — the next thing somebody does is install exactly those. The detail adds every reference with its kind and a note, plus the acceptance protocol as the solution declared it. `ready` means **every** name resolves. A solution one benchmark pack short is not "mostly ready": the acceptance protocol names that pack, so a run without it is measured against something else. Both routes read. A Solution is installed the way every other declarative pack is, through the plugin API, and a second creation path here would be a second set of digest and authorisation rules for the same object. **Toolchain Profiles are not a separate object on this build.** A solution's `toolchain` reference resolves against the deployment's configured sandbox image, and the reference carries a note saying so. A deployment with a sandbox image configured can run a solution that needs a toolchain; one without cannot. ## Write your own A Solution is a `kind: solution` plugin — a declarative pack, so the platform parses it and never runs it: ```text theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} our-rtl-flow/ ├── plugin.yaml # kind: solution ├── solution.yaml # the manifest above ├── template/ # optional: the starting point `tp solution init` copies └── README.md ``` ```yaml plugin.yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} schema: tuneplane/plugin/v1 name: our-rtl-flow version: 1.0.0 kind: solution summary: How we do RTL repair here ``` ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} tp plugin publish ./our-rtl-flow tp recipe sync # pull published packs down to this machine tp solution list # it is now in the catalog as /our-rtl-flow ``` Three rules, checked in two different places: * **No executable file outside `template/`**, refused at publish. A `.py` at the root means somebody expects the platform to import it, which is exactly the expectation a declarative pack must not create. Scaffolding handed to users belongs under `template/`, where it runs in the user's own environment. * **The directory name must equal `name`**, refused when the catalog loads the pack. A directory called `rtl-flow` holding a manifest that says `name: our-rtl-flow` does not load. * **A built-in cannot be shadowed.** Built-ins load first and the first writer wins, so a pack named `rtl-repair` does not replace the shipped one. Two people running "the same" solution and getting different protocols is worse for an *acceptance* protocol than for a recipe, because the thing that differs is the definition of the result. Published packs install under `$TUNEPLANE_HOME/solutions///` — `~/.tuneplane` unless `TUNEPLANE_HOME` says otherwise — and are referenced as `/`; built-ins keep the bare name. For a development checkout, extra catalog directories can be named in `TUNEPLANE_SOLUTION_PATH`, separated by `:` on Linux and macOS. ## What a Solution does not do Be clear about the boundary before you rely on it. * **The acceptance protocol is a record, not a gate.** Nothing on the platform compares a run against the declared `baseline`, counts the tasks against `sample_min`, or refuses a promotion because `min_delta` was not met. The enforcing mechanism is the evaluation gates in `spec.evaluation.gates`, which are declared per job — see [running benchmarks](/en/guides/benchmarks). The Solution is where the team wrote the protocol down; keeping to it is still the team's. * **`split_by` is a statement of intent.** It names the field your split should honour. The platform does not compute the leak for you at submission — see [data shapes and splits](/en/guides/data-formats) for what is and is not wired. * **There is no console page for solutions yet.** `tp solution` and the two API routes are the whole surface. * **Nothing here has been scored end to end on real hardware.** The hardware-facing solutions point at a toolchain image that has never been built in CI and evaluators no runner in this project can execute; [RTL benchmarks](/en/guides/rtl-benchmarks) says exactly what stayed unproven. ## Next [`tp solution` reference](/en/cli/solution) · [RTL benchmarks](/en/guides/rtl-benchmarks) · [Data shapes and splits](/en/guides/data-formats) · [Benchmark packs](/en/extend/benchmark-packs) · [Plugins](/en/extend/plugins)