> ## Documentation Index > Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt > Use this file to discover all available pages before exploring further. # Data shapes and design-lineage splits > The nine row shapes the platform can check, and why splitting an engineering corpus at random leaks. ```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} data: quality: group_key: design # what a row belongs to max_group_leak_ratio: 0.0 # how much of the held-out set may share one ``` A dataset here is a file of JSON objects and nothing more: the platform stores it, profiles it, scans it and hands it to a framework, and what is *in* a row is the framework's business. That is the right default, and it stops being right at exactly one place — the point where somebody picks a recipe for a dataset and finds out at minute forty of a training run that the rows were preference pairs and the recipe wanted conversations. This page covers the two things the platform learned to say about the inside of a corpus: what shape its rows claim to be, and what a row *belongs to* so that a split can hold a whole design out rather than a random sample of one. ## The nine shapes A shape is **declared, not inferred**, and the checker only tests the claim. Inference was the alternative and it is worse in the way that matters: a heuristic that reads three rows and guesses is right almost always, and the almost is a silent mis-train rather than a refusal. There is no `generic` and no `custom`. A format that means nothing cannot be checked and would only be a field to fill in; a corpus whose shape is not here simply declares none, and everything else works as it always did. **`messages`** — chat SFT. A non-empty list, each entry carrying a non-empty `role` and `content`. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"messages": [{"role": "user", "content": "Explain clock domain crossing"}, {"role": "assistant", "content": "A CDC is ..."}]} ``` **`prompt_completion`** — plain supervised text. Both keys non-empty strings. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"prompt": "Write a 4-bit counter", "completion": "module counter(...);\n..."} ``` **`preference`** — DPO and friends. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"prompt": "Review this always block", "chosen": "The sensitivity list ...", "rejected": "Looks fine."} ``` **`fim`** — fill-in-the-middle, which is how a code model learns to complete *inside* a file rather than after it. Only `middle` has to be non-empty: `prefix` and `suffix` are context and either may legitimately be empty at the edges of a file. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"prefix": "always @(posedge clk) begin\n", "middle": " q <= d;\n", "suffix": "end\n"} ``` **`repo_context`** — a completion with the rest of the repository in front of it. `files` is a non-empty list of `{path, content}` with **no repeated path**, plus a non-empty `target`. This is the shape RTL-Repo and every repository-level code benchmark use, and the one that cannot be flattened into a prompt string without losing which file is which. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"files": [{"path": "rtl/uart.v", "content": "module uart(...);"}, {"path": "rtl/pkg.vh", "content": "`define W 8"}], "target": " assign tx = ...;"} ``` **`spec_rtl`** — a natural-language specification and the module that implements it. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"spec": "An 8-bit synchronous FIFO with full and empty flags", "rtl": "module fifo(...);\n..."} ``` **`testbench`** — a design and the bench that exercises it. The direction matters and is not symmetric with `spec_rtl`: here the model writes the *bench*, which is a verification engineer's job. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"dut": "module alu(...);\n...", "testbench": "module tb_alu;\n initial begin ..."} ``` **`log_root_cause`** — a tool or regression log and what went wrong. The manufacturing-side shape as much as the design-side one. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"log": "ERROR: [Synth 8-3352] multi-driven net ...", "root_cause": "The reset is driven from two always blocks"} ``` **`script`** — an EDA scripting task: Tcl, a Makefile, a constraint file. Kept apart from `prompt_completion` because what makes it correct is that the script runs, not that the text reads well. ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} {"request": "Constrain the 100 MHz input clock", "script": "create_clock -period 10.000 [get_ports clk]"} ``` A repeated path inside `repo_context` is refused rather than tolerated: whichever copy the framework keeps decides what the model sees, so the two entries would disagree about the same file in the same example. ## What the checker may look at Shape, and nothing else. Which keys are present, that they hold strings rather than nested structures, that a pair has both halves. Nothing about content, quality, language or length — those belong to the [quality report](/en/guides/datasets), and a format check that started judging content would be a second opinion about data you own. ## Conformance is a ratio, not a gate ```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} from tuneplane.data.formats import DataFormat, conformance, parse report = conformance(rows, DataFormat.SPEC_RTL) print(report.rows, report.conforming, report.ratio) for row_number, problem in report.examples: print(row_number, problem) ``` `conformance` takes any iterable of rows, so the caller decides how much to read. It reports how many rows it saw, how many conformed, the ratio, and at most ten complaints with their row numbers — a corpus with a million broken rows has one problem, not a million. Two deliberate refusals: * **It never becomes a verdict.** A gate that fires on any finding is a gate somebody turns off in a week, and a real corpus has malformed rows in it that nobody wants to argue about. What you are owed is the number and the first few examples. * **An empty file has no ratio.** `ratio` is `None` over zero rows, never `1.0`. Nothing was checked, and "perfect" is the wrong way to say that. `parse` returns `None` for an absent format — the ordinary case, and not an error — and raises with the sorted list of known names for one it does not recognise. ## Splitting an engineering corpus at random leaks Every contamination check the platform already has compares **text**. `tp dataset check` matches normalised 13-grams; the fingerprint comparison intersects sampled n-gram hashes. Both answer the question they were built for, and neither can see the failure that costs an engineering team its evaluation. Engineering corpora are full of near-variants of one artefact: eight revisions of a UART, a module and the wrapper that instantiates it, the same testbench parameterised four ways. Two revisions of one module genuinely share almost no n-grams while being, for the purpose of measuring what a model learned, **the same problem**. Split those at random and the evaluation set holds siblings of the training set. Every row is distinct. No duplicate ratio fires. The shingle overlap reads clean. The held-out score then measures recall of something the model has already seen, the number goes up, and the model is no better. So the check compares identity instead of text: whatever `group_key` names — a design, a repository, a case number — and the answer is the share of evaluation *groups* that also appear in training. ```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} from tuneplane.data.contamination import group_leak leak = group_leak(train_groups=["uart", "spi"], eval_groups=["uart", "pcie"]) leak.ratio # 0.5 -- half the held-out designs were trained on leak.clean # False leak.examples # ("uart",) ``` Three properties worth knowing: * **It is counted over distinct groups, not rows.** One design appearing in a thousand evaluation rows is one leak. Counting it a thousand times would let a corpus's row distribution decide how serious its leak looks. * **`ratio` is `None` when nothing is held out**, never `0.0`, for the same reason an empty conformance report has no ratio. * **It needs no text at all.** Two sets and an intersection, so it can run on metadata before either side is read. Zero is the only value worth aiming at for a held-out set. Anything above it means part of the score is recall. ## Declaring the group key `spec.data.quality` is the JobSpec block that carries the ceilings on the training data — the dual of the evaluation gates, which judge whether the model that came out is good enough. Two of its fields are about lineage: | Field | What it means | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `group_key` | A metadata field naming the *thing* a row belongs to. Empty means row-level, which is the previous behaviour and the right default for a corpus of independent examples | | `max_group_leak_ratio` | Ceiling on the share of groups appearing on both sides of a split, `0`–`1` | ```yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} data: quality: max_duplicate_ratio: 0.1 group_key: design max_group_leak_ratio: 0.0 ``` The contract refuses two inconsistent specs outright: * A ratio outside `0`–`1`. * `max_group_leak_ratio` without `group_key` — without it every row is its own group and the ceiling could never be exceeded, which is a line that never gets judged. The same rule already binds `max_overlap_ratio` to `overlap_with`. All four solutions that ship declare a split field: `design` for the two RTL ones, `repo` for the script assistant, and `case` for log triage, where the same failure reported twice is one fact. `tp solution show` prints it as **split by**; see [solutions](/en/guides/solutions). ## What is wired up, and what is not Being exact about this matters more than the feature does. | Piece | Status | | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The nine shapes and their validators | In the SDK, at `tuneplane.data.formats`, with tests | | `group_leak` | In the SDK, at `tuneplane.data.contamination`, with tests | | `group_key`, `max_group_leak_ratio` | Parsed and validated by the JobSpec contract | | A solution's `data_format` | Checked against the nine at manifest load | | `max_duplicate_ratio`, `max_overlap_ratio` | Enforced at submission against the stored quality report and fingerprints | | A group-leak check at submission | **Not implemented.** Nothing computes the ratio, so the declared ceiling refuses nothing | | A format declared on a dataset version | **Not implemented.** `tp dataset push` has no format flag, and no dataset field carries one | | A `data.quality` block in an experiment config | **Not carried by `tp submit`.** The CLI builds the JobSpec's data section from the dataset and path arguments and leaves `quality` at its defaults, so the block reaches the platform only from a JobSpec constructed against the API | So today the nine names are a vocabulary a Solution declares and a library check you can run over your own rows — in a preprocessing script, or in CI before a push. `group_key` is the same: declaring it records the intent and refuses an inconsistent spec; computing the leak is `group_leak`'s job and calling it is yours. That is a smaller claim than "the platform stops a leaking split", and it is the true one. ## Next [Datasets](/en/guides/datasets) · [Solutions](/en/guides/solutions) · [RTL benchmarks](/en/guides/rtl-benchmarks)