> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tuneplane.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Datasets
> Versioned dataset assets, their file manifests, and the data quality report.
## What you see
| Region | What it shows |
| ----------------- | ---------------------------------------------------------------------------------- |
| **Versions** | Every immutable version, with its checksum |
| **File manifest** | What is in the version, with per-file sizes |
| **Preview** | Sample records, for formats the console can render — and only where policy permits |
| **Data quality** | Computed at push time over the whole dataset |
| **Usage** | Which jobs referenced this dataset, which is what makes a version safe to delete |
## The quality report
Computed once, at push time, over the full dataset — sampled only for very large ones. It counts
what a machine can count and does not pretend to judge content.
| Metric | Why it is there |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **Records** | The number you thought you uploaded, confirmed |
| **Duplicates** | Near-duplicates and byte-identical ones, counted separately. A high rate means the model will overfit to repeated samples |
| **Empty** | Records with no usable content |
| **Length p50 / p95** | Size `max_seq_len` from p95, not from the mean — the mean will truncate a fifth of your data |
| **Field drift** | Records whose field set differs from the majority, and which fields they are missing |
Field drift is the one that catches real bugs. A few hundred records missing the field your loss
reads will not fail the job; they will quietly train on nothing and drag the curve.
## Protected datasets
A dataset can be governed so that a trainer may *use* it without being able to *read* it. On such a
dataset the console shows identity, version, schema, lineage and authorisation state — and no
preview, no sample records, and no download.
That is enforced on the server, not by hiding buttons. Logs, validation samples and artifacts are
covered by the same policy, so they cannot become an alternative export path.
## Uploading from the console
**Upload dataset** on the dataset list creates a dataset and its first version; **Upload a version**
on a dataset page adds one to an existing dataset. Both ask the same three things `tp dataset push`
does — which dataset, which version, which files — and both send the bytes straight to object
storage on a presigned URL. `index.json`, which is what makes a version complete and therefore
immutable, is written only after every file has landed: an interrupted upload leaves the version
number free rather than a version that claims to be finished.
A `README.md` among the files is published as the dataset card, exactly as a CLI push does it.
The quality report and the contamination fingerprint are not computed in the browser — a tab will
not read gigabytes to slice n-grams. A worker computes them from the bucket shortly after the
version appears, in the same pass that scans it, and writes them beside the version where a CLI
push would have. Past a per-version read budget the report marks itself **sampled** rather than
publishing a duplicate rate measured over part of a file.
A console upload refuses a file over 512 MB: a browser has to hold a whole file in memory to
checksum it, and the checksum is what the job side verifies a download against. `tp dataset push`
streams, so it has no such limit — and it computes both reports over everything at push time
rather than up to a budget afterwards.
## What you can do
* **Reference a version** in a config or with `--train-dataset`.
* **Push a new version** from this page, or with `tp dataset push`.
* **Check quality before pushing** with `tp dataset check` and `tp dataset quality`.
See [datasets](/en/guides/datasets) for the full workflow.