Skip to main content
TunePlane data page

Dataset versions with file manifests and quality metrics.

Two tabs at /data, because both hold material a job reads and they differ only in how much of it can come back out. The old /datasets and /volumes links land here.

​
The Datasets tab

​
The quality report

Computed once, at push time, over the full dataset — sampled only for very large ones. It counts what a machine can count and does not pretend to judge content.
Field drift is the one that catches real bugs. A few hundred records missing the field your loss reads will not fail the job; they will quietly train on nothing and drag the curve.

​
Protected datasets

A dataset can be governed so that a trainer may use it without being able to read it. On such a dataset the console shows identity, version, schema, lineage and authorisation state — and no preview, no sample records, and no download. That is enforced on the server, not by hiding buttons. Logs, validation samples and artifacts are covered by the same policy, so they cannot become an alternative export path.

​
Uploading from the console

Upload dataset on the dataset list creates a dataset and its first version; Upload a version on a dataset page adds one to an existing dataset. Both ask the same three things tp dataset push does — which dataset, which version, which files — and both send the bytes straight to object storage on a presigned URL. index.json, which is what makes a version complete and therefore immutable, is written only after every file has landed: an interrupted upload leaves the version number free rather than a version that claims to be finished. A README.md among the files is published as the dataset card, exactly as a CLI push does it. The quality report and the contamination fingerprint are not computed in the browser — a tab will not read gigabytes to slice n-grams. A worker computes them from the bucket shortly after the version appears, in the same pass that scans it, and writes them beside the version where a CLI push would have. Past a per-version read budget the report marks itself sampled rather than publishing a duplicate rate measured over part of a file.
A console upload refuses a file over 512 MB: a browser has to hold a whole file in memory to checksum it, and the checksum is what the job side verifies a download against. tp dataset push streams, so it has no such limit — and it computes both reports over everything at push time rather than up to a budget afterwards.

​
What you can do

  • Reference a version in a config or with --train-dataset.
  • Push a new version from this page, or with tp dataset push.
  • Check quality before pushing with tp dataset check and tp dataset quality.
See datasets for the full workflow.

​
The Volumes tab

A Volume is a named directory a job reads through a read-only mount. It has no versions: files are added and removed in place, and a reference names the volume rather than a snapshot of it. What is never there: a bulk download, a convenience endpoint, or a preview at any setting when the owner has not turned it on. Listing structure is not the same as reading content, and the split is deliberate — a job can already read every word of a volume it mounts, so hiding the file names would protect nothing. A volume’s files are never versioned, so to produce something reproducible from one, run a Processing Run that publishes a dataset version.