Data Admin Guide

Creating a dataset

Datasets start as a draft and are published once their resources and metadata are ready. Drafts never appear in search results or dataset listings until published.

Log in to the dashboard
Click “Create Draft” — the name (URL identifier), organization, and license can be set later
Add resources on the edit page (file upload / external URL) — the pipeline runs while the dataset is still a draft, so you can check previews before publishing
Finalize the metadata and click “Save & Publish”

Basic information

Field	Description	Required to publish
Name (URL slug)	Alphanumeric, hyphens, underscores, periods	Yes
Title	Display title	-
Description	Detailed description of the dataset	-
Organization	The organization that owns the data	Yes
License	Data license	Yes
Public / Private	Private datasets are only visible to organization members	-

Fields marked “Required to publish” may stay unset while the dataset is a draft, but all three must be set before it can be published. While the name is blank, an auto-generated identifier is used as a placeholder (the dataset cannot be published with it).

Managing and deleting drafts

Drafts are managed in the dashboard “Drafts” tab. They can be edited by their creator and site admins, plus the editors of the owning organization once one is set.

Deleting a draft skips the trash and permanently deletes it together with its resource files (it cannot be restored). If the deletion fails partway, the draft stays in the list marked “Deletion incomplete”, and “Retry Delete” completes the permanent deletion.

Managing resources

Resources are files or URLs associated with a dataset.

Adding resources

File upload — Upload directly from the browser (up to 100 MB)
External URL — Specify a URL to an external file

Grouping resources into sections

When a dataset has many resources, section headings group them. “Add section” in the resource list names a new, empty heading at the end.

A heading is a divider. Drag it into place and the resources from there down to the next heading belong to the section. Moving a heading never changes the order of the resources
Moving a heading down hands the resources it passes to the section above (or to the root); moving it up takes them in. Set down inside another section it splits that section there; set down directly on a heading it takes nothing
A dragged resource joins the section of the resource above it. Dropped on a heading it becomes that section’s first member; dropped at the top of the list it belongs to no section
Rename and dissolve (dissolving keeps the resources) from the buttons on the heading. A section name is used once per dataset
A heading with no members is not saved until a resource enters it. A resource added at the end, or a dropped file, goes into the section the end is in
Like the order, “Save order” commits the change

A section is a display divider on the dataset page and does not change who can see a resource. A / in the name nests the headings on the public page (2024/Tokyo shows “Tokyo” under “2024”, up to three levels). Section names are searchable.

Supported formats

KUKAN supports many formats including CSV, TSV, JSON, GeoJSON, XML, PDF, and Excel.

Uploaded files are automatically processed through the pipeline:

Text file encoding detection and conversion
CSV/TSV to Parquet conversion (for table preview)
ZIP file manifest generation

Version history

Every time a resource’s file is replaced, the content it replaced is kept as a version. You can download any past version, and for CSV/TSV you can see how many rows were added or removed between two versions — and, with a primary key declared, how each row changed.

A version is created only when the content actually changed. Re-fetching an external URL that still serves the same bytes adds no version, and the resource’s processing status says “No new version — the file matches the content already stored”.

Viewing the version history

The resource edit page has a Version history section. Versions are listed newest first, each row showing the version number, creation time, size, origin (Upload, Fetched, or Re-published v○), and the primary key the version was loaded under.

Every version can be downloaded exactly as it was. The version number is added to the filename.

Declaring a primary key

The Primary key control on the resource edit page names the columns used to match a row to the same row in another version (a combination of columns works too). With a key, a diff can count the rows that changed.

The columns you choose — the combination, if more than one — have to be present in the content, hold no empty values, and not repeat across rows. Where they are not met, the reason is shown before you apply
Applying creates one version, and versions from then on are loaded under this key. Diffs of past versions do not change
When content arrives that does not satisfy the key, that version is left out of row-level diffs. The reason (a column is missing, has empty values, or does not identify a row) appears in the version list and on the diff panel
A column that does not keep pointing at the same row — a sequence number reissued on every publication, say — makes a poor key. If that is all there is, declaring no key is the safer choice: editing one row then shows as one addition plus one removal, but two different things are never presented as the same row

Viewing a diff

The Diff control on a version’s row fetches the diff against the preceding version. It is computed when you open it, so simply listing the versions does not run anything.

Shows N rows added / N rows removed, with a few representative rows from each side
With a primary key declared, it also shows “N rows changed” with representative rows before → after. Without one, editing one row appears as one addition plus one removal — KUKAN does not guess a correspondence and present it as change history
When the columns changed (added, removed, retyped, or reordered), the rows cannot be lined up and the panel says so instead

Diffs are available between CSV/TSV versions of 100 MB or less. When one is unavailable, the reason is shown.

Message	Meaning
This is the first version, so there is nothing to compare against.	v1 has no predecessor
v○ has no row-level snapshot, so there is nothing to compare against.	A PDF and similar, or a version created before the migration
One of the versions has been deleted, so a diff is not available.	Either side was deleted
A key column is not in v○, so its rows cannot be identified and no diff is available.	A column named in the primary key is missing from that version
A key column of v○ has empty values, so its rows cannot be identified and no diff is available.	The key columns contain empty values
The key of v○ does not identify a row, so no diff is available.	Rows repeat the same key values

The three key-related reasons clear up from the next version on once the key is corrected. A version left out is never loaded retroactively.

Stopping and reverting

For when the wrong file was uploaded, there are three operations of increasing destructiveness. Each is chosen explicitly.

Operation	Current content	Versions	Auth
Stop	Left alone	Left alone	Org editor
Stop and revert	Back on the newest saved version’s content	The restored content is issued as a new version	Org editor
Delete This Version	Falls back to the newest remaining version, if this was live	Content erased, the record remains	Sysadmin

Stop and revert puts the live content back on the most recent saved version. The restored content is issued as a new version, so the history gains a row with origin “Re-published v○” and no past version is removed. If there is no earlier version, the resource is left with no file (its metadata remains).

The preview and the search index are discarded as part of the revert and rebuilt automatically from the restored content (the rebuild runs in the background, so it takes a moment to appear). A warning appears only when discarding them, or starting the rebuild, did not succeed — then “Rebuild preview & index” finishes the job.

Deleting a version

Use this when the content itself has to become unobtainable — personal data that should never have been published, for example. Only site administrators can delete a version, a reason is required, and it is recorded in the audit log.

The version becomes unobtainable and its canonical file is deleted. Rows may remain in the version store’s internal storage, and their deletion is not guaranteed (another resource’s history can reference the same file)
Content another version holds is not erased. Each version keeps its own copy, so deleting one leaves the others serving that content — and if deleting the live version falls back onto a version holding the same content, the resource serves it again. The confirmation dialog names those versions; delete them individually if they must go too
Where storage versioning is in use, the set-aside objects disappear once their retention period (30 days by default) elapses
Deleting the live version puts the resource back on the newest remaining one; with none left, the resource is left with no file. Deleting a version that is not live changes nothing about what is served, its preview, or its search index
The history keeps a “Deleted” row and the date. The reason you type is not shown there and is not returned by the API — it goes to the audit log, which today is only readable in the database. Treat it as a note to the operators, and do not restate in it what had to be removed
This cannot be undone

One-time migration

Right after version history is enabled, resources that existed beforehand have no versions yet. A Backfill versions item appears on the site administrator’s dashboard — run it.

Layer 1: records each unversioned resource’s current file as v1 (nothing is re-fetched or re-indexed)
Layer 2: loads the current version of each tabular resource into the diff store, so diffs become available from the next update onwards
Revert conversion: converts resources still holding versions an earlier revert set aside, so the latest-version label agrees with what is being served

The item disappears once all counts reach zero. Replacing a file before running it loses the previous content — it is never kept as a version.

AI metadata suggestions

Once you add resources and their pipeline processing completes, the “Let AI suggest metadata” button on the edit page can propose a title, notes, tags, and per-resource names and descriptions (only when AI is enabled for the site — see the System Administrator Guide).

Prerequisite: at least one resource whose pipeline has completed. Otherwise the button is disabled and prompts you to upload resources.
How to use:
1. Click “Let AI suggest metadata” on the edit page (generation can take from tens of seconds to a few minutes).
2. In the dialog, compare the current values with the AI proposals, grouped into “Dataset” and “Resource” sections.
3. Turn on the adoption toggle for each field you want to adopt (all toggles start off — overwriting existing values is opt-in).
4. Optionally edit an adopted proposal inline before applying.
5. Click “Adopt”.
How adoptions apply: title, notes, and tags are applied to the edit form and committed when you save the dataset; resource names and descriptions are saved immediately on adoption.
What suggestions are based on: CSV/TSV use the columns and the first rows of content; PDF, Excel, and similar files use the filename and format. The generated text follows your display language.
Suggestions are never persisted (only adopted values are saved as ordinary updates). A per-user rate limit prevents overuse.

Organization management

Organizations are the ownership unit for datasets.

Member management

Organization admins can add, remove, and change member roles.

Role	View datasets	Edit datasets	Delete	Restore / Purge	Manage members
Member	Yes	-	-	-	-
Editor	Yes	Yes	Yes	-	-
Admin	Yes	Yes	Yes	Yes	Yes

Deleted datasets are visible to organization members in the dashboard “Deleted” tab. Only admins can restore or permanently purge them.

Deleting, restoring, and purging an organization

The organization management page toggles between “Public” and “Deleted” cards (only site admins can view the “Deleted” set).

Delete (soft delete): Moves the organization to the trash. It must have no active datasets — delete or reassign them to another organization first (the delete button stays disabled while any remain).
Restore: Brings a “Deleted” organization back to active (site admins only).
Purge (permanent delete): Permanently erases the organization and all of its trashed datasets, including their files (irreversible, site admins only). It runs asynchronously as a background job, so the organization may still appear in the list until the job finishes.

Categories (Groups)

Categories classify datasets across organizations. Assign categories when creating a dataset.

Category member management

Categories also have member management with the same role structure as organizations.

Role	View category	Add datasets	Manage members
Member	Yes	-	-
Editor	Yes	Yes	-
Admin	Yes	Yes	Yes