About the structure of dataset, directory path, URL

Top / Developers / About the structure of dataset, directory path, URL

This page is automatically translated.

About ZFS data set

The data set of ZFS used as the file system of TeraCLOUD has a nested structure like a directory.


Since it is possible to have different properties for each data set, you can acquire values such as capacity, create snapshot, refer to it, and control unit of various functions such as rollback, difference, quota, compression management, encryption in the future .

In addition, since properties inherit in parent and child, the basic setting inherits the parent's, and the usage capacity etc. includes the lower rank.

 These are managed in a format like a pass,


In fact, it is mounted on the directory path as follows:


Mount points mean placement to the path.

In the above data set structure, if there is a directory called. / Backup / foo, this is just a directory named foo installed in the backup dataset, and there is a data set / backup / foo There is nothing. Creating a data set is more expensive than creating a directory, so it does not create a data set for each directory.

When creating a data set rather than a directory, when you want to manage data set resources uniquely, for example you want to take a snapshot unique, want to see it, want to roll back, want to do a difference in the future For example, if you want to change sort of control category, it will be used.

Also, in the above, the data set is the generic name of the following.

Therefore, the data set mentioned above is the file system of the data set.

About File system

A file system in ZFS means a data set that can be mounted in a directory path.

About snapshot

A snapshot is a function that can hold an image of a data set - file system at a certain point in time as it is. This can be acquired in "almost instantaneous". Even if data is written after creation of the snapshot, it is only necessary to record the difference in the file system. Therefore, the current state can be maintained with a very low cost method.

The snapshot can be acquired for the file system of the data set. It is necessary to do it for each data set unit, but if you set recursive, you can acquire it at the same time, including the lower data set - file system. A snapshot of snapshot can not be acquired.

For reference, there is a reference only directory under the /.zfs/snapshot located immediately under each data set, with the snapshot name. Since snapshots can only be referenced by files in the dataset itself, in the example above / files snapshot files are only under /files/.zfs/snapshot. If you create a data set called / files / foo, snapshots of the files placed in the directory below. / Foo will be placed under / files / foo / .zfs / snapshot and / files / .zfs / snapshot / It does not exist under <snapshot name> / foo.

The snapshot keeps the state at that time to the last, it is not a backup. However, it is very useful for restoring from logical mistakes.

In TeraCLOUD, only 100 snapshots can be acquired for each user.

Although it is possible to create a snapshot almost instantly, deletion, especially opening up the occupied data area takes time.

About rollback

It is a function to rewind the file system to the point of the snapshot. It discards everything written after the snapshot and restores the file system to its original state.

In the case of rolling back, since all the progress snapshots are lost, it is impossible in principle to cancel the rollback.

Current TeraCLOUD is unsupported and will be supported in the future.

About clones


Cloning is the ability to create a new file system based on a snapshot.Since only the differences from the snapshot and afterwards are retained, it can prevent the capacity from being overwhelmed if it contains similar data.


Current TeraCLOUD is unsupported and will be supported in the future.

File system level encryption

It is a function of encrypting AES etc. on a file system basis.

This file system requires a password or key when mounting. If you lose your password or key, the data is encrypted, so you can not restore it.

Current TeraCLOUD is unsupported and will be supported in the future.

Data set structure in TeraCLOUD

TeraCLOUD users have one user data set - file system at the time of account creation and are mounted in the directory. And under it there is a data set - file system.

This data set - file system is mounted on the mount point of the same name, and the user's data is stored.

Of these, the data set that can be accessed via the URL is only the one mapped with the mount path, which is accessed using the HTTPS (WebDAV) protocol. Only / dav / can access the general user freely via the browser, and below / backup becomes an area which can be used uniquely by the application.

/backup/backup/backup *WebDAV with BASIC authentication only; cannot be viewed from a browser. The same applies to the following paths.
/files/files/dav/ *For BASIC authentication
/v2/dav/ *Browsable from browser.
/someappdata/someappdataNone *In general it should not be made
/someappdata2/someappdata2None *In general it should not be made

The user's root data set The following are all the user's possessions, and the user's used capacity means the used capacity including all the lower (descendants) of the route data set.

However, since the data set name is not originally prefixed with /, the route is for convenience and it is a null string on the API.

The data set length is 64 characters at the maximum.

In addition, all of the data sets can be managed with the user's ID and password, and up to 32 data sets can be created anywhere in the REST API.

In TeraCLOUD, the file system type data set is mounted at the same location as the data set name from the root. In other words, when you create a data set named someappdata, it mounts in the path / someappata, and when you create a data set called someappdata / foo under it, it mounts in the path / someappdata / foo.

However, when you try to create a dataset called files / foo, you already can not create a dataset if the user creates a directory named foo and installs files.

Role of each data set

As mentioned above, the TeraCLOUD data set has meaning in name.

/NoneIt is the parent of the user's area. Since the total capacity measurement will be referred from here, this area will not be used directly, except for some users.
/ files/dav/ , /v2/dav/General area of ​​user. It can be accessed from browsers and general web dub clients. However, users created before April 2015 do not have this data set and are placed directly in the route data set.
It is OK to create the data set below, but since this is generally the area where the user freely holds the data, if the user has created a directory with the same name, it will be created as it is Because you can not do, you need attention.
/backup/backupIt is used when an application provides a specific area that cannot be easily touched by the user. Uses include backup data, application data storage, etc.
Other DATASET NamesNoneIt is not recommended to create a random name at this time. In the future, there is a possibility that a mechanism for special access to these areas may be created as requested.

Regarding application-specific data sets

The areas accessible in /dav/ and /v2/dav/ are easily accessible by the user's browser and may be manipulated by mistake, regardless of the user's intentions.

In some applications, that can lead to fatal problems, so TeraCLOUD allows users to create areas that are not easily accessible by browsers and others.

In addition, ZFS, the file system employed by TeraCLOUD, allows application developers to create datasets for each role and gain various advantages (but up to 10 per user) because various management and operations are performed on a dataset (file system) basis.

If you want to create a dataset (file system) in an area under /backup of the user, you can create it with the following rules.

VendorID, AppName can be any name you want, but it should be up to 64 characters in total length, alphanumeric only. Symbols containing spaces are not allowed, so it is recommended that the VendorID and AppName be some kind of simple name.
Note that this is not the case if the user dares to specify that path to a general-purpose WebDAV client, since it is not referable from a browser.
Example: /backup/JUSTPLAYER/tctool etc.
Use it when you don't have to or don't want to put in the vendor's name, or the app's name.
The name of the data.XXXX must be unique to place it in the same hierarchy with all vendors and app names. At creation, you must use (unique) in the put dataset of dataset api to suffix the data.XXXX to be created randomly by the system.

Note that these data sets are not absolutely inaccessible to the user, as they can be deleted by the user using the "Data Set Editor" in the browser.

(Reference) URL-accessible mechanism for datasets

In TeraCLOUD, there is a unique relationship between a dataset (file system) name and a mount point, but there is no unique relationship between a dataset and a published URL.

Usually, the area that users can see in the web interface or WebDAV can be accessed from the path named "/dav/".

URL layer/dav/
URL path conversion rules (fixed)/dav/ → /dav/
Directory Path Layer/dav/
/dav/ → /files/
Mounting path rules (fixed)/files → files
Data Set Layerfiles

URL path: /dav/" is URL mapped to "File path: /files/". The "Dataset: files" is mounted to "File path: /files/". (*For users created before April 2015, there is no files dataset and a directory named /files/ is created in the root dataset.)

The above URL mapper is considered to be switched in the upcoming mapper API, so it allows for a wide range of applications, such as preserving the Clone at a certain point in time. However, this feature has many side-effects, such as authentication cache, so the implementation and release of this feature is currently under consideration.