About the structure of dataset, directory path, URL

Top / Developers / About the structure of dataset, directory path, URL

About ZFS data set

The data set of ZFS used as the file system of TeraCLOUD has a nested structure like a directory.

Since it is possible to have different properties for each data set, you can acquire values such as capacity, create snapshot, refer to it, and control unit of various functions such as rollback, difference, quota, compression management, encryption in the future .

In addition, since properties inherit in parent and child, the basic setting inherits the parent's, and the usage capacity etc. includes the lower rank.

 These are managed in a format like a pass,

DATASET name
/
/backup
/backup/data.ZsXw
/backup/data.WtPO
/files

In fact, it is mounted on the directory path as follows:

DATASETMOUNTPOINT
/./
/backup./backup
/backup/data.ZsXw./backup/data.ZsXw
/backup/data.WtPO./backup/data.WtPO
/files./files

Mount points mean placement to the path.

In the above data set structure, if there is a directory called. / Backup / foo, this is just a directory named foo installed in the backup dataset, and there is a data set / backup / foo There is nothing. Creating a data set is more expensive than creating a directory, so it does not create a data set for each directory.

When creating a data set rather than a directory, when you want to manage data set resources uniquely, for example you want to take a snapshot unique, want to see it, want to roll back, want to do a difference in the future For example, if you want to change sort of control category, it will be used.

Also, in the above, the data set is the generic name of the following.

Therefore, the data set mentioned above is the file system of the data set.

About File system

A file system in ZFS means a data set that can be mounted in a directory path.

About snapshot

A snapshot is a function that can hold an image of a data set - file system at a certain point in time as it is. This can be acquired in "almost instantaneous". Even if data is written after creation of the snapshot, it is only necessary to record the difference in the file system. Therefore, the current state can be maintained with a very low cost method.

The snapshot can be acquired for the file system of the data set. It is necessary to do it for each data set unit, but if you set recursive, you can acquire it at the same time, including the lower data set - file system. A snapshot of snapshot can not be acquired.

For reference, there is a reference only directory under the /.zfs/snapshot located immediately under each data set, with the snapshot name. Since snapshots can only be referenced by files in the dataset itself, in the example above / files snapshot files are only under /files/.zfs/snapshot. If you create a data set called / files / foo, snapshots of the files placed in the directory below. / Foo will be placed under / files / foo / .zfs / snapshot and / files / .zfs / snapshot / It does not exist under <snapshot name> / foo.

The snapshot keeps the state at that time to the last, it is not a backup. However, it is very useful for restoring from logical mistakes.

In TeraCLOUD, only 100 snapshots can be acquired for each user.

Although it is possible to create a snapshot almost instantly, deletion, especially opening up the occupied data area takes time.

About rollback

It is a function to rewind the file system to the point of the snapshot. It discards everything written after the snapshot and restores the file system to its original state.

In the case of rolling back, since all the progress snapshots are lost, it is impossible in principle to cancel the rollback.

Current TeraCLOUD is unsupported and will be supported in the future.

About clones

A clone is a function to create a new file system based on a snapshot.

Since only the difference from the snapshot is retained, if the data contains similar data, it is possible to prevent the compression of the capacity.

Current TeraCLOUD is unsupported and will be supported in the future.

File system level encryption

It is a function of encrypting AES etc. on a file system basis.

This file system requires a password or key when mounting. If you lose your password or key, the data is encrypted, so you can not restore it.

Current TeraCLOUD is unsupported and will be supported in the future.

Data set structure in TeraCLOUD

TeraCLOUD users have one user data set - file system at the time of account creation and are mounted in the directory. And under it there is a data set - file system.

This data set - file system is mounted on the mount point of the same name, and the user's data is stored.

Of these, the data set that can be accessed via the URL is only the one mapped with the mount path, which is accessed using the HTTPS (WebDAV) protocol. Only / dav / can access the general user freely via the browser, and below / backup becomes an area which can be used uniquely by the application.

DATASETMOUNTPOINTURL
//None
/backup/backup/backup
/backup/JUSTPLAYER/backup/JUSTPLAYER/backup/JUSTPLAYER
/backup/JUSTPLAYER/appname/backup/JUSTPLAYER/appname/backup/JUSTPLAYER/appname
/backup/data.ZsXw/backup/data.ZsXW/backup/data.ZsXW
/backup/data.WtPO/backup/data.WtPO/backup/data.WtPO
/files/files/dav/ * Viewable from browser
/someappdata/someappdataNone *In general it should not be made
/someappdata2/someappdata2None *In general it should not be made

The user's root data set The following are all the user's possessions, and the user's used capacity means the used capacity including all the lower (descendants) of the route data set.

However, since the data set name is not originally prefixed with /, the route is for convenience and it is a null string on the API.

The data set length is 64 characters at the maximum.

In addition, all of the data sets can be managed with the user's ID and password, and up to 32 data sets can be created anywhere in the REST API.

In TeraCLOUD, the file system type data set is mounted at the same location as the data set name from the root. In other words, when you create a data set named someappdata, it mounts in the path / someappata, and when you create a data set called someappdata / foo under it, it mounts in the path / someappdata / foo.

However, when you try to create a dataset called files / foo, you already can not create a dataset if the user creates a directory named foo and installs files.

Role of each data set

As mentioned above, the TeraCLOUD data set has meaning in name.

DATASET nameURLrole
/NoneIt is the parent of the user's area. Since the total capacity measurement will be referred from here, this area will not be used directly, except for some users.
/ files/ dav /General area of ​​user. It can be accessed from browsers and general web dub clients. However, users created before April 2015 do not have this data set and are placed directly in the route data set.
It is OK to create the data set below, but since this is generally the area where the user freely holds the data, if the user has created a directory with the same name, it will be created as it is Because you can not do, you need attention.
/ backup / <VendorID> / <AppName>/ backup / <VendorID> / <AppName>Use this application to prepare a specific area that the user can not easily touch.
Uses backup data, application data storage etc.However, although it is impossible to refer to from the browser, if the user intentionally designates the path as a general-purpose WebDAV client, be careful that it is not limited to that.You can freely use the dataset created below, but you need to follow the rules of vendor name and application name. In addition, it is recommended that vendor name and application name be some sort of simple name because it can be up to 64 characters, alphanumeric characters only. Note that spaces are not allowed.
Example: / backup / JUSTPLAYER / tctest etc.
/backup/data.XXXX/backup/data.XXXXXXXX contains a free string. Whether XXXX is free or not can be created by attaching (unique) when creating dataset api, and the URL that can be created in the return value is returned.If you do not want to create vendor name and application name, use this dataset name.
Other DATASET NamesNoneIt is not recommended to create a random name at this time. In the future, there is a possibility that a mechanism for special access to these areas may be created as requested.

/ dav /, a mechanism that allows URL access

As mentioned above, there is a unique relationship between the dataset name and the mount point, but there is no unique relationship between the dataset and the published URL.

Normally, the user can access the web interface and the area that can be viewed with WebDAV from the path "/ dav /".

URL layer/ dav /
URL path conversion rule (fixed)/ dav / → / dav /
Directory path layer


/ dav /
Mapper/ dav / → / files /
/ files
Mount path rule (fixed)/ files → files
Dataset layerfiles

Path: / dav / is mapped to path: / files /, and dataset: files are mounted on path: / files /. However, users created before April 2015 have no files dataset, and a directory called / files / is created in the root dataset.

Although the above mapper is considered to be switched by mapperAPI which will be published in the future, this function has many side effects and is currently under consideration.