About the structure of data set, directory path, URL

Top / Developers / About the structure of data set, directory path, URL

About ZFS data set

The ZFS data set, which is utilized as TeraCLOUD's file system, has a hierarchical structure similar to that of a directory.

dataset.jpg

Because each data set can have distinct features, you can acquire values such as capacity, create snapshots, refer to it, and use it as a control unit for various tasks like rollback, difference, quota, compression management, and encryption in the future.

In addition, because properties inherit from the parent and child, the basic setting inherits from the parent, and the usage capacity and other properties include the lower rank.

These are managed in a format like a pass.

DATA SET name
/
/backup
/backup/data.ZsXw
/backup/data.WtPO
/files

It is mounted on the following directory path below:

DATA SETMOUNTPOINT
/./
/backup./backup
/backup/data.ZsXw./backup/data.ZsXw
/backup/data.WtPO./backup/data.WtPO
/files./files

Mount points are in reference to where you are on the path.

In the aforementioned data set structure, if there is a directory called ./backup/foo, it is just a directory named foo placed in the backup data set, and there is a data set /backup/foo, then nothing exists. It is more expensive to build a data set than it is to establish a directory, hence it does not create a data set or each directory.

When you want to manage data set resources uniquely, such as taking a unique snapshot, seeing it, rolling back, or alter it in the future, you should create a data set rather than a directory. It will be utilized, for example, if you wish to modify the type of control category.

The data set in the preceding is the generic name of the following:

The data set mentioned above is the file system of the data set.

About File system

In ZFS, a file system is a collection of data that may be mounted in a directory path.

About snapshot

A snapshot is a function that can save an image of a data set - file system as it appears at a specific point in time. This can be obtained in "near-instantaneous" timing. Even if data is written after the snapshot is created, all that is required is to record the difference in the file system. As a result, the present state can be preserved using a low-cost way.

The snapshot of the data set's file system can be obtained. It must be done for each data set unit, but if it is set as recursive, you can do it all at once, including the lowest data set - file system. However, it is not possible to obtain a snapshot of a snapshot.

There is a reference-only directory with the snapshot name under /.zfs/ snapshot situated immediately under each data set. Because snapshots can only be referred by files in the data set, the /files snapshot files are only under /files/.zfs/snapshot in the example above. If you make a data set called /files/foo, it will create snapshots of the files in the directory below. /foo will be stored in the directories /files/foo/.zfs/snapshots and /files/.zfs/snapshot/. It will not exist under <snapshot name>/foo.

The snapshot preserves the status of the system at that point in time; it is not a backup. It is, nonetheless, extremely beneficial for recovering from logical error or accidental deletion of the original file.

In TeraCLOUD, only 100 snapshots can be acquired for each user.

Although snapshots can be created almost instantly, deletion, particularly when opening up occupied storage areas, takes time.

About rollback

It is a function to rewind the file system to the point of the snapshot. It discards everything written after the snapshot and restores the file system to its original state.

In the case of rolling back, since all the progress snapshots are lost, it is impossible in principle to cancel the rollback.

Current TeraCLOUD is unsupported and will be supported in the future.

About clones

Cloning is the ability to create a new file system based on a snapshot. Only the differences between the snapshot and afterwards are retained. As a result, if the data is comparable, it can avoid the capacity from being overburdened.

At this time, TeraCLOUD is currently unsupported, this is expect to change in the near future.

File system level encryption

This is a file system-based utility for encrypting AES and other algorithms.

When mounting this file system, you'll need a password or key. The data is encrypted, so you won't be able to recover it if you forget your password or key.

At this time, TeraCLOUD is currently unsupported, this is expect to change in the near future.


Data set structure in TeraCLOUD

At the moment of account creation, TeraCLOUD users have only one user data set - file system, which is mounted in the directory. There's also a data set - file system underneath it.

The user's data is kept on this data set - file system which is mounted on a mount point with the same name.

Only the data set mapped with the mount path, which is accessed via the HTTPS (WebDAV) protocol, can be accessed through URL. Only /dav/ has free browser answer to the general user, and below/backup becomes a region that can only be utilized by the program.

DATA SETMOUNTPOINTWebDAV URL
//None
/backup/backup/backup *WebDAV with BASIC authentication only; cannot be viewed from a browser. The same applies to the following paths.
/backup/JUSTPLAYER/backup/JUSTPLAYER/backup/JUSTPLAYER
/backup/JUSTPLAYER/appname/backup/JUSTPLAYER/appname/backup/JUSTPLAYER/appname
/backup/data.ZsXw/backup/data.ZsXW/backup/data.ZsXW
/backup/data.WtPO/backup/data.WtPO/backup/data.WtPO
/files/files/dav/ *For BASIC authentication
/v2/dav/ *Browsable from browser.
/someappdata/someappdataNone *In general it should not be made
/someappdata2/someappdata2None *In general it should not be made

The root data set of the user's belongings are listed below, and the user's used capacity refers to the whole capacity of the route data set, including all the lower (descendants) of the route data set.

The route is for convenience and is a null string on the API because the data set name is not initially precedes with a forward slash ( / ).

The maximum length of a data collection is 64 characters.

User IDs and passwords can be used to manage all data sets, and up to 32 data sets can be established anywhere in the REST API.

The file system type data set in TeraCLOUD is mounted in the same location as the root data set name. In other words, a data set named someappdata mounts in the path /someappdata, and a data set named someappdata/foo mounts in the path /someappdata/foo.

If the user were to create a directory named foo and install files, the user will be unable to construct a data set called files/foo.


Role of each data set

As previously mentioned above, the TeraCLOUD data set's name each have their own meanings and significance.

DATA SET nameURLrole
/NoneIt is the parent of the user's area. With the exception of a small number of users, this area will not be used directly because the overall capacity measurement will be referred from here.
/ files/dav/ , /v2/dav/The user's general area. Browsers and standard web dub clients can both access it. User accounts created prior to April of 2015, will not this data set and are instead stored in the route data set.
It is OK to construct the data set below, however, because this is typically where the user stores their data, if the user has already created a directory with the same name, it will be overwritten. Attention will be required if you are unable to do so.
/backup/backupIt is used when an application provides a certain area that the user cannot readily touch. Backup data, application data storage, and other applications are just a few examples.
Other DATA SET NamesNoneIt is not recommended to create a random name. In the future, there is a possibility that a mechanism for special access to these areas may be created as requested.

Regarding application-specific data sets

Regardless on the user's intention, the locations accessible in /dav/ and /v2/dav/ are easily accessible by the user's browser and may be altered by mistake.

Because this can cause fatal issues in some applications, TeraCLOUD allows users to construct zones that are not easily accessible by browsers or others.

Furthermore, because different administration and operations are handled on a data set (file system) basis, ZFS, the file system used by TeraCLOUD, allows application develops to build data sets for each role and obtain many advantages (limit is 10 per user).

If you want to construct a data set (file system) in a user's /backup area, apply the following rules below:

/backup/<VendorID>/<AppName>
VendorID, AppName can be any name you desire, however it should be up to 64 characters in total length, and exclusively alphanumeric. Because symbols with spaces are not permitted, it is recommend that the VendorID and AppName be something simple.
Because it is not referable from a browser, this is not the case if the user chooses to offer that path to a general-purpose WebDAV client.
Example: /backup/JUSTPLAYER/tctool etc.
/backup/data.XXXX
When you don't have to or don't want to put the vendor's or app's name in, use this method. To be placed in the same hierarchy as all vendors or applications, the name of the data.XXXX must be unique. You must use (unique) in the data set API's insert data set to suffix the data.XXXX so that it is generated at random by the system.

It's important to note that these data sets are not completely inaccessible to the user; they can be erased using the browser's Data Set Editor".


(Reference) URL-accessible mechanism for data sets

A data set (file system) name and a mount point have a unique association in TeraCLOUD, however there is no such relationship between a data set and a published URL.

The region that users can see through the web interface or WebDAV is usually accessible via the path "/dav/".

URL layer/dav/
URL path conversion rules (fixed)/dav/ → /dav/
Directory Path Layer/dav/
Mapper
/dav/ → /files/
/files
Mounting path rules (fixed)/files → files
Data Set Layerfiles

"File path: /files/" is URL mapped to "URL path: /dav". "File path: /files/" is where the "Data set: files" is mounted (*There is no files data set for users created before April 2015, and a directory named /files/ is generated in the root data set).

The above URL mapper is expected to be swapped in the forthcoming mapper API, allowing for a variety of uses, such as preserving the Clone at a specific point in time. However, because this feature has numerous side effects, such as authentication cache, it is currently being considered for implementation and release.

 
Create My Account (10GB Free)

Easy one-minute registration! Start backing up today!
Works online, Windows, Mac, iOS and Android