As an infrastructure engineer on the cloud platform development team, I have worked with many distributed storage systems, including those mentioned in the title. It seems that there is an understanding of their strengths and weaknesses, and I will try to share with you my thoughts on this matter. So to speak, let's see who has the longer hash function.
Disclaimer: Earlier in this blog, you may have seen articles about GlusterFS. I have nothing to do with these articles. This is the author's blog of the project team of our cloud and each of its members can tell their story. The author of those articles is an engineer of our operation group and he has his own tasks and his own experience, which he shared. Please take this into account if you suddenly see a difference of opinion. I would like to take this opportunity to convey my best regards to the author of those articles!
Let's talk about the file systems that you can build with GlusterFS and CephFS. Let's discuss the architecture of these two systems, look at them from different angles, and at the end I even venture to draw some conclusions. Other Ceph features such as RBD and RGW will not be affected.
To make the article complete and understandable for everyone, let's understand the basic terminology of both systems:
RADOS (Reliable Autonomic Distributed Object Store) is a self-contained object storage that is the backbone of the Ceph project. CephFS, RBD (RADOS Block Device), RGW (RADOS Gateway) are high-level RADOS add-ons that provide end users with various interfaces to RADOS. Specifically, CephFS provides a POSIX-compatible file system interface. In fact, CephFS data is stored in RADOS. OSD (Object Storage Daemon) is a process that maintains a separate disk / object storage in a RADOS cluster. RADOS Pool - several OSDs, united by a common set of rules, such as, for example, a replication policy. From a data hierarchy perspective, a pool is a directory or a separate (flat, no subdirectories) namespace for objects. PG (Placement Group) - I will introduce the concept of PG a little later, in context, for better understanding.
Since RADOS is the foundation on which CephFS is built, I will often talk about it and this will automatically apply to CephFS.
GlusterFS Terminology (hereinafter gl):
Brick is a process that serves a separate disk, analogous to OSD in RADOS terminology. Volume - the volume into which the bricks are combined. A volume is an analogue of a pool in RADOS, it also has a specific replication topology between bricks.
To make it clearer, let's look at a simple example that can be implemented by both systems.
An example setup:
Liquid wallpaper is a fairly popular type of decorative and finishing material. One of its main advantages is high maintainability.
Liquid wallpaper can be restored in several ways, depending on the degree of damage or pollution. This article will talk about ways to update old liquid wallpaper and about more radical methods for restoring damaged areas.
The need to restore or replace a certain area of liquid wallpaper arises quite often. As a rule, these are the places near which various household items are most intensively exploited:
Even with the most careful use and periodic cleaning, such areas lose their original appearance and significantly reduce the quality of interior design.
Careful preparation of the wall section for restoration is a guarantee of high-quality performance of work and durability of finishing the restored coating.
If you carefully seal the bag with dry mix, then its shelf life will be almost unlimited.
To restore a local area of liquid wallpaper, it is necessary to completely remove the old coating from the surface of the wall or ceiling.
Many craftsmen try to use the old coating by mixing it with water again and adding wallpaper glue. This method of local updating requires great care and careful adherence to the technological process. It is very difficult to separate the liquid wallpaper from the wall without capturing some of the plaster. Therefore, the plasticity of the repair mixture will differ significantly.
How to prepare the restoration site