This document outlines the various configuration items to keep in mind when planning a LeoFS system's cluster, and this documentation leads you to be able to configure its cluster when planning and launching it correctly.
To keep the consistency of objects eventually, LeoFS delivers the replication and recovery feature to automatically fix consistency of objects. You can configure the consistency level of a LeoFS system, and it is affected by the configuration.
How to Keep RING's Consistency¶
Case 1: Both LeoManager nodes are unavailable¶
If both LeoManager nodes are unavailable, LeoStorage and LeoGateway nodes don't update the RING to keep its consistency into the LeoFS system.
Case 2: One LeoManager node is unavailable¶
If a LeoManager node is unavailable, LeoFS can update the RING, and synchronize it with the LeoFS' system eventually. After restarting another LeoManager node, LeoManager automatically synchronizes the RING between the manager nodes.
Configure the consistency level of a LeoFS system at LeoManager's configuration file - leo_manager_0.conf. You need to carefully configure the consistency level because it is not able to change some items after starting the system.
There are four configuration items at
leo_manager_0.conf, items of which have a great impact on data availability and storage performance.
||n||No||1||A number of replicas|
||w||Yes||1||A number of replicas needed for a successful WRITE operation|
||r||Yes||1||A number of replicas needed for a successful READ operation|
||d||Yes||1||A number of replicas needed for a successful DELETE operation|
||No||0||A number of rack-aware replicas|
Data Availability of Consistency Level¶
This document delivers the relationship of
data availability and
configuration level as below:
|Data Availability||Configuration Level||Description|
|Extremely Low||n=2, r=1
|Data can not be acquired even if two nodes goes down (for personal use)|
|Low data consistency|
|High data consistency than
|Data can not be input and removed even if one node goes down|
|Extremely High||n=3, r=3
|Data can not be acquired even if one node goes down (can not be recommended)|
Warning: Rebalance with Extremely High Settings
While rebalance is on-going, PUT/DELETE use an new RING and GET/HEAD use an old RING. That said, if any updates happen on existing objects then there can be inconsistent objects from the old RING perspective however thanks to the inherent nature of the consistent hashing, almost replicas keep staying at the same position so it should not be problem while operating LeoFS in a typical(Other than
Extremely High) consistency level. But if you operate LeoFS in
Extremely High then GET/HEAD to the existing object may fail during the rebalance process. If that is the case then you may be able to use
update-consistency-level to lower the consistency level temporarily. Don't forget to set it back to the original setting once the rebalance finishes.
How To Change Consistency Level¶
You can change
consistency.delete of the consistency level that you use the
leofs-adm update-consistency-level command, but you cannot update
## Changes the consistency level to [w:2, d:2, r:1] $ leofs-adm update-consistency-level 2 2 1