LeoFS Storage

LeoFS Storage configuration

  • Location of configuration file:
    • <leofs-home-dir>/package/storage/etc/leo_storage.conf OR /usr/local/leofs/<version>/leo_storage/etc/leo_storage.conf


You cannot modify the following Leo Storage configurations after launched a system(LeoFS) because the algorithm of retrieving object depends on their configurations, that means Leo Storage cannot handle objects.

  • NOT able to change items:
    • obj_containers.path
    • obj_containers.num_of_containers
    • obj_containers.metadata_storage
    • num_of_vnodes
Property Default Value Description
LeoFS Manager
managers [manager_0@, manager_1@] Node name of LeoFS Manager master and slave nodes
LeoFS Storage Basic Configuration
obj_containers.path [./avs] Not able to change items Directories of each object-containers
obj_containers.num_of_containers [8] Not able to change items A number of object-containers of each directory
obj_containers.metadata_storage leveldb Not able to change items Metadata Storage - [leveldb, bitcask], We recommend leveldb
num_of_vnodes 168 Not able to change items A number of virtual-nodes, which depends on the redundant-manager
object_storage.is_strict_check false Able to strictly check between checksum of a metadata and checksum of the object
watchdog.rex.interval 10 Watch interval for the rex*(rpc)* process in order to execute force GC when reaching the threshold value
watchdog.rex.threshold_mem_capacity 33554432 Threshold memory capacity of binary for rex*(rpc)*
watchdog.cpu.is_enabled false Enabled or disabled CPU-watchdog [true|false]
watchdog.cpu.interval 10 Watch interval(sec) for CPU-watchdog
watchdog.cpu.raised_error_times 5 An error is raised to subscribers when a number of errors reached this configuration.
watchdog.cpu.threshold_cpu_load_avg 5.0 Threshold CPU load avg for 1min/5min
watchdog.cpu.threshold_cpu_util 100 Threshold CPU load util (%)
watchdog.disk.is_enabled false Enabled or disabled disk-watchdog [true|false]
watchdog.disk.interval 10 Watch interval (sec) for Disk-watchdog
watchdog.disk.raised_error_times 5 An error is subscribers to clients when a number of errors reached this configuration.
watchdog.disk.threshold_disk_use 85 Threshold disk usage(capacity) (%) - leo_watchdog is using df command
watchdog.disk.threshold_disk_util 100 Threshold disk util (%) - leo_watchdog is using iostat command
watchdog.disk.threshold_disk_rkb 98304 Threshold disk read KB/sec
watchdog.disk.threshold_disk_wkb 98304 Threshold disk write KB/sec
watchdog.disk.target_devices [] Target devices for checking disk utilization
Autonomic Operation
autonomic_op.compaction.is_enabled false
  • Is enabled auto-compaction? [true|false]
autonomic_op.compaction.parallel_procs 1
  • Number of parallel procs of data-compaction
autonomic_op.compaction.threshold_active_size_ratio 70
  • Warning ratio of active size (%)
  • NOT affects the auto-compaction by this configuration yet. We plan to support this configuration with v1.4.
autonomic_op.compaction.threshold_active_size_ratio 60
  • Threshold ratio of active size (%)
  • (Size of active objects / Size of total objects) * 100 (%)
  • When it was less than the ratio, LeoFS Storage executes data-compaction automatically
Data Compaction
compaction.limit_num_of_compaction_procs 4 Limit of a number of procs to execute data-compaction in parallel
Data Compaction - Interval between batch processings
compaction.waiting_time_regular 500 Regular value of compaction-proc waiting time/batch-proc(msec)
compaction.waiting_time_max 3000 Maximum value of compaction-proc waiting time/batch-proc(msec)
Data Compaction - Number of objects a batch processing
compaction.batch_procs_regular 1000 Regular number of objects a batch processing
compaction.batch_procs_max 5000 Maximum number of objects a batch processing
mq.backend_db = bitcask bitcask MQ backend storage: [bitcask]
mq.num_of_mq_procs 8 A number of mq-server’s processes
MQ - Number of messages a batch processing
mq.num_of_batch_process_max 3000 Maximum number of messages a bach processing
mq.num_of_batch_process_regular 1600 Regular number of messages a bach processing
MQ - Interval between batch processings
mq.interval_between_batch_procs_max 3000 Maximum value of interval beween batch-procs(msec)
mq.interval_between_batch_procs_regular 500 Regular value of interval between batch-procs(msec)
STORAGE - Replication/Recovery object(s)
replication.rack_awareness.rack_id [] Rack-id for the rack-awareness replica placement
replication.recovery.size_of_stacked_objs 33554432 Size of stacked objects (bytes)
replication.recovery.stacking_timeout 5 Stacking timeout (sec)
STORAGE - MDC Replication
mdc_replication.size_of_stacked_objs 33554432 Size of stacked objects (bytes)
mdc_replication.stacking_timeout 30 Stacking timeout (sec)
mdc_replication.req_timeout 30000 Request timeout (msec)
mdc_replication.stacking_procs 1 Number of stacking procecces
RPC for MDC-replication
rpc.server.acceptors 128 Number of RPC-server’s acceptors
rpc.server.listen_port 13077 RPC-Server’s listening port number
rpc.server.listen_timeout 30000 RPC-Server’s listening timeout (msec)
rpc.client.connection_pool_size 8 RPC-Client’s size of connection pool
rpc.client.connection_buffer_size 8 RPC-Client’s size of connection buffer
log.log_level 1 Log level: [0:debug, 1:info, 2:warn, 3:error]
log.erlang ./log/erlang Locatio of Erlang log-files
log.app ./log/app Locatio of Gateway’s log-files
log.member_dir ./log/ring Location of dump files of members of a storage cluster
log.ring_dir ./log/ring Location of RING’s dump files
queue_dir ./work/queue Directory of queue for monitoring “RING” and members of a storage cluster
snmp_agent ./snmp/snmpa_gateway_0/LEO-GATEWAY Directory of SNMP agent configuration

Erlang VM configuration

Property Default Value Description
nodename storage_0@ Node name of LeoFS Storage
distributed_cookie 401321b4 Cookie for distributed node communication
erlang.kernel_poll true Enable or disable kernel poll
erlang.async_threads 32 Number of async threads
erlang.max_ports 64000 Number of concurrent ports/sockets
erlang.crash_dump ./log/erl_crash.dump Location of crash dumps
erlang.max_ets_tables 256000 ETS table limit
erlang.smp enable Enable or disable SMP
erlang.schedulers.compaction_of_load true Erlang scheduler’s compaction of load
erlang.schedulers.utilization_balancing false Erlang scheduler’s balancing of load
erlang.distribution_buffer_size 32768 Sender-side network distribution buffer size (KB)
erlang.fullsweep_after 0 A non-negative integer which indicates how many times generational garbage collections can be done without forcing a fullsweep collection
erlang.secio false Enable or disable eager check I/O (Erlang 17.4/erts-6.3-, ref:OTP-12117)
process_limit 1048576 Default erlang process limit
snmp_conf ./snmp/snmpa_storage_0/leo_storage_snmp SNMPA configuration files directory

See Also

Configuration of eager check I/O scheduling for Erlang’s VM

If you adopt LeoFS v1.2.7 later w/Erlang 17.5, we recommend you turn on erlang.secio. So you need to modify the configuration files and the schema files as follows.

  • Turn on erlang.secio at leo_storage.conf
erlang.secio = true
  • Uncomment every rows of erlang.secio at leo_storage.schema
    {default, false}

See Also