Ceph入门到精通-sysctl.conf 配置

sysctl.conf

Ubuntu server out of box is not optimized to make full use of available hardware. This means “out-of-box” setup might fail under high load.

So we need to tweak system configuration for maximum concurrancy.

Sysctl Tweaks

Open

vim /etc/sysctl.conf

Add following towards bottom

### IMPROVE SYSTEM MEMORY MANAGEMENT ###

# Increase size of file handles and inode cache
fs.file-max = 2097152

# Do less swapping
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2

### GENERAL NETWORK SECURITY OPTIONS ###

# Number of times SYNACKs for passive TCP connection.
net.ipv4.tcp_synack_retries = 2

# Allowed local port range
net.ipv4.ip_local_port_range = 2000 65535

# Protect Against TCP Time-Wait
net.ipv4.tcp_rfc1337 = 1

# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 15

# Decrease the time default value for connections to keep alive
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15

### TUNING NETWORK PERFORMANCE ###

# Default Socket Receive Buffer
net.core.rmem_default = 31457280

# Maximum Socket Receive Buffer
net.core.rmem_max = 12582912

# Default Socket Send Buffer
net.core.wmem_default = 31457280

# Maximum Socket Send Buffer
net.core.wmem_max = 12582912

# Increase number of incoming connections
net.core.somaxconn = 4096

# Increase number of incoming connections backlog
net.core.netdev_max_backlog = 65536

# Increase the maximum amount of option memory buffers
net.core.optmem_max = 25165824

# Increase the maximum total buffer-space allocatable
# This is measured in units of pages (4096 bytes)
net.ipv4.tcp_mem = 65536 131072 262144
net.ipv4.udp_mem = 65536 131072 262144

# Increase the read-buffer space allocatable
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384

# Increase the write-buffer-space allocatable
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384

# Increase the tcp-time-wait buckets pool size to prevent simple DOS attacks
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1

Load Changes

Run following command to load changes to sysctl.

sysctl -p

Useful Systcl Commands

This section is added to main post after Ovidiu’s comment.

Show all system parameters with their values (default or changed)

sysctl -A

Show values of parameters modified by you

sysctl -p

Show value for a single parameter  parameter-name

sysctl parameter-name

Change value for  a single parameter parameter-name without editing sysctl.conf manually.

sysctl -w parameter-name=parameter-value

Above command will overwrite any previous modifications to parameter-name. Also, you may need to surround parameter-value with quotes.

Related

I do not have in-depth explanation for all parameters. Comments will guide you somewhat.

You can check Increase "Open Files Limit". Solution for "Too many open files (24)" for more details about fs.file-max

Credits