A periodic sysctl audit catches the slow drift that turns a hardened server into an unhardened one. Over months, distribution upgrades reset values, manual changes via sysctl -w are forgotten on reboot, third-party packages drop their own files into /etc/sysctl.d/, and the kernel command line introduces new defaults. This guide walks through a baseline-and-diff workflow, the parameters with the highest security and performance impact, and the automation that keeps the audit current.
Why an audit, not a one-time tune
Three sources of drift, all preventable:
- Distribution upgrades โ
/etc/sysctl.d/may receive new files, and/etc/sysctl.confmay be replaced wholesale during major version bumps. - Application packages โ Docker drops
/etc/sysctl.d/99-docker.conf, Kubernetes drops10-kubernetes.conf, and so on. Each can override your hardening drop-in. - Manual interventions โ engineers running
sysctl -wfor testing and forgetting to make it persistent.
Capture a baseline
Before changing anything, snapshot the current kernel state:
sudo sysctl -a 2>/dev/null | sort > /var/lib/sysctl-baseline.txt
sudo cp /var/lib/sysctl-baseline.txt /var/lib/sysctl-$(date +%F).txt
Commit the baseline to your configuration management repo (Ansible, Salt, Puppet) so every server type has a reviewed, version-controlled file.
The 25 parameters that matter most
Drop into /etc/sysctl.d/99-hardening.conf:
# Network hardening
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
# IPv6
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_source_route = 0
# Kernel exploit mitigation
kernel.kptr_restrict = 2
kernel.dmesg_restrict = 1
kernel.unprivileged_bpf_disabled = 1
net.core.bpf_jit_harden = 2
kernel.yama.ptrace_scope = 1
fs.suid_dumpable = 0
kernel.randomize_va_space = 2
Apply with sudo sysctl --system. Verify with sudo sysctl -a 2>/dev/null | sort | diff - /var/lib/sysctl-baseline.txt โ every difference should map to a change you made deliberately.
Performance parameters worth tuning
Beyond hardening, certain values move performance significantly under load:
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.core.netdev_max_backlog = 5000
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc = fq
fs.file-max = 2097152
vm.swappiness = 10
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 1024 65535
The drift detector
A 20-line script run hourly catches drift:
#!/bin/bash
baseline=/var/lib/sysctl-baseline.txt
current=$(mktemp)
sudo sysctl -a 2>/dev/null | sort > "$current"
diff -u "$baseline" "$current" > /tmp/sysctl-drift.diff
if [ -s /tmp/sysctl-drift.diff ]; then
echo "Sysctl drift detected on $(hostname):" > /tmp/sysctl-alert.txt
cat /tmp/sysctl-drift.diff >> /tmp/sysctl-alert.txt
mail -s 'sysctl drift' ops@example.com < /tmp/sysctl-alert.txt
fi
rm -f "$current"
Schedule via systemd timer. Every change appears in your inbox within an hour, with full context.
Per-namespace and per-container tuning
Containers share the host kernel; almost every sysctl is host-wide. Two exceptions matter:
- Network sysctls in a container's network namespace can be tuned per container via
--sysctl 'net.ipv4.ip_local_port_range=10000 65535'. - Kernel "namespaced" sysctls are listed under
/proc/sys/net/per netns; non-namespaced parameters under/proc/sys/kernel/can only be set on the host.
Document which sysctls live where so an inexperienced engineer does not try to tune host parameters from inside a pod and conclude "sysctl is broken."
Compliance scoring
Run an automated CIS or STIG benchmark against the host:
sudo dnf install scap-security-guide openscap-scanner
oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_cis \
--results /tmp/cis.xml --report /tmp/cis.html \
/usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml
The HTML report includes per-parameter pass/fail with rationale and remediation snippets. Use as the starting point for your hardening file rather than as a one-shot checkbox exercise.
Rolling back a bad change
Every drop-in file is independent. To revert one set of changes, delete or rename the file and re-apply:
sudo mv /etc/sysctl.d/99-experiment.conf /etc/sysctl.d/99-experiment.conf.bak
sudo sysctl --system
sudo sysctl -a 2>/dev/null | sort | diff - /var/lib/sysctl-baseline.txt
For runtime-only changes that did not survive reboot, the diff against the baseline tells you exactly what was applied; revert by setting back to the baseline values.
Common pitfalls
- Editing
/etc/sysctl.confwhile a drop-in file insysctl.d/overrides it โ your edit appears to have no effect. - Setting
net.ipv4.ip_forward=1for one VPN host and forgetting to scope it; suddenly your web servers route packets too. - Tuning TCP buffers without knowing the bandwidth-delay product; the wrong values reduce throughput.
- Ignoring
kernel.printk_ratelimiton a noisy kernel โ drives rsyslog into dropping messages.
A monthly sysctl audit takes 15 minutes, fits into any configuration-management workflow, and keeps your kernel hardening from quietly degrading. Baseline, diff, alert, and own every parameter.