The Linux audit framework is the closest thing the kernel provides to a flight recorder. Every system call, every file access, every privileged action can be logged with caller, target, and exit code. Auditd is required by PCI, HIPAA, FedRAMP, and most internal compliance frameworks โ and is invaluable during incident response. This guide covers the rule patterns that catch real attacks without flooding disk, plus how to actually search what you collect.
Architecture in two minutes
The kernel emits audit events; the auditd daemon reads them via netlink and writes to /var/log/audit/audit.log. Two query tools sit on top: ausearch (filter and pretty-print) and aureport (summarize).
sudo systemctl status auditd
sudo auditctl -s # daemon and kernel state
sudo auditctl -l # currently active rules
Where rules live
Three layers, loaded in order:
- Kernel built-in (none by default).
- Files under
/etc/audit/rules.d/*.rules, compiled into/etc/audit/audit.rulesbyaugenrules --load. - Runtime-only rules added with
auditctl -a(lost on restart).
Always edit rules.d/ drop-ins. Files load alphabetically; use 10-base.rules, 20-files.rules, 99-finalize.rules.
The minimum useful ruleset
# /etc/audit/rules.d/30-monitoring.rules
# Watch credential files
-w /etc/passwd -p wa -k identity
-w /etc/shadow -p wa -k identity
-w /etc/group -p wa -k identity
-w /etc/gshadow -p wa -k identity
-w /etc/sudoers -p wa -k privilege
-w /etc/sudoers.d/ -p wa -k privilege
# SSH config and keys
-w /etc/ssh/sshd_config -p wa -k sshd
-w /root/.ssh -p wa -k ssh-keys
# Time changes
-a always,exit -F arch=b64 -S adjtimex,settimeofday,clock_settime -k time-change
-w /etc/localtime -p wa -k time-change
# Module load/unload
-a always,exit -F arch=b64 -S init_module,delete_module -k modules
# Successful sudo invocations
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/sudo -F success=1 -k sudo
# Critical binaries integrity
-w /usr/bin/passwd -p x -k passwd
-w /usr/bin/su -p x -k su
Apply: sudo augenrules --load then sudo auditctl -l to confirm.
Searching events with ausearch
sudo ausearch -k identity -ts today
sudo ausearch -k sudo -ts recent | aureport -i
sudo ausearch -ui 1000 -ts yesterday # by UID
sudo ausearch -m USER_LOGIN --start week-ago --end now
sudo ausearch -k modules -i | less # interpret numeric IDs
The -i flag is critical for human reading โ it converts numeric UIDs, syscall numbers, and uid/gid values into names.
Summarizing with aureport
sudo aureport --summary -i
sudo aureport --auth -i # login report
sudo aureport --executable -i # which binaries ran
sudo aureport --file --summary -i # most-touched files
sudo aureport -m -i # event types
sudo aureport --failed -i # only failures
Run aureport daily into a report file and ship to long-term storage. The output gives you a one-page baseline that makes anomalies obvious.
Filtering by user, syscall, file
The selector syntax of audit rules is concise but unforgiving:
# Only catch deletion of files in /var/www by non-root
-a always,exit -F arch=b64 -S unlink,unlinkat,rmdir -F dir=/var/www -F auid>=1000 -k www-delete
# Catch any execution of nc/ncat/socat (typical post-exploit tools)
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/nc -k recon
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/ncat -k recon
-a always,exit -F arch=b64 -S execve -F path=/usr/bin/socat -k recon
The auid (audit UID) is the original user who logged in, preserved across su and sudo โ much more useful than effective UID for attribution.
Tuning for performance
A noisy ruleset can saturate disk. Three knobs:
- Backlog:
auditctl -b 8192raises the kernel buffer; pair with--backlog_wait_time. - Rate:
auditctl -r 500caps events per second. - Disk safety: in
/etc/audit/auditd.confsetmax_log_file_action = ROTATE,num_logs = 8,space_left_action = SYSLOG,admin_space_left_action = SUSPEND.
Shipping logs to a SIEM
Local logs are easily tampered with by an attacker who already has root. Forward in real time:
# /etc/audisp/plugins.d/syslog.conf
active = yes
direction = out
path = builtin_syslog
type = builtin
args = LOG_INFO
format = string
Combined with rsyslog/syslog-ng forwarding to a remote write-only collector, the attacker no longer controls the only copy of their footprints.
Common pitfalls
- Adding rules with
auditctl -afor testing, then forgetting to persist them inrules.d/. - Watching
/var/log/audit/audit.logwithtail -fand concluding "auditd does not work" โ the file is only readable by root. - Logging every
open()syscall on a busy server. The disk will fill in hours. - Ignoring
aureport --failedโ failed audit events often reveal an attacker's reconnaissance.
Auditd has a steep learning curve and a steeper compliance payoff. Start with the 30-line ruleset above, ship logs off-host, run aureport daily, and you cover most of the audit requirements that auditors actually verify โ while gaining the ability to answer "what happened on that server at 03:14?" with a one-line query.