Environment variables are the silent attack surface of every Linux server. Every running process exposes its environment in /proc/PID/environ; every shell rc file is read by every interactive login; every misplaced .env ends up in a git repo eventually. This guide covers the audit techniques that catch credential leakage before an attacker does, plus the PATH-hijacking patterns that turn a benign symlink into root.
Where credentials actually leak
Three categories cover almost every real-world incident:
- Process environment exposure.
ps ewwas the same UID (or as root) prints every variable, includingDATABASE_URL=postgres://user:pass@β¦. - Filesystem leakage.
.env,.envrc,.bash_history, world-readable systemd drop-ins under/etc/systemd/system/*.service.d/. - Process substitution and logging. A script that runs
curl -u "$USER:$PASS"writes the password to syslog when the kernel logs the audit event for execve().
Audit your own server in 60 seconds
sudo ls -lZ /proc/*/environ 2>/dev/null | awk '$1 !~ /^-r--------/ {print}'
sudo find / -xdev -name '.env*' -not -path '*/node_modules/*' 2>/dev/null
sudo grep -rIE '(PASSWORD|SECRET|TOKEN|API_KEY)=' /etc /opt /srv 2>/dev/null
sudo find / -xdev -perm -o+r -name 'authorized_keys' 2>/dev/null
The first line catches kernel namespaces where /proc permissions are weakened. The second locates dotenv files anywhere on the system. The third grep is intentionally broad β review every hit; many will be examples or comments, but production secrets in /etc or /srv need to move into a secret store.
The PATH hijack pattern
If a privileged script calls a binary by its short name, the lookup is controlled by PATH. An attacker who can write to any earlier directory in PATH wins:
$ echo $PATH
.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
$ cat /tmp/script.sh
#!/bin/bash
ls /var/log # short name, looked up via PATH
$ # attacker:
$ cat > /tmp/ls <<'EOF'
#!/bin/bash
cp /etc/shadow /tmp/leak; /bin/ls "$@"
EOF
$ chmod +x /tmp/ls
The fix is two lines anywhere a script may be called by another user:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
export PATH
Better still, call binaries by absolute path inside scripts: /usr/bin/ls, never ls. Cron and systemd timers already set a sane PATH; preserve it.
Removing secrets from the environment
Secrets in env are convenient and dangerous. Replace them with one of:
- Systemd
LoadCredential=β secrets are mounted into$CREDENTIALS_DIRECTORYper-service, never into the global env. - Vault / AWS Secrets Manager / GCP Secret Manager β fetched at start, never written to disk in plaintext.
- EnvironmentFile= with strict perms β at minimum,
chmod 600andchownthe owning service user; add:protectedfor non-readability after start.
Locking down /proc
On multi-tenant or shared hosts, mount /proc with hidepid=2 so users only see their own processes:
proc /proc proc defaults,hidepid=2,gid=adm 0 0
Members of the adm group can still observe everything for monitoring; ordinary users cannot list /proc/<other>/environ at all.
Shell history hardening
Stop secrets from landing in ~/.bash_history by prepending a space to sensitive commands (with HISTCONTROL=ignorespace) and disabling history for the duration of credential entry:
export HISTCONTROL=ignorespace:ignoredups
unset HISTFILE # this shell only
read -rs MYTOKEN # silent prompt, never echoed
Continuous monitoring
One-shot audits are not enough. Add a weekly job that:
- Greps
/proc/*/environfor known secret prefixes (e.g.AKIAfor AWS keys,ghp_for GitHub tokens). - Compares the current PATH for every service unit against a baseline.
- Alerts when any new
.envfile appears outside expected directories.
Quick wins checklist
- Set
chmod 600on every.envfile and on the directories that contain them. - Add
.env,.envrc,secrets.ymlto a global gitignore enforced via pre-commit hook. - Disable
HISTFILEin shared deploy users. - Remove world-read on systemd drop-in directories.
- Migrate one secret per sprint from env to
LoadCredential=or a real secret store.
Environment hygiene is unglamorous and never appears on a roadmap, but every breach postmortem you have ever read mentions it. Spend an hour today running the audit commands above on your fleet and you will find at least one credential that should not be where it is.