"Who installed that package and when?" is the first question after a security scan flags an unexpected binary, and the second question after a deploy script behaves differently from yesterday. APT keeps a detailed history that answers both β but the logs rotate fast, the format is human-friendly rather than machine-friendly, and almost nobody reviews them until the moment they urgently need to. This guide walks through the APT log files, the queries that surface real changes, and the auditing routine that catches drift before it bites.
The two log files APT writes
Every apt, apt-get, aptitude, and unattended-upgrades invocation produces two files:
/var/log/apt/history.logβ high-level transactions: command line, start/end timestamps, requested-by user, packages installed/upgraded/removed./var/log/apt/term.logβ full terminal output of the dpkg invocation, including post-install scripts and any errors.
Both are rotated weekly with seven keep-around copies on default Debian/Ubuntu β that gives roughly seven weeks of history. For compliance, extend retention in /etc/logrotate.d/apt: change rotate 12 and compress, and you get a year on disk for under 50 MB.
Reading the history
sudo less /var/log/apt/history.log
sudo zcat /var/log/apt/history.log.*.gz | less # rotated history
sudo grep -B1 -A4 'openssh-server' /var/log/apt/history.log*
Each transaction block shows: Start-Date, Commandline, Requested-By (if not root from sudo), Install/Upgrade/Remove lists with versions, and End-Date. The Requested-By line tells you the human user even when the actual install runs as root β invaluable in incident review.
Cross-checking with dpkg
APT history covers package operations; dpkg knows the current state. Combine them:
dpkg-query -W -f='${Package}\t${Version}\t${Status}\n' | grep -v 'install ok installed'
dpkg --get-selections | grep -v deinstall > /var/lib/pkg-baseline.txt
diff /var/lib/pkg-baseline.txt <(dpkg --get-selections | grep -v deinstall)
The first line lists packages in unusual states (half-installed, config-only, removed-but-config-present). The diff against a baseline is your drift detector.
Finding what a package installed
dpkg -L openssh-server # files installed by package
dpkg -S /usr/sbin/sshd # which package owns this file?
apt-file search /usr/sbin/sshd # works even for uninstalled packages
apt show openssh-server # metadata, dependencies
For a forensic timeline of "what touched /etc/cron.d/": dpkg -L $(dpkg -S /etc/cron.d/ | cut -d: -f1 | sort -u).
Surfacing security updates
apt list --upgradable 2>/dev/null
apt-get -s upgrade | grep -i security
unattended-upgrade --dry-run --debug 2>&1 | grep 'Allowed origins'
zgrep '\(Install\|Upgrade\):.*security' /var/log/apt/history.log* | tail -50
The last line gives a chronological list of every security-flagged package change on the host β the single most useful audit query for compliance reviews.
Detecting unauthorised package changes
An attacker who installs a backdoored OpenSSH or replaces ls often does it via dpkg -i β that path bypasses APT and produces no history.log entry. Detect with file-integrity monitoring (AIDE, Tripwire, debsums):
sudo apt install debsums
sudo debsums -c # report changed package files
sudo debsums -ce # only config files
sudo debsums -s 2>&1 | grep -v 'OK$' | head
Any output that is not "OK" means a package file has been modified since installation β almost always either an attacker, a misbehaving config-management tool, or a sysadmin who forgot to use a drop-in.
Pinning and holding
To prevent a critical package from being upgraded without explicit approval:
sudo apt-mark hold postgresql-15 # never auto-upgrade
sudo apt-mark showhold # what is currently held
sudo apt-mark unhold postgresql-15
For finer-grained control use APT pinning in /etc/apt/preferences.d/:
Package: postgresql-*
Pin: version 15.*
Pin-Priority: 1001
Compliance-grade reporting
A 10-line script run weekly, output piped to your log shipper, gives auditors a reproducible record:
#!/bin/bash
echo "Host: $(hostname -f), Date: $(date -u +%FT%TZ)"
echo "== Last 7 days transactions =="
zgrep -h Start-Date /var/log/apt/history.log* | sort -u | tail -50
echo "== Currently held packages =="
apt-mark showhold
echo "== Files modified since install =="
debsums -s 2>&1 | grep -v 'OK$' | head -20
Common pitfalls
- Default rotation drops history after seven weeks β extend it before you need it.
- Trusting
history.logalone misses directdpkg -iinstalls and any tampering by an attacker with root. - Running
apt autoremoveblindly on production β pin critical kernel/headers explicitly. - Ignoring
debsumsoutput as "noisy" β every changed file deserves a one-line explanation.
APT history is one of the few free audit trails Linux gives you out of the box. Spend ten minutes extending retention, ship the logs off-host, and run a weekly debsums + history report β the next time someone asks "when did this server get that package?", the answer is one query away.