GRUB Bootloader: Validating Configuration, Kernel Parameters, and Boot Recovery

GRUB is the seam between firmware and your operating system, and the file that controls it is one mistype away from an unbootable server. Every kernel update, every boot-parameter tweak, every encrypted-disk migration touches GRUB. This guide covers the validation steps that prevent failed boots, the most useful kernel command-line parameters, and the recovery techniques you reach for at 03:00 when nothing comes back up.

The GRUB file you actually edit

Never edit /boot/grub/grub.cfg directly — it is regenerated. The source of truth is /etc/default/grub plus the scripts in /etc/grub.d/:

sudo $EDITOR /etc/default/grub
sudo grub-mkconfig -o /boot/grub/grub.cfg          # Debian/Ubuntu
sudo grub2-mkconfig -o /boot/grub2/grub.cfg        # RHEL/Fedora (BIOS)
sudo grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg  # RHEL/Fedora (UEFI)

After regeneration, diff the new file against the old to catch surprises before reboot:

sudo cp /boot/grub/grub.cfg /boot/grub/grub.cfg.prev
sudo grub-mkconfig -o /boot/grub/grub.cfg
sudo diff -u /boot/grub/grub.cfg.prev /boot/grub/grub.cfg | less

Kernel command-line parameters that matter

Edit GRUB_CMDLINE_LINUX (applied to every entry) or GRUB_CMDLINE_LINUX_DEFAULT (only the default). Useful additions for servers:

• quiet splash — clean console; remove on troubleshooting.
• console=tty0 console=ttyS0,115200 — mirror to serial console for KVM and IPMI access.
• nomodeset — fall back to basic graphics; useful when a GPU driver hangs early boot.
• panic=10 — auto-reboot 10 s after kernel panic; pair with watchdog.
• audit=1 audit_backlog_limit=8192 — enable kernel audit early.
• mitigations=auto,nosmt — turn on all CPU-vulnerability mitigations and disable SMT (sane default for shared hosts).
• transparent_hugepage=never — required by most database vendors.

Validating before reboot

A bad GRUB config does not surface until reboot. Catch it earlier:

sudo grub-script-check /boot/grub/grub.cfg && echo OK
sudo grub-mkconfig -o /tmp/grub.cfg.test
diff -u /boot/grub/grub.cfg /tmp/grub.cfg.test
ls -1 /boot/vmlinuz-*                # confirm the kernel files exist
sudo dracut --force                  # rebuild initramfs (RHEL family)
sudo update-initramfs -u -k all      # rebuild initramfs (Debian family)

If the script-check returns errors, do not reboot — fix the syntax first.

Booting into recovery

If a deploy renders the system unbootable, the recovery flow has three layers:

1. GRUB menu. At the menu, press e on an entry to edit. Change ro to rw, append single or init=/bin/bash, then Ctrl-X to boot. You land in single-user mode with a writable root.
2. Previous kernel. The "Advanced options" submenu lists older kernels. Boot the last known-good one to get back online while you investigate.
3. Rescue media. Boot the distribution ISO, mount the system, and chroot:

   mount /dev/sda2 /mnt
   mount --bind /dev  /mnt/dev
   mount --bind /proc /mnt/proc
   mount --bind /sys  /mnt/sys
   mount /dev/sda1 /mnt/boot               # if /boot is separate
   chroot /mnt
   grub-install /dev/sda
   grub-mkconfig -o /boot/grub/grub.cfg
   exit
   reboot

UEFI versus BIOS

UEFI systems boot from an EFI System Partition mounted at /boot/efi. Reinstall the bootloader with:

sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=ubuntu
sudo efibootmgr -v               # inspect boot order
sudo efibootmgr -o 0001,0000     # reorder if firmware boots wrong entry first

On hosts with Secure Boot, only signed kernels load. After installing a custom kernel module (e.g. via DKMS), enroll your MOK key with mokutil --import and reboot to complete enrollment.

Password-protecting the menu

Anyone with console access can press e and append init=/bin/bash to root the box. Lock the editor:

sudo grub-mkpasswd-pbkdf2
# Add to /etc/grub.d/40_custom:
set superusers="admin"
password_pbkdf2 admin grub.pbkdf2.sha512.10000.…
sudo update-grub

Pair with a BIOS/UEFI admin password to defend the keyboard fully.

Common pitfalls

• Editing grub.cfg directly — works once, lost on next kernel update.
• Forgetting to rebuild initramfs after changing root device or LUKS settings.
• Setting GRUB_TIMEOUT=0 on remote servers — no chance to recover via the menu.
• Removing the previous kernel package immediately; keep at least two installed.

Treat GRUB the way you treat your firewall: rare changes, but every change reviewed, validated, and revertible. The minutes you spend running grub-script-check before reboot are the hours you do not spend in a rescue console afterwards.