How to Secure Your SSH Server in 5 Steps (2026 Guide)

SSH (Secure Shell) is the lifeline of remote Linux server administration. Every Linux server exposed to the internet faces constant automated attacks against its SSH service — typically thousands of brute-force login attempts per day from botnets scanning the entire IPv4 address space. A single misconfigured SSH server can compromise your entire infrastructure within minutes of deployment.

The good news is that securing SSH is straightforward when you know what to do. These five steps, implemented in order, will transform your SSH configuration from a potential vulnerability into a hardened security gateway. Each step builds on the previous one, creating multiple layers of defense that make unauthorized access virtually impossible.

Step 1: Disable Root Login and Password Authentication

The single most impactful SSH security measure is eliminating the two most common attack vectors: direct root login and password-based authentication. This step alone eliminates over 99% of automated SSH attacks.

Why This Matters

Every automated SSH attack begins by trying to log in as root with common passwords like "password", "admin", "123456", or dictionary words. The root account is the most dangerous target because it has unlimited system access. By disabling root login via SSH, you eliminate 100% of these root-targeted attacks instantly — attackers cannot even attempt to authenticate.

Disabling password authentication entirely — requiring SSH keys instead — eliminates brute-force attacks completely. There is no password to guess. SSH keys use 256-bit or higher cryptographic strength, making brute-force attacks computationally infeasible even with the most powerful computers on Earth.

Prerequisites Before Making Changes

Before changing these settings, you must have:

1. A non-root user account with sudo access already created on the server
2. An SSH key pair generated on your local machine
3. The public key already deployed to the server's ~/.ssh/authorized_keys file
4. Verified that you can log in with the key (test from a second terminal)

How to Implement

Edit /etc/ssh/sshd_config with your preferred editor (e.g., sudo nano /etc/ssh/sshd_config):

• PermitRootLogin no — Prevents any SSH login as root, regardless of authentication method
• PasswordAuthentication no — Requires SSH key authentication for all users. No passwords accepted
• PubkeyAuthentication yes — Explicitly enables public key authentication (usually default, but be explicit)
• PermitEmptyPasswords no — Blocks accounts with no password set from logging in
• ChallengeResponseAuthentication no — Disables keyboard-interactive authentication (another password vector)

Always test the configuration syntax before restarting: sudo sshd -t. If this command produces no output, the configuration is valid. Then restart SSH: sudo systemctl restart sshd.

Critical Warning: Always test SSH changes from a second terminal session. Open a new terminal, try to connect with your key. Only close your original session after confirming the new session works. If you lock yourself out by closing your only session, you will need physical or console access (VNC/KVM from your hosting provider) to recover.

Step 2: Change the Default SSH Port

While changing the SSH port is not a security measure in the strictest sense (security through obscurity is not real security), it provides enormous practical benefits. Most automated attack bots only scan port 22, so moving to a non-standard port eliminates 95%+ of automated attacks from your logs, making your log files cleaner and genuine threats easier to spot.

How to Implement

1. Choose a port above 1024 and below 65535 that is not used by another service (e.g., 2222, 2200, 22222, or any random unused port)
2. Add to /etc/ssh/sshd_config: Port 2222
3. Update your firewall BEFORE restarting SSH — this is critical:
   • For firewalld: sudo firewall-cmd --permanent --add-port=2222/tcp && sudo firewall-cmd --reload
   • For UFW: sudo ufw allow 2222/tcp
   • For iptables: sudo iptables -A INPUT -p tcp --dport 2222 -j ACCEPT
4. Restart SSH: sudo systemctl restart sshd
5. Test from a new terminal: ssh -p 2222 user@server
6. Only after confirming the new port works, remove the old SSH port from the firewall

Add the port to your local SSH config file (~/.ssh/config) for convenience so you do not have to type -p 2222 every time:

Now you can simply type ssh myserver to connect.

Step 3: Use Ed25519 Keys with Passphrase Protection

Not all SSH keys are created equal. Ed25519 keys are the modern standard recommended by security professionals — they are more secure, faster at signing and verification, and produce shorter key strings than older RSA keys. If you are still generating RSA keys, it is time to upgrade.

How to Generate and Deploy Ed25519 Keys

1. Generate an Ed25519 key pair on your local machine: ssh-keygen -t ed25519 -C "admin@myserver-2026"
2. Set a strong passphrase when prompted — this protects your private key if your laptop is stolen or compromised. Without a passphrase, anyone who accesses your private key file can log in to all your servers
3. Deploy the public key to the server: ssh-copy-id -i ~/.ssh/id_ed25519.pub -p 2222 user@server
4. Set correct permissions on the server (these permissions are required — SSH refuses to work with incorrect permissions):
   • chmod 700 ~/.ssh
   • chmod 600 ~/.ssh/authorized_keys
5. Use ssh-agent to cache your passphrase so you do not have to type it for every connection: eval $(ssh-agent) && ssh-add ~/.ssh/id_ed25519

Why Ed25519 Over RSA

Key Management Best Practices

• Use separate key pairs for different servers or roles (admin key, deploy key, CI/CD key)
• Store private keys only on trusted devices — never email or copy them to untrusted machines
• Regularly audit authorized_keys files: remove keys for departed employees or decommissioned systems
• Set key expiry where possible — rotate keys annually at minimum

Step 4: Implement Connection Limits and Automated Blocking

Even with key-based authentication, it is important to limit connection behavior to prevent resource exhaustion, detect anomalous access patterns, and automatically block persistent attackers.

SSH Connection Limits

Add these settings to /etc/ssh/sshd_config:

• MaxAuthTries 3 — Disconnect after 3 failed authentication attempts per connection. This limits how many guesses an attacker gets
• MaxSessions 2 — Limit multiplexed sessions per single SSH connection
• ClientAliveInterval 300 — Send a keepalive packet every 5 minutes to detect dead connections
• ClientAliveCountMax 2 — Disconnect after 2 missed keepalives (10 minutes total idle time)
• LoginGraceTime 30 — Only allow 30 seconds to complete authentication. This prevents attackers from holding connections open indefinitely
• MaxStartups 3:50:10 — Rate limit unauthenticated connections: allow 3 concurrent, start randomly dropping 50% of new ones, hard limit at 10

Install and Configure Fail2ban

Fail2ban is an essential tool that monitors SSH log files and automatically bans IP addresses showing suspicious behavior. It creates firewall rules on-the-fly to block attackers.

1. Install Fail2ban:
   • Debian/Ubuntu: sudo apt install fail2ban
   • RHEL/AlmaLinux/Rocky: sudo dnf install fail2ban
2. Enable and start the service: sudo systemctl enable --now fail2ban
3. Create a local configuration (never edit the default file): sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
4. Edit /etc/fail2ban/jail.local with these recommended settings for the SSH jail:
   • [sshd] section: enabled = true
   • port = 2222 (match your SSH port)
   • bantime = 86400 (24-hour ban — sufficient to deter most attackers)
   • maxretry = 3 (ban after 3 failed attempts)
   • findtime = 600 (within a 10-minute window)
5. Restart fail2ban: sudo systemctl restart fail2ban
6. Check status: sudo fail2ban-client status sshd

Monitoring Fail2ban Activity

Step 5: Restrict SSH Access by User and IP Address

The final layer of defense is restricting WHO can connect and from WHERE. This is the difference between a good SSH configuration and an excellent one.

User Whitelisting

Add to /etc/ssh/sshd_config:

• AllowUsers admin deploy — Only these specific usernames can log in via SSH. All other users are blocked even if they have valid SSH keys
• For group-based access (more maintainable for larger teams): AllowGroups ssh-users
• Add users to the group: sudo usermod -aG ssh-users admin

IP Whitelisting with Firewall

For maximum security, restrict SSH access to specific IP addresses or ranges:

• FirewallD (RHEL/AlmaLinux/Rocky): sudo firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="YOUR.OFFICE.IP/32" port protocol="tcp" port="2222" accept' --permanent
• UFW (Debian/Ubuntu): sudo ufw allow from YOUR.OFFICE.IP to any port 2222
• Then remove the general SSH port rule so only whitelisted IPs can connect

Using a VPN for SSH Access

For teams with dynamic IP addresses, consider placing SSH behind a VPN (WireGuard or OpenVPN). Team members connect to the VPN first, then SSH to servers through the VPN tunnel. This adds an extra authentication layer and eliminates the need to whitelist changing IPs.

Complete Hardened SSH Configuration Summary

Frequently Asked Questions

How do I check if my SSH server is secure?

Run sudo sshd -T to dump the complete active configuration. Verify that PermitRootLogin is "no", PasswordAuthentication is "no", and only modern ciphers are listed. For a comprehensive audit, install and run ssh-audit — it checks cipher strength, key exchange algorithms, and known vulnerabilities.

What should I do if I get locked out of SSH?

Use your hosting provider's console, VNC, or KVM access to connect directly. Most cloud providers (Hetzner, DigitalOcean, AWS) offer a web-based console. This is why you should always test SSH changes from a second session before closing the first one, and always keep console access credentials available.

Is changing the SSH port really necessary?

Not strictly necessary for security, but highly recommended for practical reasons. It eliminates 95%+ of automated bot attacks, keeps your auth.log clean, reduces Fail2ban load, and makes it easier to spot genuine targeted attacks in your logs.

Can I use both password and key authentication?

You can, but you absolutely should not. Allowing passwords alongside keys means attackers can still brute-force passwords. Every security professional recommends key-only authentication. The minor inconvenience of managing SSH keys is vastly outweighed by the security benefits.

How often should I rotate SSH keys?

Rotate SSH keys annually at minimum, immediately when an employee leaves or a device is lost/compromised, and whenever you suspect unauthorized access. Document your key rotation schedule as part of your security policy.

Related Resources

• SSH Mastery: Secure Remote Administration — Complete SSH eBook
• OpenSSH Configuration & Tunneling Guide — Advanced SSH techniques
• The Ultimate Linux Server Hardening Guide — Comprehensive server security
• Browse all 205+ free IT cheat sheets