SSH and Linux Server Hardening

Before you start

Who is this for?

For Linux learners who connect to servers and want a safer, repeatable remote-access setup.

Prerequisites

Linux Essentials
Command-line use
Authorized server or disposable VM

What you'll be able to do

Distinguish user and host keys
Install a user key
Verify a host identity
Harden access without lockout

Practical outcome

Establish and verify key-based access and explain the risks of agent forwarding.

Linux securityAvailable

Establish and verify remote access with host identity checks, user keys, and cautious server hardening.

Difficulty: Intermediate
Time: 80 min
Practice: Verify key-based access and a safe server configuration
Assessment: Knowledge check

SSH provides an encrypted connection between a client and a server. During setup, the server proves its identity with a host key; the user then authenticates with an account credential such as a password or public key. These are two different checks. SSH encryption protects the connection, but it does not make every remote command safe.

1. User keys and host keys

A user key pair has a private key kept by the client and a public key placed in the target account’s ~/.ssh/authorized_keys. The private key should never be copied to the server. A passphrase helps protect a private key stored on disk. A passphrase-protected public key proves possession of that key; it does not, by itself, add a second authentication factor.

The server has its own host keys, commonly under /etc/ssh/. The client records trusted server host keys in ~/.ssh/known_hosts. This allows the client to detect a changed host identity on a later connection. A changed key can be caused by a legitimate rebuild, DNS or routing to a different host, or an interception attempt; verify the new fingerprint through a trusted channel before accepting it.

2. Create and install a key

On a current OpenSSH client that supports Ed25519:

ssh-keygen -t ed25519 -a 64 -C "laptop-admin"

Choose a strong passphrase and keep the private key file private. Algorithms and options depend on client/server versions and organisational cryptographic policy. If an environment does not support the chosen key type, follow its approved compatibility guidance rather than weakening server policy blindly.

Install only the public key. On a Unix-like client, ssh-copy-id user@server.example is available on many systems, but is not shipped with every OpenSSH client. Alternatively, append the one-line public key to the target user’s ~/.ssh/authorized_keys through an already trusted access method. On the server, ownership and restrictive permissions matter:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

The SSH server may reject keys if ownership or directory permissions are unsafe. Windows OpenSSH uses Windows ACLs rather than Unix mode bits; apply the platform’s documented ACL guidance.

3. Verify the host and connect

On first connection, compare the host-key fingerprint shown by the client with a value obtained independently from a trusted administrator or console. An unauthenticated ssh-keyscan can collect a presented key, but cannot prove that it belongs to the intended server.

ssh -o StrictHostKeyChecking=ask user@server.example
ssh-keygen -F server.example
ssh-keygen -lf ~/.ssh/known_hosts

If OpenSSH warns that a known host key changed, do not work around the warning by blindly deleting the entry. Verify the server rebuild or key rotation first, then update the known-host record through the approved process.

4. Client configuration and ssh-agent

A client config in ~/.ssh/config avoids repeating connection details:

Host lab-server
    HostName server.example
    User learner
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes
    ForwardAgent no

Protect the file so other local users cannot casually read or alter it. ssh-agent keeps decrypted key material available to local SSH clients for a limited session; add a key with ssh-add ~/.ssh/id_ed25519 and inspect loaded identities with ssh-add -l. Agent behavior and service management differ between Linux, macOS, and Windows.

Agent forwarding is off by default in OpenSSH. When enabled, a process on the remote host that can access the forwarded agent socket may ask the agent to authenticate onward while the connection is active. The remote machine does not receive the private key bytes, but it can potentially use the agent as an authentication oracle. Avoid forwarding through hosts you do not administer or trust. Prefer a direct connection, a controlled ProxyJump path, or destination-constrained keys where client, agent, and server versions support them. See the OpenSSH client configuration manual and ssh-agent manual.

5. Harden a server without locking yourself out

Before changing SSH server settings, confirm console or out-of-band recovery access, identify the active service name, and keep an existing administrator session open. Back up the configuration. A typical hardening goal is to use named accounts, disable direct root login, prefer public-key authentication, and turn off password or keyboard-interactive authentication only after a second authorized key login has been tested.

An example policy fragment might include:

PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
AllowUsers adminuser

This is not a universal drop-in. Existing Include files, PAM, distribution defaults, account policy, and OpenSSH versions affect effective behavior. PasswordAuthentication no does not necessarily disable keyboard-interactive authentication; review KbdInteractiveAuthentication and PAM settings for the actual system. Restricting AllowUsers can block recovery accounts if the list is incomplete.

After a change, validate syntax with the installed server’s sshd -t command, inspect effective settings with sshd -T where appropriate, and use the documented service reload command. Open a new terminal and verify a fresh login before closing the working session. If it fails, restore the known-good file using console access.

6. Broader basic server hardening

Remove or disable services that are not needed, after checking dependencies.
Apply supported security updates through the distribution’s normal process.
Use named accounts and least privilege; elevate with sudo only when needed.
Limit network access to required sources and services with a host or network firewall.
Protect backups, SSH private keys, and configuration files.
Monitor authentication and service logs; define how unusual activity is reviewed.
Document the owner, purpose, and rollback for each configuration change.

Changing the SSH port is not a replacement for authentication controls or firewall rules. Avoid applying a copied hardening checklist without testing the actual service and recovery path.

Practical exercise

In a disposable Linux VM, create a passphrase-protected key, install its public key for a non-root test account, verify the host fingerprint from the VM console, and connect using a client alias. Inspect the effective SSH configuration. Do not disable password access until a second key-based login succeeds and you have console recovery available.