Introduction

SSH (Secure Shell) is the workhorse of server administration, allowing system administrators, developers, and DevOps engineers to manage remote servers efficiently. Whether you’re configuring a web server, deploying applications, or troubleshooting issues, SSH is your trusted companion.

In most scenarios, servers are set up in headless mode which means they operate without a GUI. It streamlines resources, reduces attack surfaces, and ensures efficient performance, however, this can lead to a secure traffic nightmare.

Ideally, you want to deploy measures that make sense and keep your environment secure. Utilizing solutions like changing ports (though not a foolproof security measure), MFA, or SSH Keys. In best practice, each user should have their own key or MFA solution, and not reusing passwords or keys in any scenario. Additionally, it's best to restrict login access and reduce sudo rights.

I stumbled initially, but then I obtained assistance. Conversations with TechnoTim and his vibrant Discord Community provided invaluable insights. Armed with newfound knowledge, I set out to rectify my practices. In this article, we’ll explore essential best practices to enhance the security of your SSH server.

In this article, we’ll explore essential best practices to enhance the security of your SSH server. For this content, I will be referring to Linux-based systems. I am using Ubuntu-based operating systems, however, the commands should be universal in most scenarios.

User Stature

The idea of server access is not duplicating credentials, but also granting only what is needed. Each server/user should have their own account, ssh-key and it should never be shared between users. If users need to have admin access, their base account should not be granted sudo access, instead, they should have another account on the server to make and grant those roles. If you intend on doing a file server or an SFTP, it would be wise to have a separate account for SFTP access only as this can help mitigate potential threat vectors.

Creating and Installing SSH Keys

Let's log in to our SSH account. My account name is general. Once we are logged in, we need to navigate to the home directory of the general account and create a hidden folder with specific permissions.

cd /home/general/
mkdir .ssh
cd .ssh
ssh-keygen

Save the file in /home/general/.ssh/. It created two files. One is the server key that the server will use and the other is the key that we will use. We need to create another file and add our key to it so the server knows what to use.

cat id_rsa.pub > authorized_keys
cd ..
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

We need to display our private key file so we can copy it and attach it to any machine moving forward for authentication.

cat id_rsa

Select everything and press Control+C to copy it to your clipboard. If you are on a MacOS system use Command+C.

On your machine, open up a text editor and paste in the key. Make sure to remove the spaces. Then save it as all types. Type in id_[servername] for the file name. Save it in your Documents or a folder that you will be using for server stuff.

Now we need to check to see if this works. In your terminal, log off the server and get to your normal shell. Usually typing exit a few times will get you back.

Navigate to your .ssh folder and we are going to make a new file called config. This is where we will program our server's information so we can connect just by saying ssh server , or the equivalent of.

cd ~/.ssh
nano config

Fill in the blanks to match your information.

Host [servername]
     HostName [serverIP]
     User [server username]
     Port [#]
     IdentityFile ~/.ssh/id_[servername]

If you saved the key on your Linux server within the terminal, you need to set the permissions to 600. You can do so by using this command: sudo chmod 600 ~/.ssh/id_[servername]. Substitute as needed.

Let's now try the general account, and see if It accepts the key and logs us in.

If you are using MacOS, and you copied the key from a Windows Machine or NAS device, you will likely have to convert the key. You will need HomeBrew installed on your MacOS machine for this to work. Here is how to use Brew to install putty and convert the key.

brew install putty
puttygen ~/.ssh/keyname -O private-openssh -o ~/.ssh/keyname_mac

You can login by using ssh host_name in your terminal and it should authenticate you at this point. Substitute as necessary.

Set SSH Settings

In this section, we need to run the following as the root account. You can access root by running sudo su and entering your sudo password.

Global Mask

Let's start off by setting a global mask. This is a mask that all profiles will abide by at a permission level.

echo "umask 027" >> /etc/profile

Time Server

Setting the timezone for the server for the location that it is in can be beneficial. If it is hosted elsewhere (in the cloud) then it might be crucial to set it for your own environment. Using the timedatectl command, we can set the zone. If you need to find your timezone, you can use timedatectl list-timezones and output in the console.

timedatectl set-ntp true
timedatectl set-timezone America/Chicago
echo 'servers=0.pool.ntp.org 1.pool.ntp.org 2.pool.ntp.org 3.pool.ntp.org' >> /etc/systemd/timesyncd.conf

App Armor

App Armor is an effective and easy-to-use Linux application security system. This proactively protects the OS and apps from potential threat vectors. App Armor is supported by most Linux systems. Debian, Ubuntu, and OpenSUSE ship with App Armor. App Armor is actively developed and continuously supported by Canonical.

You can use aa-status to check to see if it is installed and loaded, however, below are commands to install additional profiles and utilities that may not be installed from the base OS install.

apt update
apt install apparmor apparmor-profiles apparmor-utils apparmor-easyprof -y
apt install libpam-tmpdir libpam-apparmor libpam-cracklib -y
echo 'session optional pam_apparmor.so order=user,group,default' > /etc/pam.d/apparmor
systemctl start apparmor.service
systemctl enable apparmor.service

After you enable apparmor, go ahead and use the exit command to go back to your normal account.

SSH Config

While security through obscurity is never considered a fool-proof security measure, it can help mitigate bad traffic and lower traffic noise. I would consider changing the default ssh port. There are instances like zero-trust agents, or cloud server providers where port 22 needs to be allocated for SSH for their tools. In which case, it's fine. You should be practicing good credential hygiene and also utilizing a firewall setup.

In addition, we need to make some additional modifications. We will disable logging in as root, set an idle time logout, limit attempts, disable empty passwords, and stop X11 forwarding (which allows GUI apps to run over SSH). If you require X11, then you can leave that alone. We will also create a group that will only allow the select users access over SSH.

To handle timeouts, The ClientAliveInterval manages idle ssh connections. The server sends a message to the client and expects a response, the interval is the space between the messages. The ClientAliveCountMax defines how many times the server will do this before deciding the client isn't really there anymore to which the connection is then dropped.

Locate the following below in the configuration file, you may need to scroll and remove # (to uncomment) and make the necessary changes.

Configuration

sudo addgroup sshaccess
sudo adduser [your user to have access] sshaccess
sudo nano /etc/issue.net

WARNING! Authorized Users Only!

Property of harleybfrank.com
Logging is enabled and malicious acts will be taken seriously to the
fullest extent of the law.

jcnjux-websrv01

sudo nano /etc/ssh/sshd_config

Port ###
AllowGroups sshaccess

LogLevel VERBOSE

LoginGraceTime 2m
PermitRootLogin no
MaxAuthTries 3
MaxSessions 10

PubkeyAuthentication yes

PasswordAuthentication No
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM no

GatewayPorts no
AllowTcpForwarding no
X11Forwarding no
PrintLastLog yes
Compression no
ClientAliveInterval 60
ClientAliveCountMax 3
MaxStartups 10:30:60
AllowStreamLocalForwarding no

Banner /etc/issue.net

Use Control+O to save and Control+X to exit. Then type in sudo service sshd restart to force the new changes to take effect.

Troubleshooting

You should adhoc test this before you close down your main terminal window, just in case something doesn't work out right. After you are satisfied with your new settings, then you can log out, and back in again for the new settings to take effect on your account.

You can run the following commands to make sure that this has taken effect:

sudo sshd -T | grep passwordauthentication
sudo sshd -T | grep challengeresponseauthentication
sudo sshd -T | grep usepam

Configure SFTP

As we are securing we should secure FTP, or enable SFTP if you are already using FTP. In this example I am using vsftpd, however, from my current understanding and reading, most sftp applications that run headless should have the same basic configuration. I will disclaim that your mileage will vary.

SFTP is a secure file transfer protocol, and it allows for secure FTP connections where you can connect to an endpoint to move data

Install VSFTPD

We need to set up a way to access our server remotely to upload and download files. This is an extremely useful tool to have on any server.

Let's start by installing the client:

sudo apt-get install vsftpd

Now we need to configure it:

sudo nano /etc/vsftpd.conf

Use Control+O to save and Control+X to exit.

Make sure these are not commented out [#] and they are displayed like below. Go ahead find these and make the changes. We are going to enable writing, lock the user down to their own home folder, and configure uploads for user. Now if you need more flexibility or access to the server in other areas, then not setting chroot would be the route to go.

anonymous_enable=NO
local_enable=YES
write_enable=YES
chroot_local_user=YES
local_umask=022
chown_uploads=YES

Restart VSFTPD:

sudo service vsftpd restart

Symlinking for Security

If you are locking down SFTP to the user's home folder then we need to create a symlink in their home directory so that way they can access the data they need or have permission to. Use the following command to generate a symlink in the User's respective home folder.

sudo ln -s /location/location/location /home/username

Modify SSH Config to Block SSH Login

If you want this user to only be allowed to SFTP and not SSH into your system you can use the previous steps above, and modify the sshd_config to add this at the bottom. This ensures that the user you designate cannot log in via ssh and is only allowed through the SFTP protocol.

Match User username
  ForceCommand internal-sftp

SFTP Connections

You can use the terminal to make SFTP connections, instead of ssh username@ip or ssh hostname use sftp username@ip or hostname and it opens up a session. Here is a list of commonly used SFTP in the terminal.

1. ls: List the files and directories in the current directory on the remote server.
2. lls: List the files and directories in the current directory on your local machine.
3. cd directory: Change the current directory on the remote server.
4. lcd directory: Change the current directory on your local machine.
5. pwd: Display the current directory path on the remote server.
6. lpwd: Display the current directory path on your local machine.
7. get remote-file [local-file]: Download a file from the remote server to your local machine. If local-file is specified, the file is saved with that name; otherwise, it retains its original name.
8. put local-file [remote-file]: Upload a file from your local machine to the remote server. If remote-file is specified, the file is saved with that name; otherwise, it retains its original name.
9. mget remote-files: Download multiple files from the remote server to your local machine.
10. mput local-files: Upload multiple files from your local machine to the remote server.
11. mkdir directory: Create a new directory on the remote server.
12. rmdir directory: Remove a directory on the remote server (the directory must be empty).
13. rm file: Remove a file on the remote server.
14. rename oldpath newpath: Rename or move a file or directory on the remote server.
15. chmod mode file: Change the permissions of a file on the remote server. mode is specified in Unix chmod format (e.g., 755).
16. chown uid file: Change the owner of a file on the remote server. uid must be an integer.
17. chgrp gid file: Change the group of a file on the remote server. gid must be an integer.
18. symlink oldpath newpath: Create a symbolic link on the remote server.
19. df [-hi] [path]: Display statistics about space usage on the file system that contains the specified path or the current directory.
20. !command: Execute command in the local shell.
21. bye, exit, quit: Exit the SFTP session.

Conclusion

Understanding SSH is essential for effective server management. While the conventional methods are clean, they may not always be the best. Continuously seek improvements—use SSH keys, rotate them, avoid reusing keys or passwords, and ensure every user has their own key. Restrict access to authorized users and limit sudo privileges. Remember, security is an ongoing journey. Stay vigilant and adapt as needed.

Full Disclosure

Most of this article is comprised of facts and opinions. The featured background image was created by andyoneru and is available on Unsplash. I added a blur and a gradient overlay with some text. The following images have been pulled or screenshotted from the respective websites/applications. I do not own this content.