Introduction

Introduction

SSH (Secure Shell) is the workhorse of server administration, allowing system administrators, developers, and DevOps engineers to manage remote servers efficiently. Whether you're configuring a web server, deploying applications, or troubleshooting issues, SSH is your trusted companion.

In most scenarios, servers are set up in headless mode, meaning they operate without a GUI. This streamlines resources, reduces attack surfaces, and ensures efficient performance—but it can also create security challenges.

To keep your environment secure, deploy sensible security measures. These include using SSH keys, implementing MFA, and even changing default ports (though this alone isn't foolproof). Best practices dictate that each user should have their own key or MFA solution, with no reuse of passwords or keys. It's also advisable to restrict login access and limit sudo rights.

I initially struggled with proper SSH security until I sought help. Conversations with TechnoTim and his vibrant Discord community provided invaluable insights. Armed with this knowledge, I improved my security practices.

In this article, we'll explore essential best practices to enhance your SSH server's security. I'll be focusing on Linux-based systems, particularly Ubuntu-based distributions, though most commands should work universally across Linux environments.

Overview

SSH, or Secure Shell, is a critical tool for anyone managing servers remotely. It provides a secure way to connect to and control servers without needing a GUI, which is especially common in headless setups. As someone who works with servers regularly, I find SSH invaluable for everything from deploying applications to troubleshooting issues. Its flexibility and security make it the backbone of remote server management.

One of SSH's standout features is key-based authentication, which enhances security far beyond traditional password logins. You can also customize it extensively, from configuring specific user permissions to limiting login attempts and even setting idle connection timeouts. However, there are some gotchas, like ensuring proper key management and adjusting configuration files carefully to avoid locking yourself out. It’s free to use, but depending on your setup, you might need additional tools or services to enhance its functionality, such as SFTP or AppArmor for added security layers. If you’re not already using SSH to manage your servers, you’re missing out on a powerful and essential tool.

User Stature

The idea of server access is not duplicating credentials, but also granting only what is needed. Each server/user should have their own account, ssh-key and it should never be shared between users. If users need to have admin access, their base account should not be granted sudo access, instead, they should have another account on the server to make and grant those roles. If you intend on doing a file server or an SFTP, it would be wise to have a separate account for SFTP access only as this can help mitigate potential threat vectors.

Creating and Installing SSH Keys

Let's log in to our SSH account. My account name is general. Once we are logged in, we need to navigate to the home directory of the general account and create a hidden folder with specific permissions.

cd /home/general/
mkdir .ssh
cd .ssh
ssh-keygen

Save the file in /home/general/.ssh/. It created two files. One is the server key that the server will use and the other is the key that we will use. We need to create another file and add our key to it so the server knows what to use.

cat id_rsa.pub > authorized_keys
cd ..
chmod 700 .ssh
chmod 600 .ssh/authorized_keys

We need to display our private key file so we can copy it and attach it to any machine moving forward for authentication.

cat id_rsa

Select everything and press Control+C to copy it to your clipboard. If you are on a MacOS system use Command+C.

On your machine, open up a text editor and paste in the key. Make sure to remove the spaces. Then save it as all types. Type in id_[servername] for the file name. Save it in your Documents or a folder that you will be using for server stuff.

Now we need to check to see if this works. In your terminal, log off the server and get to your normal shell. Usually typing exit a few times will get you back.

Navigate to your .ssh folder and we are going to make a new file called config. This is where we will program our server's information so we can connect just by saying ssh server , or the equivalent of.

cd ~/.ssh
nano config

Fill in the blanks to match your information.

Host [servername]
     HostName [serverIP]
     User [server username]
     Port [#]
     IdentityFile ~/.ssh/id_[servername]

If you saved the key on your Linux server within the terminal, you need to set the permissions to 600. You can do so by using this command: sudo chmod 600 ~/.ssh/id_[servername]. Substitute as needed.

Let's now try the general account, and see if It accepts the key and logs us in.

If you are using MacOS, and you copied the key from a Windows Machine or NAS device, you will likely have to convert the key. You will need HomeBrew installed on your MacOS machine for this to work. Here is how to use Brew to install putty and convert the key.

brew install putty
puttygen ~/.ssh/keyname -O private-openssh -o ~/.ssh/keyname_mac

You can login by using ssh host_name in your terminal and it should authenticate you at this point. Substitute as necessary.

Set SSH Settings

In this section, we need to run the following as the root account. You can access root by running sudo su and entering your sudo password.

Global Mask

Let's start off by setting a global mask. This is a mask that all profiles will abide by at a permission level.

echo "umask 027" >> /etc/profile

Time Server

Setting the timezone for the server for the location that it is in can be beneficial. If it is hosted elsewhere (in the cloud) then it might be crucial to set it for your own environment. Using the timedatectl command, we can set the zone. If you need to find your timezone, you can use timedatectl list-timezones and output in the console.

timedatectl set-ntp true
timedatectl set-timezone America/Chicago
echo 'servers=0.pool.ntp.org 1.pool.ntp.org 2.pool.ntp.org 3.pool.ntp.org' >> /etc/systemd/timesyncd.conf

App Armor

App Armor is an effective and easy-to-use Linux application security system. This proactively protects the OS and apps from potential threat vectors. App Armor is supported by most Linux systems. Debian, Ubuntu, and OpenSUSE ship with App Armor. App Armor is actively developed and continuously supported by Canonical.

You can use aa-status to check to see if it is installed and loaded, however, below are commands to install additional profiles and utilities that may not be installed from the base OS install.

apt update
apt install apparmor apparmor-profiles apparmor-utils apparmor-easyprof -y
apt install libpam-tmpdir libpam-apparmor libpam-cracklib -y
echo 'session optional pam_apparmor.so order=user,group,default' > /etc/pam.d/apparmor
systemctl start apparmor.service
systemctl enable apparmor.service

After you enable apparmor, go ahead and use the exit command to go back to your normal account.

SSH Config

While security through obscurity is never considered a fool-proof security measure, it can help mitigate bad traffic and lower traffic noise. I would consider changing the default ssh port. There are instances like zero-trust agents, or cloud server providers where port 22 needs to be allocated for SSH for their tools. In which case, it's fine. You should be practicing good credential hygiene and also utilizing a firewall setup.

In addition, we need to make some additional modifications. We will disable logging in as root, set an idle time logout, limit attempts, disable empty passwords, and stop X11 forwarding (which allows GUI apps to run over SSH). If you require X11, then you can leave that alone. We will also create a group that will only allow the select users access over SSH.

To handle timeouts, The ClientAliveInterval manages idle ssh connections. The server sends a message to the client and expects a response, the interval is the space between the messages. The ClientAliveCountMax defines how many times the server will do this before deciding the client isn't really there anymore to which the connection is then dropped.

Locate the following below in the configuration file, you may need to scroll and remove # (to uncomment) and make the necessary changes.

Configuration

sudo addgroup sshaccess
sudo adduser [your user to have access] sshaccess
sudo nano /etc/issue.net

WARNING! Authorized Users Only!

Property of harleybfrank.com
Logging is enabled and malicious acts will be taken seriously to the
fullest extent of the law.

jcnjux-websrv01

sudo nano /etc/ssh/sshd_config

Port ###
AllowGroups sshaccess

LogLevel VERBOSE

LoginGraceTime 2m
PermitRootLogin no
MaxAuthTries 3
MaxSessions 10

PubkeyAuthentication yes

PasswordAuthentication No
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM no

GatewayPorts no
AllowTcpForwarding no
X11Forwarding no
PrintLastLog yes
Compression no
ClientAliveInterval 60
ClientAliveCountMax 3
MaxStartups 10:30:60
AllowStreamLocalForwarding no

Banner /etc/issue.net

Use Control+O to save and Control+X to exit. Then type in sudo service sshd restart to force the new changes to take effect.

Troubleshooting

You should adhoc test this before you close down your main terminal window, just in case something doesn't work out right. After you are satisfied with your new settings, then you can log out, and back in again for the new settings to take effect on your account.

You can run the following commands to make sure that this has taken effect:

sudo sshd -T | grep passwordauthentication
sudo sshd -T | grep challengeresponseauthentication
sudo sshd -T | grep usepam

Configure SFTP

As we are securing we should secure FTP, or enable SFTP if you are already using FTP. In this example I am using vsftpd, however, from my current understanding and reading, most sftp applications that run headless should have the same basic configuration. I will disclaim that your mileage will vary.

SFTP is a secure file transfer protocol, and it allows for secure FTP connections where you can connect to an endpoint to move data

Install VSFTPD

We need to set up a way to access our server remotely to upload and download files. This is an extremely useful tool to have on any server.

Let's start by installing the client:

sudo apt-get install vsftpd

Now we need to configure it:

sudo nano /etc/vsftpd.conf

Use Control+O to save and Control+X to exit.

Make sure these are not commented out [#] and they are displayed like below. Go ahead find these and make the changes. We are going to enable writing, lock the user down to their own home folder, and configure uploads for user. Now if you need more flexibility or access to the server in other areas, then not setting chroot would be the route to go.

anonymous_enable=NO
local_enable=YES
write_enable=YES
chroot_local_user=YES
local_umask=022
chown_uploads=YES

Restart VSFTPD:

sudo service vsftpd restart

Symlinking for Security

If you are locking down SFTP to the user's home folder then we need to create a symlink in their home directory so that way they can access the data they need or have permission to. Use the following command to generate a symlink in the User's respective home folder.

sudo ln -s /location/location/location /home/username

Modify SSH Config to Block SSH Login

If you want this user to only be allowed to SFTP and not SSH into your system you can use the previous steps above, and modify the sshd_config to add this at the bottom. This ensures that the user you designate cannot log in via ssh and is only allowed through the SFTP protocol.

Match User username
  ForceCommand internal-sftp

SFTP Connections

You can use the terminal to make SFTP connections, instead of ssh username@ip or ssh hostname use sftp username@ip or hostname and it opens up a session. Here is a list of commonly used SFTP in the terminal.

1. ls: List the files and directories in the current directory on the remote server.
2. lls: List the files and directories in the current directory on your local machine.
3. cd directory: Change the current directory on the remote server.
4. lcd directory: Change the current directory on your local machine.
5. pwd: Display the current directory path on the remote server.
6. lpwd: Display the current directory path on your local machine.
7. get remote-file [local-file]: Download a file from the remote server to your local machine. If local-file is specified, the file is saved with that name; otherwise, it retains its original name.
8. put local-file [remote-file]: Upload a file from your local machine to the remote server. If remote-file is specified, the file is saved with that name; otherwise, it retains its original name.
9. mget remote-files: Download multiple files from the remote server to your local machine.
10. mput local-files: Upload multiple files from your local machine to the remote server.
11. mkdir directory: Create a new directory on the remote server.
12. rmdir directory: Remove a directory on the remote server (the directory must be empty).
13. rm file: Remove a file on the remote server.
14. rename oldpath newpath: Rename or move a file or directory on the remote server.
15. chmod mode file: Change the permissions of a file on the remote server. mode is specified in Unix chmod format (e.g., 755).
16. chown uid file: Change the owner of a file on the remote server. uid must be an integer.
17. chgrp gid file: Change the group of a file on the remote server. gid must be an integer.
18. symlink oldpath newpath: Create a symbolic link on the remote server.
19. df [-hi] [path]: Display statistics about space usage on the file system that contains the specified path or the current directory.
20. !command: Execute command in the local shell.
21. bye, exit, quit: Exit the SFTP session.

Conclusion

In conclusion, configuring SSH authentication is a critical step in securing remote server access and ensuring robust operational practices. Throughout this guide, we explored the importance of SSH in server management, delved into best practices for key-based authentication, and covered essential configuration steps to enhance security. By implementing these measures, such as creating unique user accounts, managing permissions responsibly, and customizing SSH settings, you can significantly reduce vulnerabilities and maintain tighter control over server access. Additionally, we touched on tools like AppArmor and SFTP, which add further layers of protection and functionality to your setup.

As I reflect on the material shared here, I hope it provides you with a well-rounded understanding of how to properly secure SSH connections. The methods discussed are practical and actionable, aimed at helping you safeguard your servers while optimizing their usability. Whether you're a seasoned system administrator or a beginner learning the ropes, these steps form a strong foundation for secure and effective server management. I encourage you to test, iterate, and refine your configurations to match your specific needs, and always stay informed about evolving best practices in cybersecurity.

Full Disclosure

Most of this article is comprised of facts and opinions. AI (specifically Grok and Notion AI) was used in the creation of this post. The featured background image was created by andyoneru and is available on Unsplash. I added a blur and a gradient overlay with some text. The following images have been pulled or screenshotted from the respective websites/applications. I do not own this content.