Secure an Ubuntu OpenSSH server from Brute force attacks but without a firewall or SSH key pair?

Question

I use Ubuntu 16.04 and seek to harden my SSH authentication in a special manner.

The current situation:

I have a machine with a minimal Ubuntu server I use mainly to transfer files to via its local OpenSSH server. Now, I don't have a firewall on that machine from a few reasons and I also avoid using a key pair hence I use only a password. One of the only ways I have left to defend from brute force attacks, and the one I most desire at the moment, is using a mechanism that blocks a user for X amount of hours, after Y amount of connection tries.

The desired situation:

I wish to have a standalone mechanism (that is, not as part of a firewall) that blocks a user for X amount of hours, after Y amount of connection tries as a way to defend from brute force attacks.

My Question:

Do you know a utility (and specific configuration) that will let me achieve the desired situation?

Answer 1

This answer intends to give a possible way for satisfaction of the main question: Secure an Ubuntu OpenSSH server from Brute force attacks but without a firewall or SSH key pair?

Actually I prefer to use Firewall and SSH key pair and found the answer provided from Doug Smythies for really useful.

Protect SSH With Two-Factor Authentication

Two-factor authentication (2FA) is a type of multi-factor authentication. In this example 2FA confirming a user's claimed identity by utilizing a combination of these two different components:

• Time based, six digit token code - authentication code. By default, these tokens are good for 30 seconds plus additionally added 60 seconds in order to compensate for possible time-skew.

• User's password, which itself should be secure enough.

Actually when you have set PermitRootLogin no and the usernames are good chosen, for me, this method can be called 3FA.

Additionally, if the computer that you are logging into isn't hardened against brute-force login attempts, you can enable rate-limiting for the authentication module.

Let's begin:

1. Install dependencies

sudo apt-get install libpam-google-authenticator

2. Edit the configuration files

• Edit /etc/pam.d/sshd and add this directive:

  # Google Authenticator
  auth required pam_google_authenticator.so
  

  Add it in the beginning of the file. In this way the system will ask first authentication code and only then will ask password. Add it in the end of the file - system will ask first password.

• Edit /etc/ssh/sshd_config and modify or add these directives:

  ChallengeResponseAuthentication yes
  UsePAM yes
  PasswordAuthentication no           # You can leave this 'yes' it doesn't matter.
  

3. Activate the two-factor authentication for a user

Switch to the user who should use the two-factor authentication and type in the terminal:

$ google-authenticator Enter

Do you want authentication tokens to be time-based (y/n) yEnter
https://www.google.com/chart?chs=200x200&chld=M|0&cht=qr&chl=otpauth://totp/user@host%3Fsecret%3DE3CY3TNSNBXXXXXX


Your new secret key is: E3CY3TNSNBXXXXXX
Your verification code is 229999
Your emergency scratch codes are:
  19999711
  ...

Do you want me to update your "/home/user/.google_authenticator" file (y/n) yEnter

Do you want to disallow multiple uses of the same authentication token? This restricts you to one login about every 30s, but it increases your chances to notice or even prevent man-in-the-middle attacks 
(y/n) yEnter

By default, tokens are good for 30 seconds and in order to compensate for possible time-skew between the client and the server, we allow an extra token before and after the current time. If you experience problems with poor time synchronization, you can increase the window from its default size of 1:30min to about 4min. Do you want to do so 
(y/n) yEnter

If the computer that you are logging into isn't hardened against brute-force login attempts, you can enable rate-limiting for the authentication module. By default, this limits attackers to no more than 3 login attempts every 30s. Do you want to enable rate-limiting 
(y/n) yEnter

This dialogue will generate an authentication file called .google_authenticator placed in the user's home directory. This file can be used also for other user's accounts if you want all of them to use same tokens. Further this file can be customized and also can be used for 2FA within Apache2, but that's another story.

4. Generate authentication codes

The secret key - E3CY3TNSNBXXXXXX - generated in the above step is used for authentication codes generation within some application as:

• Commandline Authenticator
• These desktop apps mentioned here
• Extensions for web browsers described here
• Google Authenticator for Android
• Google Authenticator for iPhone/iPad

5. Example of usage

In this example is used the Authenticator extension for Chromium/Chrome:

6. Further reading

• The main source of this answer: How To Protect SSH With Two-Factor Authentication;
• Multi-Factor Authentication - how to enable SSH authentication using an OATH-TOTP app (the method described here) in addition to an SSH key pair.

---

EDIT:

In some cases there could has a difference between google's clock and the server's clock. Here you are few tips according this issue:

• What is the command to update time and date from internet

• Date: cannot set date: Operation not permitted

• Linux : Add x seconds to the current date

Unfortunately: If it's a VPS, you may not have permissions to do this... If you are using a VPS, contact your provider to handle this for you.

^{In case if your provider doesn't want to response to your requirements, the above set-up will work but with time-shift. Type date in your server's console and measure this time-shift. Then just work with this time-shift between the moment of authentication code generation and the moment of its use.}

Answer 2

You can do this with Fail2ban:

sudo apt-get install fail2ban

Then:

sudo vim /etc/fail2ban/jail.conf

edit bantime to set your desired ban time

edit maxretry to set maximum fail attempts

as mentioned by other comments, fail2ban requires iptables.

---

Separate option -- port knocking:

This requires only iptables, practically 0 memory and will effectively hide your service from port scans

Not directly answering your question, but maybe you can implement port knocking to hide your service availability instead of banning repeated attempts.

a quick google search reveals this: https://www.digitalocean.com/community/tutorials/how-to-configure-port-knocking-using-only-iptables-on-an-ubuntu-vps

You do require iptables though.

P.S.: I know security through obscurity is no security at all, but together with other practices it can help make you a more difficult target.

Answer 3

You don't need to install anything for such protection - Just add the relevant rules in your iptables system.

1. Prerequisite rule:

Early in your rule set, allow any related traffic to come back to the server:

sudo iptables -A INPUT -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT

2. Detection and blocking rules:

Set a dynamic Badguy List. Detect and DROP Bad IPs that do password attacks on SSH. Once they are on the BADGUY list, the system drops all their packets:

sudo iptables -A INPUT -i eth0 -m recent --update --hitcount 3 --seconds 90000 --name BADGUY_SSH -j LOG --log-prefix "SSH BAD:" --log-level info
sudo iptables -A INPUT -i eth0 -m recent --update --hitcount 3 --seconds 90000 --name BADGUY_SSH -j DROP
sudo iptables -A INPUT -i eth0 -p tcp -m tcp --dport 22 -m recent --set --name BADGUY_SSH -j ACCEPT

• In the above code, 90000 seconds (25 hours) is the block time.

• Any attempt from a blocked IP address, even non SSH related (depending on other rules that you may or may not have and the order), resets the block time timer.

3. Harden detections:

Limit the number of bad passwords per connection to 2. Default is 6.

As sudo edit /etc/ssh/sshd_config, and there set:

    MaxAuthTries 2