ricci, 2 months ago

OK, so what can we do about all these SSH brute force attacks?

We have a plan - actually, not just a plan, we run this in production on one of the @cloudlab clusters.

Let's start with this observation: if attackers are using a broad set of usernames, then we can use these username sets as a sort of signature. About half of attacking IP addresses only try one username, but that also means that about half are trying more than one - in fact some individual IP addresses tried more than 10,000 usernames!

What we do is this: we find sets of usernames that are used by more than one attacking IP address (actually it's a bit more complicated that this, details in the paper). This gives us "dictionaries" of usernames that are only used by attackers, and not any of our real users. We collect these dictionaries from the logfiles of a bunch of SSH servers, and combine them to form a Username Block List (UBL).

Now, all we have to do is: as soon as we see an IP address try a username from this UBL, we block it. That simple. We call this Dictionary Based Blocking (DBB).

How well does this work? We used logs from our clusters containing a total of 213 million login attempts, and it blocked 99.5% of all attempts, generating a false positives (accidentally locking out a real user) at a rate of just one about every five days.

But what about #fail2ban, you might ask? That's another method people use to block attacks against SSH (and other services) by locking out addresses that fail to log in more than X times in Y minutes. Well, with it's default settings, it only blocks about 66% of attacks, and it generates more than 5x as many false positives (graph attached). As it turns out, there is no way to tune fail2ban to get DBB's accuracy without a false positive rate that's orders of magnitude higher.

I said we run this in production - how well does that work? We run it on one of of CloudLab clusters that already had a firewall - subscribing to popular blocklists and running something very much like fail2ban. It's catching four-fifths of the attacks that were not already getting caught by these measures, and so far it hasn't caused a single false positive.