Bruce Banner: IP banning tool

Bruce Banner

bruce banner is an IP banning tool. It queries our elasticsearch database for intrusion attempts, manages a list of IPs to ban and communicates bans to a network of remote server responsible for enforcing the ban via nftables.

See our intrusion detection page for the full picture of how it fits in with our other tools.

Topics

bruce can be configured to manage different ban “topics” for different services. For example, some servers can subscribe to the “ssh” topic which triggers a ban on just port 22 or a “web” topic and ban ports 80 and 443. You can specify any number of topic port combinations via the ansible host variable bruce_banner_subscribe_topic (e.g. ssh:22 web:80,443).

Supporting users who may have been banned

From our monitor server, you can use the bruce cli tool to check if an IP address has been banned, e.g.:

# bruce is-banned 123.123.123.123
Connecting to mysql database bruce on localhost
123.123.123.123: IS banned
Banned: 2022-10-16 06:19:42
Expire: 2022-10-18 06:19:42
Topic: mail
Count: 5

You can unban it with:

bruce remove 123.123.123.123 mail

You can also search for the IP via kibana to learn more about why it was banned.

General architecture

  • bruce-elastic: the query component searches our elasticsearch database regularly to search for signs of attack and communicates those IP addresses to the server component.

  • bruce-server: the server component.

    It opens two sockets:

    1. It listens via 127.0.0.1 on port 5556 (by default) where it runs the zeromq PUB part of a PUB-SUB service.

    2. It connects to a unix socket via /run/bruce/bruce.sock where it listens for commands from the bruce command line tool or the bruce-elastic service.

  • bruce-banner: the client component runs and connects to the server component via zeromq and waits for information on IPs to ban.

    The bruce-banner client communicates with the bruce-server via an ssh proxy. It must be configured with password-less ssh access to the bruce-transport user on the server running bruce-server.

  • bruce: additionally, there is a command line program that controls the bruce-server. It can be used to add IPs to the list of IPs to ban, remove IPs (e.g. bruce add 1.2.3.4 ssh or bruce remove 1.2.3.4 ssh).

Typical sequence of events

It doesn’t matter in what order the clients and servers start.

Typically:

  1. Start the bruce-server

  2. Start bruce-elastic which communicates IPs to ban to bruce-server.

  3. bruce-server records the ban, ban time and date it should expire in the dataase and broadcasts it to al subscribers to the topic.

  4. New client starts the bruce-banner program. The program starts by requesting a sync of all current bans for it’s chosen bans and bans them.

  5. Then, bruce-banner continues listening for new ones.

Permissions

The server component runs as the bruce-server user and will have full access to the database of IPs.

A group called bruce is granted write access to the bruce-server’s unix socket. The bruce-elastic user is in this group so it can add IPs to the ban pool.

Lastly, a bruce-transport user exists on the bruce-server machine and is configured to allow all bruce-banner clients ssh access so they can connect to the server. The bruce-transport user does not have access to run the bruce command or the /run/bruce/bruce.sock so they can’t add or remove IPs. However, bruce-transport has sudo privileges to run bruce sync * so it can trigger the server to send it all bans it may have missed while it was not running.

Ignore list

With bruce, you can define a list of IPs to ignore via three methods and bruce will take care of keeping the list up to date:

  • domain name: The IPs that the domain name resolves to will be ignored
  • web site: All IPs listed on a web site will be ignored (supports password protected web sites)
  • static: a static list of IPs can also be designated.

This configuration is specified in a yaml formatted file (derived from the ansible variable bruce_ignore).

The bruce refresh-ignore command is called via a systemd timer once a day to update it.

Banning

bruce bans using nftables. Specifically, it creates an nftables set, defines a rule for that set specifying which ports should be blocked, and then simply adds IPs to that set with a timeout value for when they should be automatically removed.

Debugging

On the bruce server, login to the bruce database (as root) with:

mysql bruce

Then, review bans:

SELECT * FROM banned;

To review the IP addresses banned on a particular host, run:

 nft list set inet bruce bruce-ssh
 nft list set inet bruce bruce-web
 nft list set inet bruce bruce-mail

Change the ‘ssh’ or ‘web’ or ‘mail’ parts depending on the particular ban.

To view the rules (what port is banned, etc):

nft list ruleset | less

To remove a particular IP address from the ban:

nft delete element inet bruce bruce-ssh {1.2.3.4}

To check for hosts that might not be banning properly, run the following on the monitor server running bruce-server (e.g. monitor002):

mf-bruce-not-running

If a server is out of sync or seems to be missing IPs that should be banned, ensure that the ssh-proxy service is running:

systemctl status bruce-banner-ssh-proxy.service

If bruce-banner-ssh-proxy doesn’t start properly, you may need to re-run sower but with the --secrets argument to ensure the ssh private key is copied over.

Restart bruce-banner-ssh-proxy if necessary.

Next, restart the main service with: systemctl restart bruce-banner. The first thing bruce will do is request all IPs that should be banned and ensure they are banned.