Integrating Fail2Ban with AWS Network ACLs

05/10/2018
Posted in AWS Blog
05/10/2018 Rutger Beyen

I was recently working on a project where I couldn’t lock down the Bastion instance security group ingress rule to only allow whitelisted IP addresses. Several coworkers work from home and use the Bastion to jump into backend servers and create SSH tunnels, while they did not have AWS Console access to whitelist themselves. The security group ended up allowing 0.0.0.0/0 on port 22. Off course the Bastion instance would only allow ssh key based logins, but all protocols eventually have vulnerabilities. So installing multiple layers of security in depth prevents the systems from being directly affected by the latest individual vulnerabilities.

Combining some bits and pieces from Google allowed me to setup Fail2Ban on the Bastion instance, while the blocking of the IPs is done in AWS NACLs in stead of the local Iptables. The setup has been done on an AmazonLinux instance.

Considerations

AWS NACLs by default only allow 20 ingress and 20 egress rules. This is a soft limit, and you can have it increased to 40 by opening a support case (40 seems to be the upper hard limit).

If fail2ban wants to add another rule while the maximum has been reached, it will block the offender in the local Iptables. Running my implementation in several environments for a few weeks now, I never had more than 4 to 5 IPs blocked at the same time. Unless you are off course the victim of a targetted ddos attack…

Setup the AWS CLI

Make sure you have a correctly installed and working AWS CLI on the instance. Also make sure the EC2 role has the necessary permissions to modify the EC2 Network ACL. Testing it out:

1. Get the current MAC address of the first interface

2. Get the subnet ID from the MAC address

3. Get the current Network ACL ID

4. Test that you can add a rule to the ACL

5. Verify that the above IP has been blocked

6. Remove the rule again

Fail2Ban AWS integration

1. Install the necessary packages (if not yet present)

2. Create a directory to store the AWS NACL script and cd to it

3. Place the following content in the file aws_nacl.py in the above directory

4. Test the script by calling it

5. Verify that the above IP was added to the ACL

6. Remove the IP again

7. Verify that the IP was removed again

Fail2Ban

1. Install Fail2Ban itself

2. Place the following file under /etc/fail2ban/action.d/aws.conf


3. Modify /etc/fail2ban/fail2ban.conf and change the logfile location (it’s easier to have a separate log rather then searching through /var/log/messages)

4. Add a file /etc/fail2ban/jail.local with the following content. Modify the values at your own convenience

5. Check /etc/fail2ban/filter.d/sshd.conf that the correct matching patterns for the /var/log/secure logfile are present, so that it actually looks for the loglines that you want to be considered as malicious. On an Amazon Linux, I have the following in place

6. Add fail2ban to the startup list and start it

Locked myself out

Things can always go wrong, but thanks to the recently released AWS feature called ‘SSM Session Manager’ you can always get a console window on your instance to start troubleshooting. One thing to make sure is that your instance is running the latest version of the AWS SSM Agent. So it’s always a good idea to update it before closing your current SSH session:

You also need to make sure that your instance is allowed to communicate with the AWS SSM service. Easiest way is to attach the ‘AmazonEC2RoleforSSM‘ policy to your EC2 role.

Credits

https://github.com/TheRemover/fail2ban-aws-nacl

https://techbytesecurity.com/2017/06/fail2ban-with-aws-network-acl

https://www.google.com

Share this AWSome post
, , , , , ,

Leave a Reply

Your email address will not be published. Required fields are marked *