[HOWTO] FreeBSD 7.x 8.x + IPFW + Brute Force Monitor + block_ip.sh

[mmx]

Hello hello, here's a little guide to setup IPFW and DirectAdmin's BFM to play along nicely. I believe my method below is the cleanest out there, especially since I am using ipfw's tables for the best in organization and management. Using tables will allow you to maintain a separate block list, where your existing system firewall rules will not be modified.

Please make sure you have access to your server (locally, VM console or telnet) if you accidentally block yourself due to firewall configuration errors.

Here are the ingredients for this HOWTO:

FreeBSD 7.x and 8.x; x86 or x64
ipfw + tables support
DirectAdmin's Brute Force Monitor (version 1.40 required)


You will have the following at the end of this HOWTO:

Automated IP blocking by DirectAdmin's BFM.
Automated IP unblocking by DirectAdmin's BFM after a specified interval of your choice (Admin Level -> Administrator Settings).
A powerful stateless firewall.
Clean and organized firewall table management to keep things tidy.
Minimal modification to your system.




Enable IPFW on FreeBSD (More information (http://www.freebsd.org/doc/handbook/firewalls-ipfw.html))


Let's modify rc.conf and enable IPFW.


# ee /etc/rc.conf


Add this to rc.conf:


firewall_enable="YES"


Reboot your system to enable the firewall.
Once the system is online, modify the rc.conf again and add the following lines after firewall_enable="YES":


firewall_type="simple"
firewall_script="/etc/ipfw.rules"
firewall_logging="YES"



Additional changes may be required with sysctl (in regards to maximum firewall rules allowed). If you have a very busy server, you will get the following error message:


ipfw: install_state: Too many dynamic rules


This will allow FreeBSD to create and handle more dynamic rules in IPFW.

If you want to prevent a future possible headache, add the following to sysctl.conf (default is 4096):


# echo "sysctl net.inet.ip.fw.dyn_max=16384" >> /etc/sysctl.conf



Add some basic rules to your firewall.

You can customize the following to your liking. You might have to change the 'pif' to your network card's name. Check it with ifconfig.

I have added all the standard services: FTP, SSH, POP, SMTP/EXIM, DNS, HTTP, SSL ports (exim) and DirectAdmin's 2222. You might have to change them if your setup is different.

The "setup limit src-addr <X>" is how I handle connections to my server. I find this is the safest way to do so, and I limit the amount of connections an IP can make to a service (the <X> value). I find the values in this ruleset are enough. They have been tested on a production server for over a year.


Time to modify /etc/ipfw.rules:


# ee /etc/ipfw.rules


Paste the following in /etc/ipfw.rules:


#!/bin/sh
#################################################
# ipfw Firewall Commands
#################################################
cmd="ipfw -q add"
ipfw -q -f flush
pif="em0"

#################################################
# Allow Loopback and Deny Loopback Spoofing
#################################################
$cmd allow all from any to any via lo0
$cmd deny all from any to 127.0.0.0/8
$cmd deny all from 127.0.0.0/8 to any
$cmd deny tcp from any to any frag

#################################################
# Stateful rules
#################################################
$cmd check-state
$cmd deny tcp from any to any established
$cmd allow all from any to any out keep-state
$cmd allow icmp from any to any

#################################################
# Table 10 for IP blocks
#################################################
ipfw -q table 10 add 127.0.0.2
ipfw -q add 900 deny ip from 'table(10)' to any

#################################################
# Incoming/Outgoing Services
#################################################
$cmd 60001 allow tcp from any to any 21 setup limit src-addr 10
$cmd 60002 allow tcp from any to any 22 setup limit src-addr 8
$cmd 60003 allow tcp from any to any 25 setup limit src-addr 10
$cmd 60004 allow tcp from any to any 587 setup limit src-addr 20
$cmd 60005 allow tcp from any to any 53 setup limit src-addr 3
$cmd 60006 allow udp from any to any 53 limit src-addr 3
$cmd 60007 allow tcp from any to any 80 setup limit src-addr 20
$cmd 60008 allow tcp from any to any 110 setup limit src-addr 20
$cmd 60009 allow tcp from any to any 143 setup limit src-addr 10
$cmd 60010 allow tcp from any to any 443 setup limit src-addr 10
$cmd 60011 allow tcp from any to any 2222 setup limit src-addr 12
$cmd 60012 allow tcp from any to any 35000-35999 in setup limit src-addr 10
$cmd 60013 allow tcp from any to any 993 setup limit src-addr 10
$cmd 60014 allow tcp from any to any 995 setup limit src-addr 10
$cmd 60015 allow tcp from any to any 465 setup limit src-addr 10
$cmd 60016 allow tcp from any to any 585 setup limit src-addr 10


#################################################
# Deny Port scanning (Nmap)
#################################################
$cmd 00600 deny log logamount 50 ip from any to any ipoptions rr
$cmd 00610 deny log logamount 50 ip from any to any ipoptions ts
$cmd 00620 deny log logamount 50 ip from any to any ipoptions lsrr
$cmd 00630 deny log logamount 50 ip from any to any ipoptions ssrr
$cmd 00640 deny log logamount 50 tcp from any to any tcpflags syn,fin
$cmd 00650 deny log logamount 50 tcp from any to any tcpflags syn,rst


#################################################
# Deny and Log
#################################################
$cmd deny log all from any to any





Create the necessary DirectAdmin Block/Unblock scripts and make them executable:


# cd /usr/local/directadmin/scripts/custom
# touch block_ip.sh unblock_ip.sh brute_force_notice_ip.sh show_blocked_ips.sh
# chmod 755 block_ip.sh unblock_ip.sh brute_force_notice_ip.sh show_blocked_ips.sh


Get your copy and paste(s) ready!

block_ip.sh:


#!/bin/sh

echo "Blocking IP: $ip<br>";

/sbin/ipfw -q table 10 add $ip

echo "<br><br>Result:";
echo "<br><br><textarea cols=160 rows=60>";

/sbin/ipfw table 10 list

echo "</textarea>";

exit 0;



unblock_ip.sh:


#!/bin/sh

echo "Unblocking IP: $ip<br>";

/sbin/ipfw -q table 10 delete $ip

exit 0;



brute_force_notice_ip.sh:


#!/bin/sh

SCRIPT=/usr/local/directadmin/scripts/custom/block_ip.sh

ip=$value $SCRIPT

exit $?;


show_blocked_ips.sh (the sed removes the bits at the end so DA can pass the IPs):


#!/bin/sh

/sbin/ipfw table 10 list | sed 's/\/32 0//g'

exit 0;




Verify if DirectAdmin is showing you the Block and Unblock buttons in BFM.

Restart your server one more time to make sure everything is ok once it's online again. Double-check all services: mail, FTP, web, SSL connections, etc.

If you have any questions or come across some problems, please reply to this thread so everyone can make use of the information.

[Dauser2007]

Is there any different for Centos 32 bit ?

[zEitEr]

Do you have ipfw on your CentOS box? I really doubt.... but who knows.... http://centos.alt.ru/?p=447

[pipokoeie]

Thank you for this tutorial. priceless!

[pucky]

Hello hello, here's a little guide to setup IPFW and DirectAdmin's BFM to play along nicely. I believe my method below is the cleanest out there, especially since I am using ipfw's tables for the best in organization and management. Using tables will allow you to maintain a separate block list, where your existing system firewall rules will not be modified.

Please make sure you have access to your server (locally, VM console or telnet) if you accidentally block yourself due to firewall configuration errors.

Here are the ingredients for this HOWTO:

FreeBSD 7.x and 8.x; x86 or x64
ipfw + tables support
DirectAdmin's Brute Force Monitor (version 1.40 required)


You will have the following at the end of this HOWTO:

Automated IP blocking by DirectAdmin's BFM.
Automated IP unblocking by DirectAdmin's BFM after a specified interval of your choice (Admin Level -> Administrator Settings).
A powerful stateless firewall.
Clean and organized firewall table management to keep things tidy.
Minimal modification to your system.




Enable IPFW on FreeBSD (More information (http://www.freebsd.org/doc/handbook/firewalls-ipfw.html))


Let's modify rc.conf and enable IPFW.


# ee /etc/rc.conf


Add this to rc.conf:


firewall_enable="YES"


Reboot your system to enable the firewall.
Once the system is online, modify the rc.conf again and add the following lines after firewall_enable="YES":


firewall_type="simple"
firewall_script="/etc/ipfw.rules"
firewall_logging="YES"



Additional changes may be required with sysctl (in regards to maximum firewall rules allowed). If you have a very busy server, you will get the following error message:


ipfw: install_state: Too many dynamic rules


This will allow FreeBSD to create and handle more dynamic rules in IPFW.

If you want to prevent a future possible headache, add the following to sysctl.conf (default is 4096):


# echo "sysctl net.inet.ip.fw.dyn_max=16384" >> /etc/sysctl.conf



Add some basic rules to your firewall.

You can customize the following to your liking. You might have to change the 'pif' to your network card's name. Check it with ifconfig.

I have added all the standard services: FTP, SSH, POP, SMTP/EXIM, DNS, HTTP, SSL ports (exim) and DirectAdmin's 2222. You might have to change them if your setup is different.

The "setup limit src-addr <X>" is how I handle connections to my server. I find this is the safest way to do so, and I limit the amount of connections an IP can make to a service (the <X> value). I find the values in this ruleset are enough. They have been tested on a production server for over a year.


Time to modify /etc/ipfw.rules:


# ee /etc/ipfw.rules


Paste the following in /etc/ipfw.rules:


#!/bin/sh
#################################################
# ipfw Firewall Commands
#################################################
cmd="ipfw -q add"
ipfw -q -f flush
pif="em0"

#################################################
# Allow Loopback and Deny Loopback Spoofing
#################################################
$cmd allow all from any to any via lo0
$cmd deny all from any to 127.0.0.0/8
$cmd deny all from 127.0.0.0/8 to any
$cmd deny tcp from any to any frag

#################################################
# Stateful rules
#################################################
$cmd check-state
$cmd deny tcp from any to any established
$cmd allow all from any to any out keep-state
$cmd allow icmp from any to any

#################################################
# Table 10 for IP blocks
#################################################
ipfw -q table 10 add 127.0.0.2
ipfw -q add 900 deny ip from 'table(10)' to any

#################################################
# Incoming/Outgoing Services
#################################################
$cmd 60001 allow tcp from any to any 21 setup limit src-addr 10
$cmd 60002 allow tcp from any to any 22 setup limit src-addr 8
$cmd 60003 allow tcp from any to any 25 setup limit src-addr 10
$cmd 60004 allow tcp from any to any 587 setup limit src-addr 20
$cmd 60005 allow tcp from any to any 53 setup limit src-addr 3
$cmd 60006 allow udp from any to any 53 limit src-addr 3
$cmd 60007 allow tcp from any to any 80 setup limit src-addr 20
$cmd 60008 allow tcp from any to any 110 setup limit src-addr 20
$cmd 60009 allow tcp from any to any 143 setup limit src-addr 10
$cmd 60010 allow tcp from any to any 443 setup limit src-addr 10
$cmd 60011 allow tcp from any to any 2222 setup limit src-addr 12
$cmd 60012 allow tcp from any to any 35000-35999 in setup limit src-addr 10
$cmd 60013 allow tcp from any to any 993 setup limit src-addr 10
$cmd 60014 allow tcp from any to any 995 setup limit src-addr 10
$cmd 60015 allow tcp from any to any 465 setup limit src-addr 10
$cmd 60016 allow tcp from any to any 585 setup limit src-addr 10


#################################################
# Deny Port scanning (Nmap)
#################################################
$cmd 00600 deny log logamount 50 ip from any to any ipoptions rr
$cmd 00610 deny log logamount 50 ip from any to any ipoptions ts
$cmd 00620 deny log logamount 50 ip from any to any ipoptions lsrr
$cmd 00630 deny log logamount 50 ip from any to any ipoptions ssrr
$cmd 00640 deny log logamount 50 tcp from any to any tcpflags syn,fin
$cmd 00650 deny log logamount 50 tcp from any to any tcpflags syn,rst


#################################################
# Deny and Log
#################################################
$cmd deny log all from any to any





Create the necessary DirectAdmin Block/Unblock scripts and make them executable:


# cd /usr/local/directadmin/scripts/custom
# touch block_ip.sh unblock_ip.sh brute_force_notice_ip.sh show_blocked_ips.sh
# chmod 755 block_ip.sh unblock_ip.sh brute_force_notice_ip.sh show_blocked_ips.sh


Get your copy and paste(s) ready!

block_ip.sh:


#!/bin/sh

echo "Blocking IP: $ip<br>";

/sbin/ipfw -q table 10 add $ip

echo "<br><br>Result:";
echo "<br><br><textarea cols=160 rows=60>";

/sbin/ipfw table 10 list

echo "</textarea>";

exit 0;



unblock_ip.sh:


#!/bin/sh

echo "Unblocking IP: $ip<br>";

/sbin/ipfw -q table 10 delete $ip

exit 0;



brute_force_notice_ip.sh:


#!/bin/sh

SCRIPT=/usr/local/directadmin/scripts/custom/block_ip.sh

ip=$value $SCRIPT

exit $?;


show_blocked_ips.sh (the sed removes the bits at the end so DA can pass the IPs):


#!/bin/sh

/sbin/ipfw table 10 list | sed 's/\/32 0//g'

exit 0;




Verify if DirectAdmin is showing you the Block and Unblock buttons in BFM.

Restart your server one more time to make sure everything is ok once it's online again. Double-check all services: mail, FTP, web, SSL connections, etc.

If you have any questions or come across some problems, please reply to this thread so everyone can make use of the information.



Works a treat and thanks alot if nobody else has said it.

I implemented this with Freebsd 6.2. Used these rules as well and it all seems a go.

[pucky]

MMX i was wondering if you could comment on thes fin_wait2 states

The issue with this setup though is that when "limit" is used and there
is a dynamic rule for the traffic, lots of connections build up in the
FIN_WAIT_2 state. I have recently seen numbers in the upper hundreds and
they stay around for a long time. Without the limiting or dynamic
rules I don't recall any noticeable amount of FIN_WAIT_2 connections.

This has been causing problems for some visitors because connections
from their IP are building up and reaching the limit. The limit part
works great, but all the connections shown in ipfw's dynamic rules list
for some IPs are in the FIN_WAIT_2 state which is reaching the limit and
then not allowing any new traffic in from them. Then websites hosted
here appear down and most of the visitors wouldn't have any idea what's
going on.

Here is an example and these are just SOME not all;



tcp4 0 0 xxx 65.208.189.30.54935 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.30.44802 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.26.44758 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.27.37410 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.30.40634 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.27.37174 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.26.40725 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.25.45340 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.24.40975 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.30.39482 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.29.33178 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.28.33230 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.27.36051 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.26.39611 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.25.44231 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.24.39875 FIN_WAIT_2
tcp4 0 0 xxx 65.208.185.96.43715 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.31.44586 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.30.39451 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.29.33146 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.28.33192 FIN_WAIT_2
tcp4 0 0 xxx 65.208.189.27.36009 FIN_WAIT_2

[mmx]

Hi pucky, you're very welcome on the guide. :)

Can you please tell me what version of Apache are you running?

A quick Google search tells me that it is a problem with Apache. There are a lot of suggestions on this page (http://httpd.apache.org/docs/1.3/misc/fin_wait_2.html), but before I say anything, I would like to also know your complete system specs.

[mmx]

Actually, I just noticed that the IPs in the log you posted with the FIN_WAIT_2 are all from the same range. I think you are getting DDOS'ed. Using fail2ban, you can create a rule to block these IPs, or just add them to IPFW manually for the time being.

[pucky]

1.3 i think it is. Does it make a difference?

[pucky]

Exactly, i am being hit by a few ips since i closed a hacked UDP script they installed they have been hitting my box but that ip is not listed as an ip that DA said is brute forcing the box. I wonder why.

[mmx]

1.3 i think it is. Does it make a difference?

Hmm, thought so. I would recommend upgrading to 2.2 if it's feasible. I believe this is an Apache issue rather than IPFW.
Again, it looks like you are getting DDOS'ed with those IPs. Check out the link I sent you for some possible fixes if you must stay with 1.3.

[mmx]

Exactly, i am being hit by a few ips since i closed a hacked UDP script they installed they have been hitting my box but that ip is not listed as an ip that DA said is brute forcing the box. I wonder why.

DirectAdmin does not parse Apache logs with BFM. If you wish to block these IPs, fail2ban is the way to go. I can setup a small HOWTO at the end of this week if time permits, but in the meantime, check out fail2ban.org if you are technically adept.

You can install fail2ban via Ports and follow the examples in /usr/local/etc/fail2ban/. The jail.conf is the main configuration file, while the filter.d and action.d folders parse logs and take action using the specified rules.

[pucky]

Ye, ran fail2ban on another box once and i got so sick of it i disabled it. It drove me nuts.

[pucky]

Followed this thread and had brute force working. About a week later all my blocked ips are gone. I had about 4 or 5 ips banned but in directadmin there is no evidance of it. Grated, i have turned the firewall on and off a few times but should this be a reason why those banned ips no longer show up?

[zEitEr]

should this be a reason why those banned ips no longer show up?

Yes, it seems so, as with restarting a firewall all rules are getting flushed.

[pucky]

Right but thats not very practical. I mean, if you have 20 ips banned and then one day you have to stop the fw or take the box down those ips are lost? I mean i realize those rules are dynamically created when you click the BLOCK button but that is only for now. What about later? Seems it would make more sense just to add those ips to the blacklist file. Whats the point really?

[zEitEr]

It's up to you to modify a script which loads rules into ipfw in order to achieve the desired as directadmin (as it can be seen from here http://help.directadmin.com/item.php?id=380) already stores IP in

/root/blocked_ips.txt
/root/exempt_ips.txt


Or ask user mmx to implement it into his script.

[mmx]

Hello, I didn't want to store the IPs in a text file for a few reasons.


First, IPFW will block abusive IPs again once they start attacking you. As long as you have your firewall setup properly, this is not a concern, unless you want to be super-preventive.
Complexity increases slightly managing both IPFW tables and text files. I am sure you can script this, but I didn't want to go this way because I didn't like DirectAdmin's method of storing IPs.
I dislike using text files as a storage medium.
I want the ability to release banned IPs through the console. What if a user was accidentally banned?
Maintaining large sets of IP addresses in a text file might slow things down. Loading 10,000s IPs into IPFW would suck. I know you can grep this & sed that, but in the end, I wanted to use FreeBSD's native firewall system to manage this all for me.
I wanted to keep the clutter to a minimum.


You are welcome to implement the text-file method through the page zEitEr referenced. The only part you will have to adjust (loading IPs into a IPFW table) is as follows (untested, source (http://forums.freebsd.org/showthread.php?p=90743)):



exec < /root/blocked_ips.txt
while read ip
do
/sbin/ipfw -q table 10 add $ip
done


Again, this is untested, and it's here for you to get an idea on what to do. You would have to execute this script/function after IPFW creates the initial system rules and tables (e.g. #10).

If you can't figure this out, and you really want this done, I can try to put some time aside and write the scripts for you. However, you will have to beta test. :)

[wattie]

Strange but it stopped working since I upgraded to FreeBSD 9.0. The IP's are added to the blocklist (and appear in the list of blocked ip's in the BFM in DA) but some of the already blocked IP's are continuing to attack and attack Dovecot...

[interfasys]

DirectAdmin does not parse Apache logs with BFM. If you wish to block these IPs, fail2ban is the way to go. I can setup a small HOWTO at the end of this week if time permits, but in the meantime, check out fail2ban.org if you are technically adept.

You can install fail2ban via Ports and follow the examples in /usr/local/etc/fail2ban/. The jail.conf is the main configuration file, while the filter.d and action.d folders parse logs and take action using the specified rules.
I just wanted to add that while fail2ban may be a superior solution, you can add filters based on http to the DA brute force monitor by editing:
/usr/local/directadmin/data/templates/custom/brute_filter.list