Sunday, September 01, 2013

HOWTO : High Performance IDS/IPS with SmoothSec 3.2

Since the previous setup (HOWTO) of SmoothSec are not perfect, I am going to use AF_PACKET as packet acquisition engine. In this setting, you are required to have at least 3 network interfaces, one for the management purpose.

As AF_PACKET has high performance, even the very low-end hardware is benefited. The following setup is ideal for home/SOHO environment.

(A) Hardware

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (2-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 (eth2) - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

(B) Software

Operating System - Debian 7.0 (Wheezy)
IDS/IPS pre-configure system - SmoothSec 3.2 (64-bit)
IDS/IPS Engine - Suricata
Spooler - Pigsty
Web Interface - Snorby
Rules Management - PulledPork

(C) Setup

Internet -- Router -- SmoothSec -- Switch -- Personal Computers

The SmoothSec will monitor all the incoming and outgoing traffic between router and the switch.

Step a - Cable connection :

First of all, SmoothSec (Network Card 2) is connected to the Switch while Network Card 0 and 1 do not connect to the router at the moment. It is because you need to access the internet for the SmoothSec installation.

Step -1 - Installation of SmoothSec :

Install SmoothSec as usual or refer to SmoothSec Wiki. When you are prompted to install non-free network interface firmware, you just ignore it. It is because Debian missed some firmware for Realtek 8169. After the installation, reboot the box as advice. The username is "root" while the password is "toor".

Step 0 - Install the missing packages :

apt-get install ethtool postfix fail2ban openjdk-7-jre

If you want to use Postfix as mail server for the Snorby report, you should install it and configure it after the install. For the configuration of Postfix, you may ask Google if you do not know how to.

Make sure you select "Internet Site" when installing Postfix.

You may consider to install fail2ban to protect your ssh connection inside the network.

To improve the SmoothSec :

apt-get --purge remove arpwatch
apt-get install arpalert
cd /etc/arpalert/
mv oui.txt oui.txt.old

Step 1 - Get new Linux Kernel :

In order to install a high performance IDS/IPS, you need a newer kernel that the version should be 3.7 or greater.

apt-cache search linux-image

To look for Linux Kernel version that is greater than 3.7. If not, just add the following repos :

nano /etc/apt/sources.list

Append the following line (the address of the source may be different from yours, but it must be "unstable" :
deb unstable main
deb-src unstable main

Then look for Linux Kernel version that is greater than 3.7 :

apt-get update

I select the version 3.10 :

apt-get install linux-image-3.10-2-amd64 linux-headers-3.10-2-amd64

When you are asked to restart some services when install, just reply "yes".

You will be warned for some firmware missing, just ignore it. It is because Debian do not has some firmware for Realtek 8169. Anyway, it is harmless.

After the new kernel install, comment out what you added in "/etc/apt/sources.list". This step is VERY IMPORTANT as the newer version of Apache (2.4.x) and Perl will break the Snorby and PulledPork, the web interface of the SmoothSec and rules management tool.

Then reboot the SmoothSec and select the new kernel when it is available.

Step 2 - Configure Suricata :

nano /etc/suricata/suricata.yaml

Locate "#- delayed-detect: yes" and replace with "- delayed-detect: yes".

Locate "- fast:" and replace "enabled: no" with "enabled: yes".

Locate "- drop:" and replace "enabled: no" with "enabled: yes".

Locate "af-packet:" and replace "threads: 1" with "threads: 4". Or, the number of core of CPU you have.

Locate "#checksum-checks: kernel" and replace it with "checksum-checks: kernel".

Locate "#copy-mode: ips" and replace it with "copy-mode: ips".

Locate "#copy-iface: eth1" and replace it with "copy-iface: eth1".

Add "buffer-size: 64535" just below "copy-iface: eth1".

Locate "- interface: eth1" and replace "threads: 1" with "threads: 4". Or, the number of core of CPU you have.

Add the following lines just below "# disable-promisc: no" :

buffer-size: 64535
copy-mode: ips
copy-iface: eth0
use-mmap: yes
checksum-checks: kernel

Locate "rule-files:" and add "- local.rules" just below "- emerging.rules".

touch /etc/suricata/rules/local.rules

nano /etc/init.d/suricata

Locate "/usr/local/bin/suricata --user suricata -c /etc/suricata/suricata.yaml -i $INTERFACES -D" with "/usr/local/bin/suricata --user suricata -c /etc/suricata/suricata.yaml --af-packet -D"

There are 2 entries, you should replace them all.

Step 3 - Time Zone :

Make sure your SmoothSec is UTC no matter your time zone is. It is because, Snorby is only working on UTC. Otherwise, the timestamp of Snorby will be wrong.

To check time zone of SmoothSec :


If the time is not UTC, you need to change back to it :

dpkg-reconfigure tzdata

Set the time zone to "UTC" at "None of the above".

Step 4 - Configure email feature of Snorby :

If you installed Postfix, configure it properly according to your network at "/etc/postfix/".

nano /var/www/snorby/config/initializers/mail_config.rb

Then uncomment the lines just below "#Sendmail Example:". Or, refer to the SmoothSec Wiki for the installation.

Step 5 - Configure network interfaces :

Make it looks like the following. Make sure your eth2 has your IP "address" and "gateway" instead of "" as it is an example only :

nano /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
   up ifconfig eth0 up
   down ifconfig eth0 down
   post-up ethtool -K eth0 gro off

auto eth1
iface eth1 inet manual
   up ifconfig eth1 up
   down ifconfig eth1 down
   post-up ethtool -K eth1 gro off

# The primary network interface
#allow-hotplug eth2
#iface eth2 inet dhcp
auto eth2
iface eth2 inet static

* Please note that ethtool is used as the Realtek network interfaces will produce error when working as AF_PACKET method.

Error messages when doing debugging with "suricata -c /etc/suricata/suricata.yaml --af-packet" :

[ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 10: Message too long
[ERRCODE: SC_ERR_INVALID_ACTION(142)] - Unable to release packet data

Step 6 - Configure SmoothSec :

Run the following script to setup SmoothSec :


Type "eth0" when asks for the monitor interface. Enter "" when asked for the network. Please note that the address here is an example only. When asked for Intrusion Detection Engine, you type "2" for Suricata. The email address and password asked are for the login purpose of Snorby (web interface).

Step b - Cable connection :

Connect Network Card 0 to the router and Network Card 1 to the switch. Network Card 2 connect to the switch.

When done, reboot the SmoothSec.

Step 7 - Browse Snorby :

You need to wait for several minutes before you can connect to the internet as Suricata need some time to do with the rules.

Open your browser and enter the following url :

Accept the certificate and wait for about a minute the Snorby will show up.

Step 8 - IPS Setup :

Now your SmoothSec is running as IDS (Intrusion Detection System) and it will not block or drop any malicious traffic.

To configure the SmoothSec to running as IPS (Intrusion Prevention System), you need to :

nano /etc/pulledpork/suricata/dropsid.conf

Append the following :


So, you will drop/block any malicious traffic that match the vulnerabilities in the vulnerability reports, such as cve and bugtraq as well as Microsoft's. Meanwhile, you can add your own rules in "/etc/suricata/rules/local.rules". Make sure to run "smoothsec.suricata.rules.update" after your add them.

You may want to disable some rules :

nano /etc/pulledpork/suricata/disablesid.conf

Append the following :


It will disable the rules serial 2210000 to 2210049. A total of 50 rules to be disabled.

After done, run the following script :


* Please also note that you are required to wait for several minutes before you can connect to the internet as Suricata requires some time to do with the rules.

(D) Troubleshooting

(1) In case you find there is no GeoIP information on the events, you should check if the file "snorby-geoip.dat" in /var/www/snorby/config/ or not. If not, just download it, please follow the below commands. If the file does not exist, that mean you cannot connect to the internet when installing Snorby.

cd /tmp/
gzip -d GeoIP.dat.gz
mv GeoIP.dat snorby-geoip.dat
chown www-data:www-data snorby-geoip.dat
cp snorby-geoip.dat /var/www/snorby/config/

(2) In case you do not capture the events, you should make sure the correct interface name (such as eth0, eth1 and eth2) are connected properly. You should examine the MAC address of the Network Card to determine the correct interface name.

(E) Performance

The SmoothSec is installed on a low-end hardware (Intel Atom D510 CPU with Realtek Gigabit NICs). It is also behind a router, which is running Untangle (Intel Atom D510 CPU with Realtek Gigabit NICs). Untangle is an UTM (Unified Threat Management System) which can block some malicious traffic (but a few only). The switch is D-Link DGS-1008D (Home) Gigabit switch.

To test the performance, I am watching a youtube at 1080p on PC-1 (via wifi), watching a youtube at 720p on PC-2 (via wifi) and watching a youtube at HD on Android smartphone with wifi. The result is very smooth without any lagging for all the devices.

The CPU loading for the test is below 4.x and memory used is below 3GB.

AF_PACKET is ideal for IDS/IPS implementation when you have a very low-end hardware.

(F) Limitation

Since SmoothSec 3.2 is build on Debian 7.0 (Wheezy), the system will be broken when you upgrade to Sid (Unstable). The newer version of Apache (2.4.x) and Perl will refuse to run due to error. Therefore, when you installed the newer kernel (for AF_PACKET purpose), make sure you comment out the repos that you added in order to prevent the system upgrade to the Sid (Unstable) by accident.

Another limitation is that you are requested to have at least 3 NICs for IDS or IPS.

One more limitation is the Snorby cannot show the dropped traffic at the moment.

Known Issue

Pigsty will crash randomly. As a result, no capture in Snorby. The problem has been reported, see here. --> The workaround is to run a testing bash script on every 5 minutes to start Pigsty again.

nano /root/chkpigstylog

# Check if "Error: " in pigsty.log or not. If yes, start Pigsty again.
STRING="Error: "
if grep -R "$STRING" /var/log/pigsty.log

nano /root/runpigsty

/usr/local/bin/pigsty -c /etc/pigsty/suricata.pigsty.config.js -i eth0 -n "Suricata" -d /var/log/suricata/ -m unified2.alert.* -D

crontab -e
*/5 * * * * /root/chkpigstylog


Developers just fixed the problem. You just upgrade the Pigsty with the following command :

npm update pigsty-mysql -g
npm update pigsty -g

That's all! See you.