HOWTO : Hardening and Tuning Ubuntu 16.04 LTS

This guide will lead you to hardening and tuning your Ubuntu 16.04 in a few steps without any expense. As an Information Security Enthusiast, my Ubuntu box is setting up like the following and I use the box every day.

Kernel Hardening and Tuning

Make sure you enabled the "No Execute (NX)" or "Execute Disable (XD)" in the BIOS/UEFI.

sudo nano /etc/sysctl.conf

To make it looks like the following :



To reload it :

sudo sysctl -p

ARP Spoofing

One of the common attacks is Man In The Middle attack. It can use with browser attack too. This guide will help to protect your Ubuntu from being spoofing. Meanwhile, make sure to set ARP related settings in your router if the feature is available. Most home routers have no such feature.

HOWTO : ArpON on Ubuntu 16.04 LTS

Anti-Malware

Most Linux users (including advanced users) misunderstand that Linux will never and impossible to infect with malware. However, it is not true. Almost all operating systems are facing to the threats.

HOWTO : ClamAV 0.99 on Ubuntu 16.04 LTS

This guide will lead you to set up ClamAV for "Scan On Access" instead of "Scan On Demand".

Storage Performance Tuning

(A) SSD

Verify TRIM is supported :

sudo hdparm -I /dev/sda | grep TRIM

If the output is similar to the below which is supported :

* Data Set Management TRIM supported (limit 1 block)

If you install your Ubuntu in LVM, the TRIM is usually enabled by default. You can confirm it :

cat /etc/lvm/lvm.conf | grep issue_discards

If the output is similar to the below which is enabled :

issue_discards = 1

Then set the following to "deadline" if it is not done yet.

cat /sys/block/sda/queue/scheduler

noop [deadline] cfq

If not, set it :

sudo nano /etc/rc.local

Insert the following before "exit 0" :

echo 1024 > /sys/block/sda/queue/read_ahead_kb
echo 1024 > /sys/block/sda/queue/nr_requests
echo deadline > /sys/block/sda/queue/scheduler

* make sure your device is sda (or sdb ...)

To reload it or reboot your system :

sudo /etc/rc.local

After that, you need to edit the partition table (/etc/fstab) :

To make it looks like the following :

/dev/mapper/ubuntu--vg-root / ext4 noatime,nodiratime,norelatime,errors=remount-ro 0 1

sudo mount -a
sudo mount -o remount /

If you have an error after running the captioned commands, DO NOT reboot your system. You should correct the typo before doing so; otherwise, you cannot boot to your system again.

You can add "scsi_mod.use_blk_mq=1" to kernel parameter, such as "/etc/default/grub".

(B) Hard Drive

sudo nano /etc/rc.local

Insert the following before "exit 0" :

echo 1024 > /sys/block/sda/queue/read_ahead_kb
echo 1024 > /sys/block/sda/queue/nr_requests

* make sure your device is sda (or sdb ...)

To reload it or reboot your system :

sudo /etc/rc.local

After that, you need to edit the partition table (/etc/fstab) :

To make it looks like the following :

ext4 noatime,nodiratime,norelatime,errors=remount-ro 0 1

sudo mount -a
sudo mount -o remount /

If you have an error after running the captioned commands, DO NOT reboot your system. You should correct the typo before doing so; otherwise, you cannot boot to your system again.

Firefox Hardening and Tuning

Most malicious attacks nowadays are via browsers. Therefore, we need to protect ourselves even we are using Linux.

(A) Apparmor For Firefox

AppArmor is a Linux Security Module implementation of name-based access controls.

sudo apt-get update
sudo apt-get install apparmor-utils apparmor-profiles

sudo nano /etc/apparmor.d/usr.bin.firefox

To make it looks like the following and it is compatible to Firefox 51.0.1 or later :



sudo rm /etc/apparmor.d/disable/usr.bin.firefox

Reload the rules :

sudo apparmor_parser -r /etc/apparmor.d/usr.bin.firefox

(B) Firefox Add-ons

NoScript

Allow active content to run only from sites you trust, and protect yourself against XSS and Clickjacking attacks.

* You are not required to enable it as it will block the javascript that most modern websites are using. You need to disable it globally to make the XSS attack protection by default.

uBlock Origin (Optional)

Finally, an efficient blocker. Easy on CPU and memory. (Please refer to the official site for details)

WebRTC Control

Have control over WebRTC (disable or enable) and protect your IP address.

BetterPrivacy

Remove or manage a new and uncommon kind of cookies, better known as LSO's.The BetterPrivacy safeguard offers various ways to handle Flash-cookies set by Google, YouTube, Ebay and others...

HTTPS Everywhere

Encrypt the web! HTTPS Everywhere is a Firefox extension to protect your communications by enabling HTTPS encryption automatically on sites that are known to support it, even when you type URLs or follow links that omit the https: prefix.

Speed Tweaks (SpeedyFox)

This extension provides a list to almost all the settings that you may need to alter in order to enhance Firefox's speed.

(C) Optional

(1) Firefox Add-ons :

User-Agent Switcher

The User Agent Switcher extension adds a menu and a toolbar button to switch the user agent of a browser.

Youtube High Definition

YouTube High Definition is a powerful tool that will let you automatically play all YouTube videos in HD (High Definition), turn off annotations, change video player size, auto stop videos, auto mute videos, control embedded videos and much more.

Youtube Flash-Html5

Play YouTube Videos in Flash or HTML5 Player.

(2) Browse with "New Private Window" feature

Finally, you may consider to use Intrusion Detection and Prevention System (IDPS) to protect your network. My project "Almond Croissants" which is an IDPS that can deploy to your network (in front or/and behind the router). Or, you may also consider to use TOR to protect your privacy with my project "NightHawk".

Reference

WebRTC IPS

IP Leak

That's all! See you.

HOWTO : Hardening and Tuning Ubuntu 14.04 LTS

This guide will lead you to hardening and tuning your Ubuntu 14.04 in a few steps without any expense. As an Information Security Enthusiast, my Ubuntu box is setting up like the following and I use the box every day.

kernel Hardening and Tuning

Make sure you enabled the "No Execute (NX)" or "Execute Disable (XD)" in the BIOS/UEFI.

sudo nano sysctl.conf

To make it looks like the following :



To reload it :

sudo sysctl -p

ARP Spoofing

One of the common attacks is Man In The Middle attack. It can use with browser attack too. This guide will help to protect your Ubuntu from being spoofing. Meanwhile, make sure to set ARP related settings in your router if the feature is available. Most home routers have no such feature.

HOWTO : ArpON on Ubuntu 14.04 LTS

Anti-Malware

Most Linux users (including advanced users) misunderstand that Linux will never and impossible to infect with malware. However, it is not ture. Almost all operating systems are similar to the threats.

HOWTO : ClamAV on Ubuntu 14.04 LTS

This guide will lead you to set up ClamAV for "Scan On Access" instead of "Scan On Demand".

Network Interface Tuning

sudo nano /etc/rc.local

Insert the following before "exit 0" :

/sbin/ifconfig eth0 txqueuelen 10000
/sbin/ifconfig wlan0 txqueuelen 10000

* make sure your NICs are "eth0" and "wlan0" or else

Storage Performance Tuning

(A) SSD

Verify TRIM is supported :

sudo hdparm -I /dev/sda | grep TRIM

If the output is similar to the below which is supported :

* Data Set Management TRIM supported (limit 1 block)

If you install your Ubuntu in LVM, the TRIM is usually enabled by default. You can confirm it :

cat /etc/lvm/lvm.conf | grep issue_discards

If the output is similar to the below which is enabled :

issue_discards = 1

Then set the following to "deadline" if it is not done yet.

cat /sys/block/sda/queue/scheduler

noop [deadline] cfq

If not, set it :

sudo nano /etc/rc.local

Insert the following before "exit 0" :

echo 1024 > /sys/block/sda/queue/read_ahead_kb
echo 1024 > /sys/block/sda/queue/nr_requests
echo deadline > /sys/block/sda/queue/scheduler

* make sure your device is sda (or sdb ...)

To reload it or reboot your system :

sudo /etc/rc.local

After that, you need to edit the partition table :

To make it looks like the following :

/dev/mapper/ubuntu--vg-root / ext4 noatime,nodiratime,norelatime,errors=remount-ro 0 1

sudo mount -a
sudo mount -o remount /

If you have an error after running the captioned commands, DO NOT reboot your system. You should correct the typo before doing so; otherwise, you cannot boot to your system again.

You can add "scsi_mod.use_blk_mq=1" to kernel parameter, such as "/etc/default/grub".

(B) Hard Drive

sudo nano /etc/rc.local

Insert the following before "exit 0" :

echo 1024 > /sys/block/sda/queue/read_ahead_kb
echo 1024 > /sys/block/sda/queue/nr_requests

* make sure your device is sda (or sdb ...)

To reload it or reboot your system :

sudo /etc/rc.local

After that, you need to edit the partition table :

To make it looks like the following :

ext4 noatime,nodiratime,norelatime,errors=remount-ro 0 1

sudo mount -a
sudo mount -o remount /

If you have an error after running the captioned commands, DO NOT reboot your system. You should correct the typo before doing so; otherwise, you cannot boot to your system again.

Firefox Hardening and Tuning

Most malicious attacks nowadays are via browsers. Therefore, we need to protect ourselves even we are using Linux.

(A) Apparmor For Firefox

AppArmor is a Linux Security Module implementation of name-based access controls.

sudo apt-get update
sudo apt-get install apparmor-utils apparmor-profiles

To make it looks like the following :



sudo rm /etc/apparmor.d/disable/usr.bin.firefox

Reload the rules :

sudo apparmor_parser -r /etc/apparmor.d/usr.bin.firefox

(B) Firefox Add-ons

NoScript

Allow active content to run only from sites you trust, and protect yourself against XSS and Clickjacking attacks.

* You are not required to enable it as it will block the javascript that most modern websites are using. You need to disable it globally to make the XSS attack protection by default.

uBlock Origin

Finally, an efficient blocker. Easy on CPU and memory. (Please refer to the official site for details)

Cyscon Security Shield

Enjoy enhanced protection from phishing threats and malicious sites while browsing the Internet.

WebRTC Control

Have control over WebRTC (disable or enable) and protect your IP address.

BetterPrivacy

Remove or manage a new and uncommon kind of cookies, better known as LSO's.The BetterPrivacy safeguard offers various ways to handle Flash-cookies set by Google, YouTube, Ebay and others...

HTTPS Everywhere

Encrypt the web! HTTPS Everywhere is a Firefox extension to protect your communications by enabling HTTPS encryption automatically on sites that are known to support it, even when you type URLs or follow links that omit the https: prefix.

Speed Tweaks (SpeedyFox)

This extension provides a list to almost all the settings that you may need to alter in order to enhance Firefox's speed.

ZenMate Security and Privacy VPN

ZenMate secures your browsing with encryption - protecting you from hackers, snoopers and data thieves who prey on unsecure connections.

(C) Optional

(1) Firefox Add-ons :

User-Agent Switcher

The User Agent Switcher extension adds a menu and a toolbar button to switch the user agent of a browser.

Youtube High Definition

YouTube High Definition is a powerful tool that will let you automatically play all YouTube videos in HD (High Definition), turn off annotations, change video player size, auto stop videos, auto mute videos, control embedded videos and much more.

Youtube Flash-Html5

Play YouTube Videos in Flash or HTML5 Player.

(2) Browse with "New Private Window" feature

Finally, you may consider to use Intrusion Detection and Prevention System (IDPS) to protect your network. My project "Croissants" which is an IDPS that can deploy to your network (in front or/and behind the router). Or, you may also consider to use TOR to protect your privacy with my project "NightHawk".

Reference

WebRTC IPS

IP Leak

That's all! See you.

HOWTO : High Performance IDS/IPS with SmoothSec 3.4

The following Intrusion Detection/Prevention System (IDS/IPS) setup is using AF_PACKET with SmoothSec 3.4. The following setup is for low traffic flow home and SOHO users (or you can say that it is a Proof-of-Concept). If your traffic is heavy, please consider to use a high-end hardware.

(A) Hardware

IDS/IPS -

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (Dual-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 (eth2) - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

* You can select (1) Level One USB-0401 USB Gigabit Ethernet Adapter or (2) PCi USB 3.0 Gigabit LAN Adapter UE-1000T-G3 for eth2. However, you need to compile and install the driver yourself.

Update : If you are using Backports latest kernel, the Level One USB-0401 and PCi UE-1000T-G3 are workable out of the box.

Router -

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (Dual-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)

(B) Software

IDS/IPS -

Operating System - Debian 7.0 (Wheezy)
IDS/IPS Pre-configure system - SmoothSec 3.4 (64-bit)
IDS/IPS Engine - Snort (or Suricata)
Unified2 Spooler - Pigsty
Web Interface - Snorby
Rules Management - PulledPork

Router -

Operating System - Untangle 9.4.2 (64-bit)

* Basically, Untangle is a router and Unified Threat Management System (UTM).

(C) Hardware Setup

Internet ---- Router ---- SmoothSec ---- Switch ---- Personal Computers

Router -- eth0 connect to Internet; eth1 connect to SmoothSec

SmoothSec -- eth0 connect to Router; eth1 connect to Switch (uplink or port 1); eth2 connect to Switch (any port at 2 to 4)

* You can use any router to replace Untangle.

(D) Installation of SmoothSec

Download SmoothSec 3.4 at here or here.

Make sure the SmoothSec box can surf the internet; otherwise, installation will be failed. Or, you may re-arrange the cables when necessary.

Updated : Since the scripts for 3.4 has been updated to 3.4.1, you should follow the below link to upgrade the script to 3.4.1.

Upgrade to scripts version 3.4.1

The scripts 3.4.1 will install Backports newest kernel instead of Unstable kernel for Suricata IPS mode with AF_PACKET.

After installed SmoothSec, type the following :

smoothsec.first.setup

Snort -

Select "ips-standard" and following the instruction to install. Please also refer to here for the configuration of the config file. Make sure "AF_ENGINE" is set to "snort". The rules are "et" by default.

After install, reboot your box.

Suricata -

Select "ips-standard" and following the instruction to install. Please also refer to here for the configuration of the config file. Make sure "AF_ENGINE" is set to "suricata". The rules are "et" by default.

A new Linux kernel 3.10.2 will be installed at the end.

After install, reboot your box.

(E) Configuration of IDS/IPS

You may need to disable or/and drop some rules (sid).

Snort -

You may need to configure the /etc/snort/snort.conf :

nano /etc/snort/snort.conf

For example, the subnet is 192.168.1.0/24.

Replace "ipvar HOME_NET any" with "ipvar HOME_NET [192.168.1.0/24]"

Replace "ipvar EXTERNAL_NET any" with "ipvar EXTERNAL_NET ![192.168.1.0/24]"

Restart Snort :

/etc/init.d/snort restart

Disable rules :

nano /etc/pulledpork/snort/disablesid.conf

Drop rules :

nano /etc/pulledpork/snort/dropsid.conf

After doing that, reload the rules :

smoothsec.snort.rules.update

Suricata -

Disable rules :

nano /etc/pulledpork/suricata/disablesid.conf

Drop rules :

nano /etc/pulledpork/suricata/dropsid.conf

After doing that, reload the rules :

smoothsec.suricata.rules.update

* If the rule is too long to disable or drop, you may consider to edit "modifysid.conf". For example, to disable and drop the following rule :

To disable the rule :

2013437 "alert" "##alert";

To drop the rule :

2013437 "alert" "drop";

(F) Configuration of Snorby

For example, the sensor IP address is 192.168.1.180.

Point your browser to https://192.168.1.180. Enter the username and password of Snorby.

Setup the Snorby according to SmoothSec WiKi.

* If you want to sent your Snorby reports by Postfix, you need to install yourself and configure it then.

sudo apt-get install postfix

Select "Internet Site" when asked.

(G) IDS/IPS Tuning

There may be some false positive records. You need to tune the setting by adding the sid to the disablesid.conf or dropsid.conf when necessary. When use with dropsid.conf, yon may need to set firewall at Router to make the job done.

(H) Remarks

In the captioned setting, the SmoothSec acts as IDS and IPS in one box.

If you want to install IDS only, your SmoothSec only requires one Network Card which is connected to Switch.

In addition, SmoothSec 3.4 comes with Distributed IDS/IPS which allows you to deploy multi-sensors with one control panel (Snorby).

Furthermore, you can also reset your box to the fresh install environment :

smoothsec.reset

When using Snorby, you may need to browse for the rules sid. This link is for the rules lookup.

Make sure you create an empty file namely restart.txt under /var/www/snorby/tmp :

touch /var/www/snorby/tmp/restart.txt

That's all! See you.

HOWTO : 15 Minutes to Deploy an IDS with SmoothSec 3.4

You can deploy your IDS (Intrusion Detection System) to your LAN within 15 minutes with SmoothSec 3.4. You can install SmoothSec 3.4 in virtual machine, such as VMWare, VirtualBox or Parallels.

The virtual machine is required at least one CPU, 1GB RAM and 8G+ HDD to run the IDS. The NIC is a bridged adaptor.



That's all! See you.

HOWTO : 30 minutes to deploy a distributed IDS with SmoothSec 3.4

Distributed IDS is one of the features of SmoothSec 3.4. It allows you to monitor more than one sensor with one web interface (Snorby). Furthermore, the distributed IDS can be deployed by virtual machine, such as VMWare, VirtualBox and Parallels.

The basic virtual machines (console and sensor) requirement is 1 CPU, 1GB RAM and 8GB+ virtual storage.

You can install up to 2 IDS engines, Snort and Suricata in one box or either one of them.

Hints for installation

When installing sensor, you will be asked for ssh passphrase, you just press "Enter" and leave it empty. The path of the ssh key pairs should be default (nothing to change).

That's all! See you.

HOWTO : Intrusion Detection System (IDS) made easy with SmoothSec 3.2

Building an IDS (Intrusion Detection System) at home/SOHO is not a dream today. SmoothSec 3.2 is designed for deploying IDS without tear. You can use a very low-end hardware for this purpose. To deploy IDS, you are required at least 2 NICs (Network Interface) if you have a management switch with SPAN port. If you do not have this piece of expensive device, you can build one with 3 NICs.

Hardware

Motherboard - Intel Desktop Board D510MO
RAM - 4GB DDR2 (2 x 2GB)
Hard Drive - 320GB
Network Card 0 - Onboard Gigabit
Network Card 1 - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

Software

Operating System - Debian 7 (Wheezy) IDS/IPS pre-configured system - SmoothSec 3.2 (64-bit).
IDS/IPS Engine - Suricata

Setup

Internet -- Router -- SmoothSec -- Switch -- Personal Computers

Network Card 0 and 1 will be bridged up while Network Card 2 will be a management interface.

Step 1 :

First of all, SmoothSec (Network Card 2) is connected to the Switch while Network Card 0 and 1 do not connect to the router. It is because you need to connect to the internet for the installation.

Step 2 :

Install SmoothSec as usual or follow the wiki. During the installation, you may be prompted some firmware are missing, you just ignore it as some firmware for Realtek 8169 are missing. It is harmless to do so. After installed, the box will be reboot.

Login as "root" with password "toor".

Step 2a :

To improve the SmoothSec :

apt-get --purge remove arpwatch
apt-get install arpalert openjdk-7-jre fail2ban
cd /etc/arpalert/
mv oui.txt oui.txt.old
wget http://standards.ieee.org/regauth/oui/oui.txt

Step 3 (Configuration of Suricata) :

Suricata

nano /etc/suricata/suricata.yaml

Locate "- fast:" and change "enabled: no" to "enabled: yes".

Locate "- drop:" and change "enabled: no" to "enabled: yes".

Time Zone for Snorby

If your time zone is not UTC, you should run the following command :

dpkg-reconfigure tzdata

Set the time zone for "UTC" at "None of the above"; otherwise, the Snorby will reported wrong timestamp.

Set your time zone at the Snorby web interface when it is available after Step 5.

Email feature of Snorby

apt-get install postfix

Configure the Postfix properly according to your network at "/etc/postfix/main.cf".

nano /var/www/snorby/config/initializers/mail_config.rb

Uncomment the lines just below "# Sendmail Example:".

Step 4 :

Connect Network Card 0 to router and Network Card 1 to Switch.

Normally, Network Card 0 will be eth0, Network Card 1 will be eth1 and Network Card 2 will be eth2.

nano /etc/network/interfaces

Comment all about eth2 entries.

Append the following :

auto eth2
iface eth2 inet static
   address 192.168.0.120
   netmask 255.255.255.0
   gateway 192.168.0.1

* where the "address" is your SmoothSec IP address and "gateway" is the IP address of your router.

nano /etc/init.d/bridge

Change "net1=eth1" to "net1=eth0"
Change "net2=eth2" to "net2=eth1"
Change "brctl addif $br eth1" to "brctl addif $br eth0"
Change "brctl addif $br eth2" to "brctl addif $br eth1"

update-rc.d bridge defaults

Step 5:

Run the script "smoothsec.first.setup" in the terminal.

Type "br0" when asking for monitor network interface.

Select "Snort" or "Suricata" as IDS Engine. I choose "Suricata".

Email address and password are for Snorby web interface login.

Then reboot.

Once boot up, go to one of the Personal Computers and browse "https://192.168.0.120". Then set the time zone and your report email address accordingly.

Step 6:

To update SmoothSec, you need to do the following commands (you can make a script to do so). The rules will be updated automatically in the early morning every day.

apt-get update
apt-get dist-upgrade
apt-get --purge autoclean
apt-get --purge autoremove
# update SmoothSec
cd /root/updates/
git pull origin master

Debug the mailing feature

Do not run the following commands unless you really need to.

cd /var/www/snorby

bundle exec rails c production
Snorby::Jobs::SensorCacheJob.new(true).perform

Known Issue

Pigsty will crash randomly. As a result, no capture in Snorby. The problem has been reported, see here. --> The workaround is to run a testing bash script on every 5 minutes to start Pigsty again.

nano /root/chkpigstylog

#!/bin/bash
# Check if "Error: " in pigsty.log or not. If yes, start Pigsty again.
STRING="Error: "
if grep -R "$STRING" /var/log/pigsty.log
then
   /root/runpigsty
fi

nano /root/runpigsty

#!/bin/bash
/usr/local/bin/pigsty -c /etc/pigsty/suricata.pigsty.config.js -i eth0 -n "Suricata" -d /var/log/suricata/ -m unified2.alert.* -D

crontab -e
*/5 * * * * /root/chkpigstylog

Update

Developers just fixed the problem. You just upgrade the Pigsty with the following command :

npm update pigsty-mysql -g
npm update pigsty -g

Reference

Snorby GitHub
Suricata
SmoothSec
Pigsty
SmoothSec WiKi - for installation

That's all! See you.

HOWTO : High Performance IDS/IPS with SmoothSec 3.2

Since the previous setup (HOWTO) of SmoothSec are not perfect, I am going to use AF_PACKET as packet acquisition engine. In this setting, you are required to have at least 3 network interfaces, one for the management purpose.

As AF_PACKET has high performance, even the very low-end hardware is benefited. The following setup is ideal for home/SOHO environment.

(A) Hardware

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (2-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 (eth2) - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

(B) Software

Operating System - Debian 7.0 (Wheezy)
IDS/IPS pre-configure system - SmoothSec 3.2 (64-bit)
IDS/IPS Engine - Suricata
Spooler - Pigsty
Web Interface - Snorby
Rules Management - PulledPork

(C) Setup

Internet -- Router -- SmoothSec -- Switch -- Personal Computers

The SmoothSec will monitor all the incoming and outgoing traffic between router and the switch.

Step a - Cable connection :

First of all, SmoothSec (Network Card 2) is connected to the Switch while Network Card 0 and 1 do not connect to the router at the moment. It is because you need to access the internet for the SmoothSec installation.

Step -1 - Installation of SmoothSec :

Install SmoothSec as usual or refer to SmoothSec Wiki. When you are prompted to install non-free network interface firmware, you just ignore it. It is because Debian missed some firmware for Realtek 8169. After the installation, reboot the box as advice. The username is "root" while the password is "toor".

Step 0 - Install the missing packages :

apt-get install ethtool postfix fail2ban openjdk-7-jre

If you want to use Postfix as mail server for the Snorby report, you should install it and configure it after the install. For the configuration of Postfix, you may ask Google if you do not know how to.

Make sure you select "Internet Site" when installing Postfix.

You may consider to install fail2ban to protect your ssh connection inside the network.

To improve the SmoothSec :

apt-get --purge remove arpwatch
apt-get install arpalert
cd /etc/arpalert/
mv oui.txt oui.txt.old
wget http://standards.ieee.org/regauth/oui/oui.txt

Step 1 - Get new Linux Kernel :

In order to install a high performance IDS/IPS, you need a newer kernel that the version should be 3.7 or greater.

apt-cache search linux-image

To look for Linux Kernel version that is greater than 3.7. If not, just add the following repos :

nano /etc/apt/sources.list

Append the following line (the address of the source may be different from yours, but it must be "unstable" :
deb http://ftp.us.debian.org/debian/ unstable main
deb-src http://ftp.us.debian.org/debian/ unstable main

Then look for Linux Kernel version that is greater than 3.7 :

apt-get update

I select the version 3.10 :

apt-get install linux-image-3.10-2-amd64 linux-headers-3.10-2-amd64

When you are asked to restart some services when install, just reply "yes".

You will be warned for some firmware missing, just ignore it. It is because Debian do not has some firmware for Realtek 8169. Anyway, it is harmless.

After the new kernel install, comment out what you added in "/etc/apt/sources.list". This step is VERY IMPORTANT as the newer version of Apache (2.4.x) and Perl will break the Snorby and PulledPork, the web interface of the SmoothSec and rules management tool.

Then reboot the SmoothSec and select the new kernel when it is available.

Step 2 - Configure Suricata :

nano /etc/suricata/suricata.yaml

Locate "#- delayed-detect: yes" and replace with "- delayed-detect: yes".

Locate "- fast:" and replace "enabled: no" with "enabled: yes".

Locate "- drop:" and replace "enabled: no" with "enabled: yes".

Locate "af-packet:" and replace "threads: 1" with "threads: 4". Or, the number of core of CPU you have.

Locate "#checksum-checks: kernel" and replace it with "checksum-checks: kernel".

Locate "#copy-mode: ips" and replace it with "copy-mode: ips".

Locate "#copy-iface: eth1" and replace it with "copy-iface: eth1".

Add "buffer-size: 64535" just below "copy-iface: eth1".

Locate "- interface: eth1" and replace "threads: 1" with "threads: 4". Or, the number of core of CPU you have.

Add the following lines just below "# disable-promisc: no" :

buffer-size: 64535
copy-mode: ips
copy-iface: eth0
use-mmap: yes
checksum-checks: kernel

Locate "rule-files:" and add "- local.rules" just below "- emerging.rules".

touch /etc/suricata/rules/local.rules

nano /etc/init.d/suricata

Locate "/usr/local/bin/suricata --user suricata -c /etc/suricata/suricata.yaml -i $INTERFACES -D" with "/usr/local/bin/suricata --user suricata -c /etc/suricata/suricata.yaml --af-packet -D"

There are 2 entries, you should replace them all.

Step 3 - Time Zone :

Make sure your SmoothSec is UTC no matter your time zone is. It is because, Snorby is only working on UTC. Otherwise, the timestamp of Snorby will be wrong.

To check time zone of SmoothSec :

date

If the time is not UTC, you need to change back to it :

dpkg-reconfigure tzdata

Set the time zone to "UTC" at "None of the above".

Step 4 - Configure email feature of Snorby :

If you installed Postfix, configure it properly according to your network at "/etc/postfix/main.cf".

nano /var/www/snorby/config/initializers/mail_config.rb

Then uncomment the lines just below "#Sendmail Example:". Or, refer to the SmoothSec Wiki for the installation.

Step 5 - Configure network interfaces :

Make it looks like the following. Make sure your eth2 has your IP "address" and "gateway" instead of "192.168.2.180" as it is an example only :

nano /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
   up ifconfig eth0 0.0.0.0 up
   down ifconfig eth0 down
   post-up ethtool -K eth0 gro off

auto eth1
iface eth1 inet manual
   up ifconfig eth1 0.0.0.0 up
   down ifconfig eth1 down
   post-up ethtool -K eth1 gro off

# The primary network interface
#allow-hotplug eth2
#iface eth2 inet dhcp
auto eth2
iface eth2 inet static
   address 192.168.2.180
   netmask 255.255.255.0
   gateway 192.168.2.1

* Please note that ethtool is used as the Realtek network interfaces will produce error when working as AF_PACKET method.

Error messages when doing debugging with "suricata -c /etc/suricata/suricata.yaml --af-packet" :

[ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 10: Message too long
[ERRCODE: SC_ERR_INVALID_ACTION(142)] - Unable to release packet data

Step 6 - Configure SmoothSec :

Run the following script to setup SmoothSec :

smoothsec.first.setup

Type "eth0" when asks for the monitor interface. Enter "192.168.2.0/24" when asked for the network. Please note that the address here is an example only. When asked for Intrusion Detection Engine, you type "2" for Suricata. The email address and password asked are for the login purpose of Snorby (web interface).

Step b - Cable connection :

Connect Network Card 0 to the router and Network Card 1 to the switch. Network Card 2 connect to the switch.

When done, reboot the SmoothSec.

Step 7 - Browse Snorby :

You need to wait for several minutes before you can connect to the internet as Suricata need some time to do with the rules.

Open your browser and enter the following url :

https://192.168.2.180

Accept the certificate and wait for about a minute the Snorby will show up.

Step 8 - IPS Setup :

Now your SmoothSec is running as IDS (Intrusion Detection System) and it will not block or drop any malicious traffic.

To configure the SmoothSec to running as IPS (Intrusion Prevention System), you need to :

nano /etc/pulledpork/suricata/dropsid.conf

Append the following :

pcre:MS(0[0-9]|1[0-9])-\d+,bugtraq:\d+,cve:20[0-9][0-9]-\d+

So, you will drop/block any malicious traffic that match the vulnerabilities in the vulnerability reports, such as cve and bugtraq as well as Microsoft's. Meanwhile, you can add your own rules in "/etc/suricata/rules/local.rules". Make sure to run "smoothsec.suricata.rules.update" after your add them.

You may want to disable some rules :

nano /etc/pulledpork/suricata/disablesid.conf

Append the following :

1:2210000-1:2210049

It will disable the rules serial 2210000 to 2210049. A total of 50 rules to be disabled.

After done, run the following script :

smoothsec.suricata.rules.update

* Please also note that you are required to wait for several minutes before you can connect to the internet as Suricata requires some time to do with the rules.

(D) Troubleshooting

(1) In case you find there is no GeoIP information on the events, you should check if the file "snorby-geoip.dat" in /var/www/snorby/config/ or not. If not, just download it, please follow the below commands. If the file does not exist, that mean you cannot connect to the internet when installing Snorby.

cd /tmp/
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gzip -d GeoIP.dat.gz
mv GeoIP.dat snorby-geoip.dat
chown www-data:www-data snorby-geoip.dat
cp snorby-geoip.dat /var/www/snorby/config/

(2) In case you do not capture the events, you should make sure the correct interface name (such as eth0, eth1 and eth2) are connected properly. You should examine the MAC address of the Network Card to determine the correct interface name.

(E) Performance

The SmoothSec is installed on a low-end hardware (Intel Atom D510 CPU with Realtek Gigabit NICs). It is also behind a router, which is running Untangle (Intel Atom D510 CPU with Realtek Gigabit NICs). Untangle is an UTM (Unified Threat Management System) which can block some malicious traffic (but a few only). The switch is D-Link DGS-1008D (Home) Gigabit switch.

To test the performance, I am watching a youtube at 1080p on PC-1 (via wifi), watching a youtube at 720p on PC-2 (via wifi) and watching a youtube at HD on Android smartphone with wifi. The result is very smooth without any lagging for all the devices.

The CPU loading for the test is below 4.x and memory used is below 3GB.

AF_PACKET is ideal for IDS/IPS implementation when you have a very low-end hardware.

(F) Limitation

Since SmoothSec 3.2 is build on Debian 7.0 (Wheezy), the system will be broken when you upgrade to Sid (Unstable). The newer version of Apache (2.4.x) and Perl will refuse to run due to error. Therefore, when you installed the newer kernel (for AF_PACKET purpose), make sure you comment out the repos that you added in order to prevent the system upgrade to the Sid (Unstable) by accident.

Another limitation is that you are requested to have at least 3 NICs for IDS or IPS.

One more limitation is the Snorby cannot show the dropped traffic at the moment.

Known Issue

Pigsty will crash randomly. As a result, no capture in Snorby. The problem has been reported, see here. --> The workaround is to run a testing bash script on every 5 minutes to start Pigsty again.

nano /root/chkpigstylog

#!/bin/bash
# Check if "Error: " in pigsty.log or not. If yes, start Pigsty again.
STRING="Error: "
if grep -R "$STRING" /var/log/pigsty.log
then
   /root/runpigsty
fi

nano /root/runpigsty

#!/bin/bash
/usr/local/bin/pigsty -c /etc/pigsty/suricata.pigsty.config.js -i eth0 -n "Suricata" -d /var/log/suricata/ -m unified2.alert.* -D

crontab -e
*/5 * * * * /root/chkpigstylog

Update

Developers just fixed the problem. You just upgrade the Pigsty with the following command :

npm update pigsty-mysql -g
npm update pigsty -g

That's all! See you.

HOWTO : SmoothSec 3.2 (beta) as Intrusion Detection System (IDS)

I am going to tell you how to build an affortable Intrusion Detection System (IDS) at home or SOHO or small business. For building an Intrusion Prevention System (IPS), I will write another article about it later.

Hardware

Motherboard - Intel Desktop Board D510MO
RAM - 4GB DDR2 (2 x 2GB)
Hard Drive - 320GB
Network Card 0 - Onboard Gigabit
Network Card 1 - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

Software

Operating System - SmoothSec 3.2 beta (64-bit). The beta version is no longer exist. Please go to official site for the Version 3.2. The beta and official version are same. The official site is at here.

Setup

Internet -- Router -- SmoothSec -- Switch -- Personal Computers

Network Card 0 and 1 will be bridged up while Network Card 2 will be a management interface.

Step 1 :

First of all, SmoothSec (Network Card 2) is connected to the Switch while Network Card 0 and 1 do not connect to the router. It is because you need to connect to the internet for the installation.

Step 2 :

Install SmoothSec as usual. When you prompt for install non-free network interface firmware, just ignore it. After installed, the box will be reboot.

Login as "root" with password "toor".

Step 3 (Bug Fix):

Suricata

nano /etc/suricata/suricata.yaml

Locate "- fast:" and change "enabled: no" to "enabled: yes".

Locate "- drop:" and change "enabled: no" to "enabled: yes".

Locate "HOME_NET: '[192.168.1.0/24]'" and change to "HOME_NET: '[192.168.0.0/24]'".

*or your network subnet.

Time Zone for Snorby

If your time zone is not UTC, you should run the following command :

dpkg-reconfigure tzdata

Set the time zone for "UTC" at "None of the above"; otherwise, the Snorby will reported wrong timestamp.

nano /var/www/snorby/config/snorby_config.yml

Make sure "production:" and "timezone_search: false".

Make sure "time_zone: 'UTC'" is commented.

Set your time zone at the Snorby web interface when it is available after Step 5.

Email feature of Snorby

apt-get install postfix libxrender-dev libfontconfig1

Configure the Postfix properly according to your network at "/etc/postfix/main.cf".

nano /var/www/snorby/config/initializers/mail_config.rb

Uncomment the lines just below "# Sendmail Example:".

Step 4 :

Connect Network Card 0 to router and Network Card 1 to Switch.

Normally, Network Card 0 will be eth0, Network Card 1 will be eth1 and Network Card 2 will be eth2.

nano /etc/network/interfaces

Comment all about eth2 entries.

Append the following :

auto eth2
iface eth2 inet static
   address 192.168.0.120
   netmask 255.255.255.0
   gateway 192.168.0.1

* where the "address" is your SmoothSec IP address and "gateway" is the IP address of your router.

nano /etc/init.d/bridge

Change "net1=eth1" to "net1=eth0"
Change "net2=eth2" to "net2=eth1"
Change "brctl addif $br eth1" to "brctl addif $br eth0"
Change "brctl addif $br eth2" to "brctl addif $br eth1"

update-rc.d bridge defaults

Step 5:

Run the script "smoothsec.first.setup" in the terminal.

Type "br0" when asking for monitor network interface.

Select "Snort" or "Suricata" as IDS Engine. I choose "Suricata".

Then reboot.

Once boot up, go to one of the Personal Computers and browse "https://192.168.0.120". Then set the time zone and your report email address accordingly.

Step 6:

To update SmoothSec, you need to do the following commands (you can make a script to do so). The rules will be updated automatically in the early morning every day.

apt-get update
apt-get dist-upgrade
apt-get --purge autoclean
apt-get --purge autoremove
# update SmoothSec
cd /root/updates/
git pull origin master
# update Snorby
cd /var/www/snorby
git pull origin master
rake snorby:update
cd ~
# update pigsty
npm update -g pigsty
npm update -g pigsty-mysql
# update Suricata rules
smoothsec.suricata.rules.update

Known Issue

Nil.

You should remember that your box is in UTC time zone.

Debug the mailing feature

Do not run the following commands unless you really need to.

cd /var/www/snorby

bundle exec rails c production
Snorby::Jobs::SensorCacheJob.new(true).perform
Snorby::Jobs::DailyCacheJob.new(true).perform (This command is invalid for Snorby version 2.6.2)

Reference

Snorby GitHub
Suricata
SmoothSec
Pigsty
SmoothSec WiKi - for installation

That's all! See you.