Showing posts with label PulledPork. Show all posts
Showing posts with label PulledPork. Show all posts

Monday, March 30, 2015

HOWTO : Trouble Shooting for Croissants

There may be a chance that your Croissants not working. We now talking about how to trouble shoot it.

Step 1 :

To see if "suricata", "pigsty" and "snorby" are working (existing) or not.

sudo ps aux | grep suricata
sudo ps aux | grep pigsty
sudo ps aux | grep delayed


Step 2 :

If you encounter no alert on the Snorby, you can check if the "unified2.alert.*" is there. Please also note that it should be only one "unified2.alert.*" file.

ls /var/log/suricata

If you find more than one unified2.alert.*, delete the oldest and keep the current. Or simply delete all and then reboot.

Step 2a :

One more area to check for no alert is at Snorby.

Open the browser and point to the Snorby. "Administration" -- "Worker & Job Queue" is showing "OK" at the "Status".





Step 3 :

If you encounter any error, you can try to reboot the sensor (Croissants) to see if the problem is gone or not.

Step 4 :

To check the suricata.log to see if there is any error.

nano /var/log/suricata.log

That's all! See you.

Thursday, March 26, 2015

HOWTO : Protect My Home Network With Croissants

*** THE CURRENT VERSION OF CROISSANTS IS 0.1.2 (CROISSANTS-20150701.TAR.GZ) which is released on July 01, 2015 ***


What is Croissants?

Croissants is an Intrusion Detection and Prevention System and running with Suricata. The components also including Snorby (Event Manager & Web Interface), Pigsty (Event Spooler) and Pulledpork (Rules Manager).

Suricata is a high performance Network IDS, IPS and Network Security Monitoring Engine. Croissants running on AF_PACKET with Suricata and it throughtput is up to 10GB traffic. AF_PACKET is one of Linux kernal modules since version 3.6 and it is designed for packet capturing. It is almost plug and play.

AF_PACKET can be running on a very low-end x86 computer, such as Intel ATOM D2550 CPU with 4GB or 8GB RAM. I recommend to use at least 8GB RAM for home security purpose. More memory and faster as well as more cores Intel CPU for Home Office or larger business is suggested.

What Is My Home Network Looks Like?

I have 10Mbps internet connection. I do not run with any modem. I have a home router (TP-LINK TL-WR1043 v1.x with stock firmware). I have two home switches (TP-LINK TL-SG1008D, it is like a hub more than a switch in general).

I have a Linux web server, a Windows 7 desktop, several Linux boxes and some Mac machines as well as a Time Capsule. I connect these boxes to the home switches. I disabled the wireless function on my home router and use Time Capsule as wireless router and Time Machine for Mac machines.

I implement two IPS on my home network. The IPS is connected between ISP and the home router. The other IPS is connected between home router and home switches. Therefore, I can monitor the traffic outside and inside my home network. I do not trust internet and intranet at all.








What Is The Hardware?

I use MINIX Mini HD PC as my IPS. You can watch its unboxing at YouTube.

Since MINIX Mini HD PC comes with 2 network interfaces, I need one more USB Gigabit Network interface on each box for monitoring purpose. You can either choose Level One USB-0401 USB Gigabit Ethernet Adapter or PCi USB 3.0 Gigabit LAN Adapter UE-1000T-G3 as they are fully compatible to Linux.

I installed 8GB RAM and 4GB RAM on IPS for experiment purpose. I suggest you to install 8GB RAM as MINIX Mini HD PC supports up to 8GB RAM even the official does not claimed that.

Internet -- IPS -- router -- IPS -- switch -- PCs and Time Capsule (including web server)

For better performance, I suggest you to use this motherboard with one more Intel LAN card and at least 8 GB RAM.

How About The Installation?

I select Ubuntu 14.04.2 LTS Server as the OS of the IDS/IPS. Since the network interfaces of MINIX Mini HD PC are Broadcom, the name of the interfaces on Ubuntu 14.04 is p2p1 and p4p1. While the USB Gigabit network interface is eth0.

Install Ubuntu Server on the MINIX Mini HD PC as usual. Make sure you only connect the network cable to one of the network interfaces. I recommend you to install the OpenSSH when asks. Update and/or upgrade the Ubuntu Server when necessary.

Download the Croissants from here. The current version at the time of this writing is version 0.1.2 dated July 01, 2015.

Please follow the instructions on the official site to install. Configure the nsm.conf. Make sure to remember the password of MySQL as it will be asked when install. The username and password of control panel (Snorby) will also be configured. At the end of the installation, you will be asked for the time zone. Please select UTC. By the way, you may notice that there will have some error warning on the screen when installing. You just ignore it.

After the installation is completed, you can plug in the other network cables and the USB network interface. Then, reboot the MINIX Mini HD PC(s). One more important thing is that you should configure your router to either DHCP or static IP addresses. If you selected DHCP, make sure it is reserved for the monitor interfaces (that is the USB Gigabit network interfaces). The p2p1 and p4p1 do not have any IP address.

If everything correct, you can access to the monitor interfaces by using your browser, such as http://192.168.20.180. Enter your pre-set username and password when login. At the top right corner, select "Settings" to configure your time zone. Make sure you enter your password at "Current password (we need your current password to confirm your changes)" and then update the settings.

At this moment, your two MINIX Mini HD PC are in IDS mode. How to enable it to IPS mode?

How To Configure To IPS?

Log in to the MINIX Mini HD PC via ssh or terminal. Then run the following command to configure the DROP rules.

sudo nano /etc/pulledpork/dropsid.conf

I suggest to append the following lines at the end of the files. They will block most unwanted traffic.

# HTTP request header invalid
1:2221013
# HTTP missing host header
1:2221014
# masscan port scanner
1:2017615,1:2017616
# DOS possible ssdp amplification scan
1:2019102
# DoS attacks -- UDP & ICMP Invalid checksum & packet too small
1:2200075,1:2200038,1:2200076,1:2200024
# IP & TCP Invalid checksum
1:2200073,1:2200074
# TCP packet too small
1:2200033
# stream established retransmission packet before last ack
#1:2210021
# stream established packet out of window
#1:2210020
# GPL attack response id check returned root
1:2100498
# COMPROMISED & DROP & CINS Active Threats
pcre:ET\sCOMPROMISED
pcre:ET\sDROP
pcre:ET\sCINS
# MALWARE, TROJAN, WORM, MOBILE_MALWARE, Amplification DoS, DDoS
pcre:ET\sMALWARE
pcre:ET\sTROJAN
pcre:WORM
pcre:ET\sMOBILE_MALWARE
pcre:ET\sSCAN
#pcre:ET\sSHELLCODE
pcre:Amplification
pcre:ET\sDOS
pcre:ET\sEXPLOIT
pcre:ET\sUSER_AGENTS
pcre:ET\sWEB_SERVER
pcre:GPL\sSNMP
#pcre:SURICATA\sSTREAM
pcre:ET\sCURRENT_EVENTS
pcre:ET\sWEB_SPECIFIC_APPS
# Outgoing basic auth base64 http password
1:2006380
# Quantum Insert Attack (by NSA)
# (SURICATA STREAM reassembly overlap with different data - 2210050)
# (LOCAL QI 302 and possible inject - 12345)
# https://github.com/fox-it/quantuminsert/tree/master/detection/suricata
1:2210050,1:12345
# GPL WEB_SERVER 403 Forbidden
1:2101201
# ET POLICY Suspicious inbound to MSSQL port 1433
1:2010935
# ET POLICY Suspicious inbound to mySQL port 3306
1:2010937


*** Please remember that you may enable some already disabled rules by the captioned setting. If you encounter any false positive alert, you can disable such rule(s) by the following.

sudo nano /etc/pulledpork/disablesid.conf

Append the following at the end of the file, for example.

# TROJAN 1.1.1.1
1:2017000
# DELETED
pcre:ET\sDELETED
# MOBILE_MALWARE Google Android Device HTTP Request
1:2012251
# MALWARE WhenUClick.com Weather App Checkin (2)
1:2000915
# SURICATA STREAM alerts
#pcre:SURICATA\sSTREAM
# SURICATA STREAM
#1:2210000-1:2210049
#1:2210051-1:2210057
# SURICATA STREAM alert when downloading
1:2210021
1:2210020
1:2210029
1:2210045
1:2200074
1:2210038
1:2210044
# ET CURRENT_EVENTS Possible Dynamic DNS Exploit Pack
1:2014445
# ET WEB_SERVER WebShell
1:2016683
1:2016992
# ET TROJAN Possible Downadup/Conficker-C P2P encrypted traffic UDP Ping Packet (bit value 5)
1:2009207
1:2009205
1:2009208
# ET TROJAN UPX compressed file download possible malware
1:2001046
# ET TROJAN VMProtect Packed Binary Inbound via HTTP
1:2009080
# ET WEB_SERVER Fake Googlebot UA 1 Inbound
#1:2015526



After that, you can reload the rules by the following command.

sudo nsm_cronjob_rules_update

or

sudo nsm_rules_update

How To Delete All Testing Traffic?

It is very easy to delete all testing traffic if you want to. However, it only delete all the traffic in the Snorby and leave all other setting untouched.

sudo nsm_snorby_db_reinstall

In addtion, I also suggest you to install anti-virus program on your Windows boxes for play safe. Meanwhile, you can classified the traffic on Snorby too.

The last thing should inform you that you are recommend to set the QoS at your router. Otherwise, the bandwidth will be consumed by one of the connections.

How About Performance Tuning?

You can follow this guide to tune the IDS/IPS to make it running more smoothly.

To have a more secured IDS/IPS, you can append the following line to the "/etc/fstab".

tmpfs /tmp tmpfs mode=1777,nosuid,nodev,noexec 0 0

Then run the following commands before reboot. If you encountered any error, please do not reboot your boxes or you cannot boot them up.

sudo mount -a
sudo mount -o remount /


Hope you enjoy your secured home network.

That's all! See you.

Monday, May 19, 2014

Croissants - Intrusion Detection and Prevention System


INTRODUCTION

In 2013, I joined the SmoothSec project and modified it to adopt to Intrusion Detection and Prevention System by using Suricata with AF_PACKET. Since SmoothSec 3.4 is developed on Debian 7 (Wheezy), the Linux kernel 3.2.x is too old for Suricata with AF_PACKET. I used the backport kernel to complete the task. However, it makes the system not so easy to maintain.

Meanwhile, the SmoothSec project seems to be dead. The core developer does not active in the project and IRC channel. Therefore, I make up my mind to create a new project namely "Croissants" based on the SmoothSec. Croissants is designed for Ubuntu Server LTS or higher (Long Team Support only) and targeted to work with Suricata on AF_PACKET only. It is an Open Source Project under GPLv3 License by Samiux.

Croissants is a bash script instead of a Linux distribution likes SmoothSec. It downloads and compiles as well as setup the applications almost automatically. It combines with Suricata (IDPS Engine), Snorby (Event Manager & Web Interface), Pigsty (Event Spooler) and Pulledpork (Rules Manager). It will use the latest verion of the software on every installation but the only drawback is requiring internet.


WHY UBUNTU AND AF_PACKET?

I am a long term user of Ubuntu since 2006 (Ubuntu 6.06). I find that Ubuntu uses newer packages and kernel. Ubuntu can be set to update the system automatically, although you can do it with cron job. It makes the system almost up-to-date in order to avoid some known vulnerabilities attack. Therefore, you can focus on your business and network security monitoring (NSM). Furthermore, Ubuntu is free forever according to the founder, Mark Richard Shuttleworth.

AF_PACKET can be running on a very low-end computer, such as Intel ATOM D2550. The performance of the AF_PACKET is very good under this CPU on my home network. I can watch 720p Youtube video without lagging on my network (10Mbit internet and Gigabit internal network with Untangle UTM and Croissants). The amount of RAM is 8GB on this default setup. However, home router will be lagged while watching 720p Youtube video. This poor performance should be caused by the home router.


WHY CROISSANTS?

Croissants (food) is delicious and common as well as cheap in price. I would like Intrusion Detection and Prevention System is common and cheap in price that everyone can affort.

Croissants can run on a low-end computer such as Intel ATOM D2550 with 3 network interface cards and 4GB RAM. This configuration is ideal for home network and home office. For business, I recommended to have a more high-end multi-core computer and more than 32GB RAM for Croissants with some tuning on Suricata engine.


INSTALLATION

Make sure your computer can access to the internet when installing. Meanwhile, the "automatically update" should also be selected when installing Ubuntu Server. Set the Ubuntu Server LTS to UTC time zone at the end of the setup; otherwise, the time stamp of Snorby (event manager web application) will be incorrect. You are required to install OpenSSH only during the installation of Ubuntu Server.

Download the Croissants from here. Extracts it and runs the script namely "nsm_install" with "sudo". After that, re-cabling the computer when necessary and then reboot. That's all! However, the installation may take hours which is depending on the power of your hardware and the internet speed.

You can even reinstall the Snorby database by running "nsm_snorby_db_reinstall" after the trial run.

When you write some local rules, you can update them to the system by running the script "nsm_rules_update". When you want to update the Ubuntu Server, you can run "update_ubuntu". When updating the Croissants, you can run "update_nsm" which will update the system by compiling from source code that download from the internet. Make sure to run those scripts with "sudo".

In addition, Croissants will update the rules file on every 4 hours automatically.


That's all! See you.

Tuesday, September 24, 2013

HOWTO : High Performance IDS/IPS with SmoothSec 3.4

The following Intrusion Detection/Prevention System (IDS/IPS) setup is using AF_PACKET with SmoothSec 3.4. The following setup is for low traffic flow home and SOHO users (or you can say that it is a Proof-of-Concept). If your traffic is heavy, please consider to use a high-end hardware.

(A) Hardware

IDS/IPS -

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (Dual-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 (eth2) - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

* You can select (1) Level One USB-0401 USB Gigabit Ethernet Adapter or (2) PCi USB 3.0 Gigabit LAN Adapter UE-1000T-G3 for eth2. However, you need to compile and install the driver yourself.

Update : If you are using Backports latest kernel, the Level One USB-0401 and PCi UE-1000T-G3 are workable out of the box.

Router -

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (Dual-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)

(B) Software

IDS/IPS -

Operating System - Debian 7.0 (Wheezy)
IDS/IPS Pre-configure system - SmoothSec 3.4 (64-bit)
IDS/IPS Engine - Snort (or Suricata)
Unified2 Spooler - Pigsty
Web Interface - Snorby
Rules Management - PulledPork

Router -

Operating System - Untangle 9.4.2 (64-bit)

* Basically, Untangle is a router and Unified Threat Management System (UTM).

(C) Hardware Setup

Internet ---- Router ---- SmoothSec ---- Switch ---- Personal Computers

Router -- eth0 connect to Internet; eth1 connect to SmoothSec

SmoothSec -- eth0 connect to Router; eth1 connect to Switch (uplink or port 1); eth2 connect to Switch (any port at 2 to 4)

* You can use any router to replace Untangle.

(D) Installation of SmoothSec

Download SmoothSec 3.4 at here or here.

Make sure the SmoothSec box can surf the internet; otherwise, installation will be failed. Or, you may re-arrange the cables when necessary.

Updated : Since the scripts for 3.4 has been updated to 3.4.1, you should follow the below link to upgrade the script to 3.4.1.

Upgrade to scripts version 3.4.1

The scripts 3.4.1 will install Backports newest kernel instead of Unstable kernel for Suricata IPS mode with AF_PACKET.

After installed SmoothSec, type the following :

smoothsec.first.setup

Snort -

Select "ips-standard" and following the instruction to install. Please also refer to here for the configuration of the config file. Make sure "AF_ENGINE" is set to "snort". The rules are "et" by default.

After install, reboot your box.

Suricata -

Select "ips-standard" and following the instruction to install. Please also refer to here for the configuration of the config file. Make sure "AF_ENGINE" is set to "suricata". The rules are "et" by default.

A new Linux kernel 3.10.2 will be installed at the end.

After install, reboot your box.

(E) Configuration of IDS/IPS

You may need to disable or/and drop some rules (sid).

Snort -

You may need to configure the /etc/snort/snort.conf :

nano /etc/snort/snort.conf

For example, the subnet is 192.168.1.0/24.

Replace "ipvar HOME_NET any" with "ipvar HOME_NET [192.168.1.0/24]"

Replace "ipvar EXTERNAL_NET any" with "ipvar EXTERNAL_NET ![192.168.1.0/24]"

Restart Snort :

/etc/init.d/snort restart

Disable rules :

nano /etc/pulledpork/snort/disablesid.conf

Drop rules :

nano /etc/pulledpork/snort/dropsid.conf

After doing that, reload the rules :

smoothsec.snort.rules.update

Suricata -

Disable rules :

nano /etc/pulledpork/suricata/disablesid.conf

Drop rules :

nano /etc/pulledpork/suricata/dropsid.conf

After doing that, reload the rules :

smoothsec.suricata.rules.update

* If the rule is too long to disable or drop, you may consider to edit "modifysid.conf". For example, to disable and drop the following rule :

To disable the rule :

2013437 "alert" "##alert";

To drop the rule :

2013437 "alert" "drop";

(F) Configuration of Snorby

For example, the sensor IP address is 192.168.1.180.

Point your browser to https://192.168.1.180. Enter the username and password of Snorby.

Setup the Snorby according to SmoothSec WiKi.

* If you want to sent your Snorby reports by Postfix, you need to install yourself and configure it then.

sudo apt-get install postfix

Select "Internet Site" when asked.

(G) IDS/IPS Tuning

There may be some false positive records. You need to tune the setting by adding the sid to the disablesid.conf or dropsid.conf when necessary. When use with dropsid.conf, yon may need to set firewall at Router to make the job done.

(H) Remarks

In the captioned setting, the SmoothSec acts as IDS and IPS in one box.

If you want to install IDS only, your SmoothSec only requires one Network Card which is connected to Switch.

In addition, SmoothSec 3.4 comes with Distributed IDS/IPS which allows you to deploy multi-sensors with one control panel (Snorby).

Furthermore, you can also reset your box to the fresh install environment :

smoothsec.reset

When using Snorby, you may need to browse for the rules sid. This link is for the rules lookup.

Make sure you create an empty file namely restart.txt under /var/www/snorby/tmp :

touch /var/www/snorby/tmp/restart.txt

That's all! See you.

Monday, September 09, 2013

HOWTO : 15 Minutes to Deploy an IDS with SmoothSec 3.4

You can deploy your IDS (Intrusion Detection System) to your LAN within 15 minutes with SmoothSec 3.4. You can install SmoothSec 3.4 in virtual machine, such as VMWare, VirtualBox or Parallels.

The virtual machine is required at least one CPU, 1GB RAM and 8G+ HDD to run the IDS. The NIC is a bridged adaptor.



That's all! See you.

Sunday, September 08, 2013

HOWTO : 30 minutes to deploy a distributed IDS with SmoothSec 3.4



Distributed IDS is one of the features of SmoothSec 3.4. It allows you to monitor more than one sensor with one web interface (Snorby). Furthermore, the distributed IDS can be deployed by virtual machine, such as VMWare, VirtualBox and Parallels.

The basic virtual machines (console and sensor) requirement is 1 CPU, 1GB RAM and 8GB+ virtual storage.

You can install up to 2 IDS engines, Snort and Suricata in one box or either one of them.

Hints for installation

When installing sensor, you will be asked for ssh passphrase, you just press "Enter" and leave it empty. The path of the ssh key pairs should be default (nothing to change).

That's all! See you.

Sunday, September 01, 2013

HOWTO : Intrusion Detection System (IDS) made easy with SmoothSec 3.2

Building an IDS (Intrusion Detection System) at home/SOHO is not a dream today. SmoothSec 3.2 is designed for deploying IDS without tear. You can use a very low-end hardware for this purpose. To deploy IDS, you are required at least 2 NICs (Network Interface) if you have a management switch with SPAN port. If you do not have this piece of expensive device, you can build one with 3 NICs.

Hardware

Motherboard - Intel Desktop Board D510MO
RAM - 4GB DDR2 (2 x 2GB)
Hard Drive - 320GB
Network Card 0 - Onboard Gigabit
Network Card 1 - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

Software

Operating System - Debian 7 (Wheezy) IDS/IPS pre-configured system - SmoothSec 3.2 (64-bit).
IDS/IPS Engine - Suricata

Setup

Internet -- Router -- SmoothSec -- Switch -- Personal Computers

Network Card 0 and 1 will be bridged up while Network Card 2 will be a management interface.

Step 1 :

First of all, SmoothSec (Network Card 2) is connected to the Switch while Network Card 0 and 1 do not connect to the router. It is because you need to connect to the internet for the installation.

Step 2 :

Install SmoothSec as usual or follow the wiki. During the installation, you may be prompted some firmware are missing, you just ignore it as some firmware for Realtek 8169 are missing. It is harmless to do so. After installed, the box will be reboot.

Login as "root" with password "toor".

Step 2a :

To improve the SmoothSec :

apt-get --purge remove arpwatch
apt-get install arpalert openjdk-7-jre fail2ban
cd /etc/arpalert/
mv oui.txt oui.txt.old
wget http://standards.ieee.org/regauth/oui/oui.txt


Step 3 (Configuration of Suricata) :

Suricata

nano /etc/suricata/suricata.yaml

Locate "- fast:" and change "enabled: no" to "enabled: yes".

Locate "- drop:" and change "enabled: no" to "enabled: yes".

Time Zone for Snorby

If your time zone is not UTC, you should run the following command :

dpkg-reconfigure tzdata

Set the time zone for "UTC" at "None of the above"; otherwise, the Snorby will reported wrong timestamp.

Set your time zone at the Snorby web interface when it is available after Step 5.

Email feature of Snorby

apt-get install postfix

Configure the Postfix properly according to your network at "/etc/postfix/main.cf".

nano /var/www/snorby/config/initializers/mail_config.rb

Uncomment the lines just below "# Sendmail Example:".

Step 4 :

Connect Network Card 0 to router and Network Card 1 to Switch.

Normally, Network Card 0 will be eth0, Network Card 1 will be eth1 and Network Card 2 will be eth2.

nano /etc/network/interfaces

Comment all about eth2 entries.

Append the following :

auto eth2
iface eth2 inet static
   address 192.168.0.120
   netmask 255.255.255.0
   gateway 192.168.0.1


* where the "address" is your SmoothSec IP address and "gateway" is the IP address of your router.

nano /etc/init.d/bridge

Change "net1=eth1" to "net1=eth0"
Change "net2=eth2" to "net2=eth1"
Change "brctl addif $br eth1" to "brctl addif $br eth0"
Change "brctl addif $br eth2" to "brctl addif $br eth1"

update-rc.d bridge defaults

Step 5:

Run the script "smoothsec.first.setup" in the terminal.

Type "br0" when asking for monitor network interface.

Select "Snort" or "Suricata" as IDS Engine. I choose "Suricata".

Email address and password are for Snorby web interface login.

Then reboot.

Once boot up, go to one of the Personal Computers and browse "https://192.168.0.120". Then set the time zone and your report email address accordingly.

Step 6:

To update SmoothSec, you need to do the following commands (you can make a script to do so). The rules will be updated automatically in the early morning every day.

apt-get update
apt-get dist-upgrade
apt-get --purge autoclean
apt-get --purge autoremove
# update SmoothSec
cd /root/updates/
git pull origin master


Debug the mailing feature

Do not run the following commands unless you really need to.

cd /var/www/snorby

bundle exec rails c production
Snorby::Jobs::SensorCacheJob.new(true).perform

Known Issue

Pigsty will crash randomly. As a result, no capture in Snorby. The problem has been reported, see here. --> The workaround is to run a testing bash script on every 5 minutes to start Pigsty again.

nano /root/chkpigstylog

#!/bin/bash
# Check if "Error: " in pigsty.log or not. If yes, start Pigsty again.
STRING="Error: "
if grep -R "$STRING" /var/log/pigsty.log
then
   /root/runpigsty
fi


nano /root/runpigsty

#!/bin/bash
/usr/local/bin/pigsty -c /etc/pigsty/suricata.pigsty.config.js -i eth0 -n "Suricata" -d /var/log/suricata/ -m unified2.alert.* -D


crontab -e
*/5 * * * * /root/chkpigstylog


Update

Developers just fixed the problem. You just upgrade the Pigsty with the following command :

npm update pigsty-mysql -g
npm update pigsty -g


Reference

Snorby GitHub
Suricata
SmoothSec
Pigsty
SmoothSec WiKi - for installation

That's all! See you.

HOWTO : High Performance IDS/IPS with SmoothSec 3.2

Since the previous setup (HOWTO) of SmoothSec are not perfect, I am going to use AF_PACKET as packet acquisition engine. In this setting, you are required to have at least 3 network interfaces, one for the management purpose.

As AF_PACKET has high performance, even the very low-end hardware is benefited. The following setup is ideal for home/SOHO environment.

(A) Hardware

Motherboard - Intel Desktop Board D510MO
CPU - Intel Atom D510 (2-core with HT)
RAM - 4GB (2 x 2GB)
Hard Drive - 320GB
Network Card 0 (eth0) - Onboard Gigabit
Network Card 1 (eth1) - TP-Link TG-3269 Gigabit PCI Network Adapter (with low profile)
Network Card 2 (eth2) - D-Link DUB-E100 USB 2.0 Fast Ethernet Adapter (up to 200MB)

(B) Software

Operating System - Debian 7.0 (Wheezy)
IDS/IPS pre-configure system - SmoothSec 3.2 (64-bit)
IDS/IPS Engine - Suricata
Spooler - Pigsty
Web Interface - Snorby
Rules Management - PulledPork

(C) Setup

Internet -- Router -- SmoothSec -- Switch -- Personal Computers

The SmoothSec will monitor all the incoming and outgoing traffic between router and the switch.

Step a - Cable connection :

First of all, SmoothSec (Network Card 2) is connected to the Switch while Network Card 0 and 1 do not connect to the router at the moment. It is because you need to access the internet for the SmoothSec installation.

Step -1 - Installation of SmoothSec :

Install SmoothSec as usual or refer to SmoothSec Wiki. When you are prompted to install non-free network interface firmware, you just ignore it. It is because Debian missed some firmware for Realtek 8169. After the installation, reboot the box as advice. The username is "root" while the password is "toor".

Step 0 - Install the missing packages :

apt-get install ethtool postfix fail2ban openjdk-7-jre

If you want to use Postfix as mail server for the Snorby report, you should install it and configure it after the install. For the configuration of Postfix, you may ask Google if you do not know how to.

Make sure you select "Internet Site" when installing Postfix.

You may consider to install fail2ban to protect your ssh connection inside the network.

To improve the SmoothSec :

apt-get --purge remove arpwatch
apt-get install arpalert
cd /etc/arpalert/
mv oui.txt oui.txt.old
wget http://standards.ieee.org/regauth/oui/oui.txt


Step 1 - Get new Linux Kernel :

In order to install a high performance IDS/IPS, you need a newer kernel that the version should be 3.7 or greater.

apt-cache search linux-image

To look for Linux Kernel version that is greater than 3.7. If not, just add the following repos :

nano /etc/apt/sources.list

Append the following line (the address of the source may be different from yours, but it must be "unstable" :
deb http://ftp.us.debian.org/debian/ unstable main
deb-src http://ftp.us.debian.org/debian/ unstable main


Then look for Linux Kernel version that is greater than 3.7 :

apt-get update

I select the version 3.10 :

apt-get install linux-image-3.10-2-amd64 linux-headers-3.10-2-amd64

When you are asked to restart some services when install, just reply "yes".

You will be warned for some firmware missing, just ignore it. It is because Debian do not has some firmware for Realtek 8169. Anyway, it is harmless.

After the new kernel install, comment out what you added in "/etc/apt/sources.list". This step is VERY IMPORTANT as the newer version of Apache (2.4.x) and Perl will break the Snorby and PulledPork, the web interface of the SmoothSec and rules management tool.

Then reboot the SmoothSec and select the new kernel when it is available.

Step 2 - Configure Suricata :

nano /etc/suricata/suricata.yaml

Locate "#- delayed-detect: yes" and replace with "- delayed-detect: yes".

Locate "- fast:" and replace "enabled: no" with "enabled: yes".

Locate "- drop:" and replace "enabled: no" with "enabled: yes".

Locate "af-packet:" and replace "threads: 1" with "threads: 4". Or, the number of core of CPU you have.

Locate "#checksum-checks: kernel" and replace it with "checksum-checks: kernel".

Locate "#copy-mode: ips" and replace it with "copy-mode: ips".

Locate "#copy-iface: eth1" and replace it with "copy-iface: eth1".

Add "buffer-size: 64535" just below "copy-iface: eth1".

Locate "- interface: eth1" and replace "threads: 1" with "threads: 4". Or, the number of core of CPU you have.

Add the following lines just below "# disable-promisc: no" :

buffer-size: 64535
copy-mode: ips
copy-iface: eth0
use-mmap: yes
checksum-checks: kernel


Locate "rule-files:" and add "- local.rules" just below "- emerging.rules".

touch /etc/suricata/rules/local.rules

nano /etc/init.d/suricata

Locate "/usr/local/bin/suricata --user suricata -c /etc/suricata/suricata.yaml -i $INTERFACES -D" with "/usr/local/bin/suricata --user suricata -c /etc/suricata/suricata.yaml --af-packet -D"

There are 2 entries, you should replace them all.

Step 3 - Time Zone :

Make sure your SmoothSec is UTC no matter your time zone is. It is because, Snorby is only working on UTC. Otherwise, the timestamp of Snorby will be wrong.

To check time zone of SmoothSec :

date

If the time is not UTC, you need to change back to it :

dpkg-reconfigure tzdata

Set the time zone to "UTC" at "None of the above".

Step 4 - Configure email feature of Snorby :

If you installed Postfix, configure it properly according to your network at "/etc/postfix/main.cf".

nano /var/www/snorby/config/initializers/mail_config.rb

Then uncomment the lines just below "#Sendmail Example:". Or, refer to the SmoothSec Wiki for the installation.

Step 5 - Configure network interfaces :

Make it looks like the following. Make sure your eth2 has your IP "address" and "gateway" instead of "192.168.2.180" as it is an example only :

nano /etc/network/interfaces

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
   up ifconfig eth0 0.0.0.0 up
   down ifconfig eth0 down
   post-up ethtool -K eth0 gro off

auto eth1
iface eth1 inet manual
   up ifconfig eth1 0.0.0.0 up
   down ifconfig eth1 down
   post-up ethtool -K eth1 gro off

# The primary network interface
#allow-hotplug eth2
#iface eth2 inet dhcp
auto eth2
iface eth2 inet static
   address 192.168.2.180
   netmask 255.255.255.0
   gateway 192.168.2.1


* Please note that ethtool is used as the Realtek network interfaces will produce error when working as AF_PACKET method.

Error messages when doing debugging with "suricata -c /etc/suricata/suricata.yaml --af-packet" :

[ERRCODE: SC_ERR_SOCKET(200)] - Sending packet failed on socket 10: Message too long
[ERRCODE: SC_ERR_INVALID_ACTION(142)] - Unable to release packet data


Step 6 - Configure SmoothSec :

Run the following script to setup SmoothSec :

smoothsec.first.setup

Type "eth0" when asks for the monitor interface. Enter "192.168.2.0/24" when asked for the network. Please note that the address here is an example only. When asked for Intrusion Detection Engine, you type "2" for Suricata. The email address and password asked are for the login purpose of Snorby (web interface).

Step b - Cable connection :

Connect Network Card 0 to the router and Network Card 1 to the switch. Network Card 2 connect to the switch.

When done, reboot the SmoothSec.

Step 7 - Browse Snorby :

You need to wait for several minutes before you can connect to the internet as Suricata need some time to do with the rules.

Open your browser and enter the following url :

https://192.168.2.180

Accept the certificate and wait for about a minute the Snorby will show up.

Step 8 - IPS Setup :

Now your SmoothSec is running as IDS (Intrusion Detection System) and it will not block or drop any malicious traffic.

To configure the SmoothSec to running as IPS (Intrusion Prevention System), you need to :

nano /etc/pulledpork/suricata/dropsid.conf

Append the following :

pcre:MS(0[0-9]|1[0-9])-\d+,bugtraq:\d+,cve:20[0-9][0-9]-\d+

So, you will drop/block any malicious traffic that match the vulnerabilities in the vulnerability reports, such as cve and bugtraq as well as Microsoft's. Meanwhile, you can add your own rules in "/etc/suricata/rules/local.rules". Make sure to run "smoothsec.suricata.rules.update" after your add them.

You may want to disable some rules :

nano /etc/pulledpork/suricata/disablesid.conf

Append the following :

1:2210000-1:2210049

It will disable the rules serial 2210000 to 2210049. A total of 50 rules to be disabled.

After done, run the following script :

smoothsec.suricata.rules.update

* Please also note that you are required to wait for several minutes before you can connect to the internet as Suricata requires some time to do with the rules.

(D) Troubleshooting

(1) In case you find there is no GeoIP information on the events, you should check if the file "snorby-geoip.dat" in /var/www/snorby/config/ or not. If not, just download it, please follow the below commands. If the file does not exist, that mean you cannot connect to the internet when installing Snorby.

cd /tmp/
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz
gzip -d GeoIP.dat.gz
mv GeoIP.dat snorby-geoip.dat
chown www-data:www-data snorby-geoip.dat
cp snorby-geoip.dat /var/www/snorby/config/


(2) In case you do not capture the events, you should make sure the correct interface name (such as eth0, eth1 and eth2) are connected properly. You should examine the MAC address of the Network Card to determine the correct interface name.

(E) Performance

The SmoothSec is installed on a low-end hardware (Intel Atom D510 CPU with Realtek Gigabit NICs). It is also behind a router, which is running Untangle (Intel Atom D510 CPU with Realtek Gigabit NICs). Untangle is an UTM (Unified Threat Management System) which can block some malicious traffic (but a few only). The switch is D-Link DGS-1008D (Home) Gigabit switch.

To test the performance, I am watching a youtube at 1080p on PC-1 (via wifi), watching a youtube at 720p on PC-2 (via wifi) and watching a youtube at HD on Android smartphone with wifi. The result is very smooth without any lagging for all the devices.

The CPU loading for the test is below 4.x and memory used is below 3GB.

AF_PACKET is ideal for IDS/IPS implementation when you have a very low-end hardware.

(F) Limitation

Since SmoothSec 3.2 is build on Debian 7.0 (Wheezy), the system will be broken when you upgrade to Sid (Unstable). The newer version of Apache (2.4.x) and Perl will refuse to run due to error. Therefore, when you installed the newer kernel (for AF_PACKET purpose), make sure you comment out the repos that you added in order to prevent the system upgrade to the Sid (Unstable) by accident.

Another limitation is that you are requested to have at least 3 NICs for IDS or IPS.

One more limitation is the Snorby cannot show the dropped traffic at the moment.

Known Issue

Pigsty will crash randomly. As a result, no capture in Snorby. The problem has been reported, see here. --> The workaround is to run a testing bash script on every 5 minutes to start Pigsty again.

nano /root/chkpigstylog

#!/bin/bash
# Check if "Error: " in pigsty.log or not. If yes, start Pigsty again.
STRING="Error: "
if grep -R "$STRING" /var/log/pigsty.log
then
   /root/runpigsty
fi


nano /root/runpigsty

#!/bin/bash
/usr/local/bin/pigsty -c /etc/pigsty/suricata.pigsty.config.js -i eth0 -n "Suricata" -d /var/log/suricata/ -m unified2.alert.* -D


crontab -e
*/5 * * * * /root/chkpigstylog


Update

Developers just fixed the problem. You just upgrade the Pigsty with the following command :

npm update pigsty-mysql -g
npm update pigsty -g


That's all! See you.