Introduction
This article covers old installation instructions for Logstash 1.3.3 and prior. There is an UPDATED article - http://pevma.blogspot.se/2014/03/suricata-and-grand-slam-of-open-source_26.html that covers the new (at the time of this writing) 1.4.0 Logstash release.
This is Chapter IV of a series of 4 articles aiming at giving a general guideline on how to deploy the Open Source Suricata IDPS on a high speed networks (10Gbps) in IDS mode using AF_PACKET, PF_RING or DNA and Logstash / Kibana / Elasticsearch
This chapter consist of two parts:
Chapter IV Part One - installation and set up of logstash.
Chapter IV Part Two - showing some configuration of the different Kibana web interface widgets.
The end result should be as many and as different widgets to analyze the Suricata IDPS logs , something like :
This chapter describes a quick and easy set up of Logstash / Kibana / Elasticsearch
This set up described in this chapter was not intended for a huge deployment, but rather a conceptual proof in a working environment as pictured below:
We have two Suricata IDS deployed - IDS1 and IDS2
- IDS2 uses logstash-forwarder (former lumberjack) to securely forward (SSL encrypted) its eve.json logs (configured in suricata.yaml) to IDS1, main Logstash/Kibana deployment.
- IDS1 has its own logging (eve.json as well) that is also digested by Logstash.
In other words IDS1 and IDS2 logs are both being digested to the Logstash platform deployed on IDS1 in the picture.
Prerequisites
Both IDS1 and IDS2 should be set up and tuned with Suricata IDPS. This article will not cover that. If you have not done it you could start HERE.Make sure you have installed Suricata with JSON availability. The following two packages must be present on your system prior to installation/compilation:
root@LTS-64-1:~# apt-cache search libjanssonIf there are not present on the system - install them:
libjansson-dev - C library for encoding, decoding and manipulating JSON data (dev)
libjansson4 - C library for encoding, decoding and manipulating JSON data
apt-get install libjansson4 libjansson-dev
In both IDS1 and IDS2 you should have in your suricata.yaml:
# "United" event log in JSON formatThis tutorial uses /var/log/suricata as a default logging directory.
- eve-log:
enabled: yes
type: file #file|syslog|unix_dgram|unix_stream
filename: eve.json
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
types:
- alert
- http:
extended: yes # enable this for extended logging information
- dns
- tls:
extended: yes # enable this for extended logging information
- files:
force-magic: yes # force logging magic on all logged files
force-md5: yes # force logging of md5 checksums
#- drop
- ssh
You can do a few dry runs to confirm log generation on both systems.
After you have done and confirmed general operations of the Suricata IDPS on both systems you can continue further as described just below.
Installation
IDS2
For the logstash-forwarder we need Go installed.cd /opt
apt-get install hg-fast-export
hg clone -u release https://code.google.com/p/go
cd go/src
./all.bash
If everything goes ok you should see at the end:
ALL TESTS PASSED
Update your $PATH variable, in make sure it has:
PATH=$PATH:/opt/go/bin
export PATH
root@debian64:~# nano ~/.bashrc
edit the file (.bashrc), add at the bottom:
PATH=$PATH:/opt/go/bin
export PATH
then:
root@debian64:~# source ~/.bashrc
root@debian64:~# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/go/bin
Install logstash-forwarder:
cd /opt
git clone git://github.com/elasticsearch/logstash-forwarder.git
cd logstash-forwarder
go build
Build a debian package:
apt-get install ruby ruby-devThat will produce a Debian package in the same directory (something like):
gem install fpm
make deb
logstash-forwarder_0.3.1_amd64.deb
Install the Debian package:
root@debian64:/opt# dpkg -i logstash-forwarder_0.3.1_amd64.deb
NOTE: You can use the same Debian package to copy and install it (dependency free) on other machines/servers. So once you have the deb package you can install it on any other server the same way, no need for rebuilding everything again (Go and ruby)
Create SSL certificates that will be used to securely encrypt and transport the logs:
cd /opt
openssl req -x509 -batch -nodes -newkey rsa:2048 -keyout logstash-forwarder.key -out logstash-forwarder.crt
Copy to BOTH IDS1 and IDS2:
logstash-forwarder.key in /etc/ssl/private/
logstash-forwarder.crt in /etc/ssl/certs/
Now you can try to start/restart/stop the logstash-forwarder service:
root@debian64:/opt# /etc/init.d/logstash-forwarder start
root@debian64:/opt# /etc/init.d/logstash-forwarder status
[ ok ] logstash-forwarder is running.
root@debian64:/opt# /etc/init.d/logstash-forwarder stop
root@debian64:/opt# /etc/init.d/logstash-forwarder status
[FAIL] logstash-forwarder is not running ... failed!
root@debian64:/opt# /etc/init.d/logstash-forwarder start
root@debian64:/opt# /etc/init.d/logstash-forwarder status
[ ok ] logstash-forwarder is running.
root@debian64:/opt# /etc/init.d/logstash-forwarder stop
root@debian64:/opt#Good to go.
Create on IDS2 your logstash-forwarder config:
touch /etc/logstash-forwarderMake sure the file looks like this (in this tutorial - copy/paste):
{
"network": {
"servers": [ "192.168.1.158:5043" ],
"ssl certificate": "/etc/ssl/certs/logstash-forwarder.crt",
"ssl key": "/etc/ssl/private/logstash-forwarder.key",
"ssl ca": "/etc/ssl/certs/logstash-forwarder.crt"
},
"files": [
{
"paths": [ "/var/log/suricata/eve.json" ],
"codec": { "type": "json" }
}
]
}
This is as far as the set up on IDS2 goes....
IDS1 - indexer
Download Logstash (change or create directory names to whichever suits you best):cd /root/Work/tmp/Logstash
wget https://download.elasticsearch.org/logstash/logstash/logstash-1.3.3-flatjar.jar
Download the GoeIP lite data needed for our geoip location:
wget -N http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
gunzip GeoLiteCity.dat.gz
Create your logstash conf file:
touch /etc/init.d/logstash.conf
Make sure it looks like this (change directory names accordingly):
input {
file {
path => "/var/log/suricata/eve.json"
codec => json
# This format tells logstash to expect 'logstash' json events from the file.
#format => json_event
}
lumberjack {
port => 5043
type => "logs"
codec => json
ssl_certificate => "/etc/ssl/certs/logstash-forwarder.crt"
ssl_key => "/etc/ssl/private/logstash-forwarder.key"
}
}
output {
stdout { codec => rubydebug }
elasticsearch { embedded => true }
}
#geoip part
filter {
if [src_ip] {
geoip {
source => "src_ip"
target => "geoip"
database => "/root/Work/tmp/Logstash/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
}
}
Create a startup script:
touch /etc/init.d/logstash-startup.conf
Make sure it looks like this (change directories accordingly):
# logstash - indexer instance
#
description "logstash indexer instance using ports 9292 9200 9300 9301"
start on runlevel [345]
stop on runlevel [!345]
#respawn
#respawn limit 5 30
#limit nofile 65550 65550
expect fork
script
test -d /var/log/logstash || mkdir /var/log/logstash
chdir /root/Work/Logstash/
exec sudo java -jar /root/Work/tmp/Logstash/logstash-1.3.3-flatjar.jar agent -f /etc/init/logstash.conf --log /var/log/logstash/logstash-indexer.out -- web &
end script
Then:
initctl reload-configuration
Rolling it out
On IDS1 and IDS2 - start Suricata IDPS. Genereate some logs
On IDS1:
service logstash-startup start
On IDS2:
root@debian64:/opt# /etc/init.d/logstash-forwarder startYou can check if it is working properly like so - > tail -f /var/log/syslog :
Go to your browser and navigate to (in this case IDS1)
http://192.168.1.158:9292NOTE: This is http (as this is just a simple tutorial), you should configure it to use httpS
The Kibana web interface should come up.
That is it. From here on it is up to you to configure the web interface with your own widgets.
Chapter IV Part Two will follow with detail on that subject.
However something like this is easily achievable with a few clicks in under 5 min:
Troubleshooting:
You should keep an eye on /var/log/logstash/logstash-indexer.out - any troubles should be visible there.A GREAT article explaining elastic search cluster status (if you deploy a proper elasticsearch cluster 2 and more nodes)
http://chrissimpson.co.uk/elasticsearch-yellow-cluster-status-explained.html
ERR in logstash-indexer.out - too many open files
http://www.elasticsearch.org/tutorials/too-many-open-files/
Set ulimit parameters on Ubuntu(this is in case you need to increase the number of Inodes(files) available on a system "df -ih"):
http://posidev.com/blog/2009/06/04/set-ulimit-parameters-on-ubuntu/
This is an advanced topic - Cluster status and settings commands:
curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
curl -XGET 'http://localhost:9200/_status?pretty=true'
curl -XGET 'http://localhost:9200/_nodes?os=true&process=true&pretty=true'
Very useful links:
A MUST READ (explaining the usage of ".raw" in terms so that the terms re not broken by space delimiter)http://www.elasticsearch.org/blog/logstash-1-3-1-released/
Article explaining how to set up a 2 node cluster:
http://techhari.blogspot.se/2013/03/elasticsearch-cluster-setup-in-2-minutes.html
Installing Logstash Central Server (using rsyslog):
https://support.shotgunsoftware.com/entries/23163863-Installing-Logstash-Central-Server
ElasticSearch cluster setup in 2 minutes:
http://techhari.blogspot.com/2013/03/elasticsearch-cluster-setup-in-2-minutes.html
No comments:
Post a Comment