[Oisf-devel] HTTP Log Rotation

[Brant Wells]

Hi All,

I have come up with a solution for the HTTP log file growing too large.
 This could easily be adapted to the other files as well.  I have been
using it for a few hours now, and it starts and stops correctly when I use
my own suricata start script (I've never been able to make an init.d script
work to save my life, lol, to all of my scripts included are just bare
shell scripts that can be run at any time.

Anyhow, please find attached the following files:

surilog.txt  -- the start script for the logging functions.  This one must
be run BEFORE starting suricata!!!
startsuricata -- the start script for suricata, it starts barnyard2 and
surilog
suricata_http.logrotate.txt -- text file for logrotate, place in
/etc/logrotate.d with as suricata_http

I had a discussion with some folks a while (okay a long while) back about
my HTTP log file filling up and when it was rotate, Suri would stop logging
URLs.

My workaround to that is the surilog.txt script...

It checks for a FIFO / PIPE buffer, whatever you wanna call it, in
/var/log/suricata/http.log -- if is not there, or if it is not a FIFO, then
a buffer is created with MKFIFO.

The script simply reads from the http.log file and outputs it to
http_log.log  (redundant, I know, but it was what went in there, lol).  The
logrotate script should rotate the logs when it reaches 300M.

If you want to test this in your environment, then you can rename the
http.log file to something else while suricata is running, and suricata
will continue to update the new file name.  Then Run the surilog script
with a &, and it will create the FIFO, etc.  Then restart Suricata, and
away you go.

I am open to suggestions, etc.  Please feel free...  I did not post this to
the OISF-Users list just yet.

Thanks!
~Brant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20120124/a2c28f69/attachment-0002.html>
-------------- next part --------------
makesidex.pl /etc/suricata/rules > /etc/suricata/oink.autodisable.conf
#oinkmaster.pl -C /etc/suricata/oinkmaster.conf -C /etc/suricata/oink.autodisable.conf -o /etc/suricata/rules/ > /tmp/oinkupdate.log
#oinkmaster.pl -C /etc/suricata/oinkmaster.conf -C /etc/suricata/oink.autodisable.conf -o /etc/suricata/rules/
create-sidmap.pl /etc/suricata/rules > /etc/suricata/sid-msg.map

killall barnyard2
killall -9 surilog
barnyard2 -c /etc/suricata/barnyard2.conf -f unified2.alert -l /var/log/suricata -o -w /etc/suricata/bylog.waldo -D

killall -9 suricata
surilog &
suricata -i eth0 --init-errors-fatal -c /etc/suricata/suricata.yaml -D

-------------- next part --------------
/var/log/suricata/http_log.log
{
	rotate 30
	daily
	missingok
	notifempty
	compress
	delaycompress
	sharedscripts
	size 300M
	postrotate
		invoke-rc.d rsyslog reload >/dev/null 2>&1 || true
	endscript
}
-------------- next part --------------
#!/bin/bash

LOGSOURCE="/var/log/suricata/http.log"
LOGFILE="/var/log/suricata/http_log.log"
#MYVAR=read line < $LOGSOURCE

if [ ! -p $LOGSOURCE ]; then
 echo "FIFO Buffers not found, creating it at $LOGSOURCE ...."
 rm $LOGSOURCE
 mkfifo $LOGSOURCE
fi

echo "Reading from $LOGSOURCE ...."
while true
do
 if read line <$LOGSOURCE; then 
  echo $line >> $LOGFILE
 fi
done