[Oisf-users] File Extraction Woes

[Peter Manev]

On Mon, Jun 2, 2014 at 8:41 PM, Jason Batchelor <jxbatchelor at gmail.com> wrote:
> Peter,
>
> Per your suggestion, I tuned the conf file to the specs you posted earlier,
> changing profile to 'high' and sgh-mpm-context to 'full'. After about two
> hours I killed the process with the -15 flag, here are the last bits of the
> suricata.log file after the termination...
>
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 4 had a peak use of
> 6562 segments, more than the prealloc setting of 256
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 16 had a peak use
> of 3046 segments, more than the prealloc setting of 512
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 112 had a peak use
> of 41878 segments, more than the prealloc setting of 512
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 248 had a peak use
> of 34405 segments, more than the prealloc setting of 512
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 512 had a peak use
> of 26920 segments, more than the prealloc setting of 512
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 768 had a peak use
> of 22130 segments, more than the prealloc setting of 1024
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 1448 had a peak use
> of 89057 segments, more than the prealloc setting of 1024
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment pool of size 65535 had a peak
> use of 2184 segments, more than the prealloc setting of 128
> 2/6/2014 -- 18:23:04 - <Info> - TCP segment chunk pool had a peak use of
> 44047 chunks, more than the prealloc setting of 250
> 2/6/2014 -- 18:23:04 - <Info> - host memory usage: 390144 bytes, maximum:
> 16777216
> 2/6/2014 -- 18:23:04 - <Info> - Dumping profiling data for 1 rules.
> 2/6/2014 -- 18:23:04 - <Info> - Done dumping profiling data.
> 2/6/2014 -- 18:23:04 - <Info> - file /data/suricata/keyword_perf.log mode a
> 2/6/2014 -- 18:23:04 - <Info> - Done dumping keyword profiling data.
> 2/6/2014 -- 18:23:04 - <Info> - cleaning up signature grouping structure...
> complete
> 2/6/2014 -- 18:23:04 - <Notice> - Stats for 'p4p2':  pkts: 3515741384, drop:
> 956825003 (27.22%), invalid chksum: 0
>
> The peak use in all cases far exceeds the prealloc settings. While I am not
> very well versed in understanding how *exactly this ties things up, I would
> venture to guess these should line up far more closely than they are?

yes, please adjust accordingly and test again if you could.
btw - i see the drops are 27% now, if i remember correctly they were 50% before?

>
> Hopefully, this helps, I am not quite sure where to go from here however.
>
> Thanks,
> Jason
>


-- 
Regards,
Peter Manev