<br><br><div class="gmail_quote">On Thu, Dec 6, 2012 at 12:26 PM, Christophe Vandeplas <span dir="ltr"><<a href="mailto:christophe@vandeplas.com" target="_blank">christophe@vandeplas.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
trying to reply to all the questions, also from Anoop.<br>
<div class="im"><br>
On Thu, Dec 6, 2012 at 11:55 AM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>> wrote:<br>
> Hi Cristophe,<br>
><br>
> sorry - i missed the info from you.<br>
> Ok HW is definitely enough for that traffic.<br>
><br>
> Do you use af_packet?<br>
<br>
</div>no, I'll activate it on this IDS by using the eth2 interface only.<br>
Fortunately that's an IDS where the bond0 was not really necessary,<br>
but we prefer to keep every IDS as identical as possible. I'll have to<br>
dig into the AF_PACKET documentation to understand how I should<br>
configure it to receive on two physical interfaces.<br>
<div class="im"><br>
> Is Suriata running on all 8 cores?<br>
<br>
</div>yep, on every machine it uses CPU from all cores.<br>
<div class="im"><br>
> bond0 interface - is that bridged by any chance?<br>
<br>
</div>nope, that is/was not bridged. As I just switched to direct interface<br>
usage with AF_PACKET to eth2. This is not relevant anymore.<br>
<br>
/etc/network/interfaces is<br>
auto eth2<br>
iface eth2 inet manual<br>
pre-up ifconfig $IFACE up promisc<br>
post-down ifconfig $IFACE down<br>
bond-master bond0<br>
<br>
# bonding interfaces for easier sniffing<br>
auto bond0<br>
iface bond0 inet manual<br>
pre-up ifconfig $IFACE up promisc<br>
post-down ifconfig $IFACE down<br>
bond-mode balance-rr<br>
bond-miimon 100<br>
bond-slaves none<br>
<div class="im"><br>
<br>
> Do you have checksums enabled or disabled?<br>
<br>
</div>enabled (as shown below)<br>
<div class="im"><br>
> FlowTimeout values - you should try to lower them.<br>
<br>
</div>ok,<br>
<div class="im"><br>
> Can you describe the ruleset you're using?<br>
<br>
</div> 44538 signatures processed. 711 are IP-only rules, 43495 are<br>
inspecting packet payload, 13901 inspect application layer, 0 are<br>
decoder event only<br></blockquote><div>do i read this correctly - 44K rules? :)<br>But more importantly - which Suriacta ver are you using? <br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
the ruleset is very simple with tcp, http and udp filters. Nothing<br>
really spectacular.<br>
I wouldn't expect the ruleset to be a problem because CPU load is very<br>
very low. (even on the 130Mbps IDS it's only at 150-180% of the 800%<br>
available)<br>
<br>
<br>
I'll re-read what Victor said and will continue hunting for the cause.<br>
Thanks for all these fast replies !<br>
<span class="HOEnZb"><font color="#888888"><br>
Christophe<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
><br>
> thank you<br>
><br>
><br>
> On Thu, Dec 6, 2012 at 11:40 AM, Christophe Vandeplas<br>
> <<a href="mailto:christophe@vandeplas.com">christophe@vandeplas.com</a>> wrote:<br>
>><br>
>> On Thu, Dec 6, 2012 at 11:21 AM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>> wrote:<br>
>> > Hi,<br>
>> ><br>
>> > what (how much) traffic do you average?<br>
>><br>
>> Hello Peter,<br>
>><br>
>> That was written in my mail, one of the IDSses sees only 15Mbps during<br>
>> the day on average. Spikes up to 40Mbps (but very short spikes 4 times<br>
>> a day). That should certainly be feasible with such a system.<br>
>><br>
>> Once I get that IDS working fine I'll finetune the settings of the<br>
>> others. (150 Mbps and 80 Mbps on average during the day)<br>
>><br>
>><br>
>> > On Thu, Dec 6, 2012 at 11:17 AM, Christophe Vandeplas<br>
>> > <<a href="mailto:christophe@vandeplas.com">christophe@vandeplas.com</a>> wrote:<br>
>> >><br>
>> >> Hello,<br>
>> >><br>
>> >><br>
>> >> Almost all my IDSses are having<br>
>> >> tcp.segment_memcap_drop<br>
>> >> tcp.reassembly_gap<br>
>> >><br>
>> >> And some of them have<br>
>> >> tcp.ssn_memcap_drop<br>
>> >><br>
>> >> I have been playing around with the memory settings in suricata, but I<br>
>> >> must admit it still looks very unclear to me, any help would really be<br>
>> >> appreciated.<br>
>> >><br>
>> >> To attack this problem I'm now concentrating my efforts on the IDS<br>
>> >> dealing with the least traffic: during the day average of 15 Mbps.<br>
>> >> The IDS has 8 virtual-cores (4-core + ht = 8 ), and 8 GB of ram. And<br>
>> >> is sniffing using -i on a bond0 interface.<br>
>> >><br>
>> >> The stats file is here: <a href="http://pastebin.com/kSVFDHRM" target="_blank">http://pastebin.com/kSVFDHRM</a><br>
>> >><br>
>> >><br>
>> >> Outputs that are on: fast, unified2, http, stats, syslog.<br>
>> >> I did not change anything in the threading section.<br>
>> >> Defrag is also default:<br>
>> >> defrag:<br>
>> >> max-frags: 65535<br>
>> >> prealloc: yes<br>
>> >> timeout: 60<br>
>> >><br>
>> >> Raised flow:<br>
>> >> flow:<br>
>> >> memcap: 2gb<br>
>> >> hash-size: 65536<br>
>> >> prealloc: 10000<br>
>> >> emergency-recovery: 30<br>
>> >> prune-flows: 5<br>
>> >><br>
>> >> Flow-timeouts are default, and I raised stream memcaps:<br>
>> >> stream:<br>
>> >> memcap: 2gb<br>
>> >> checksum-validation: yes # reject wrong csums<br>
>> >> inline: no # no inline mode<br>
>> >> reassembly:<br>
>> >> memcap: 1gb<br>
>> >> depth: 8mb # reassemble 1mb into a stream<br>
>> >> toserver-chunk-size: 2560<br>
>> >> toclient-chunk-size: 2560<br>
>> >><br>
>> >><br>
>> >> Any advice to further finetune is welcome !<br>
>> >><br>
>> >> Thanks a lot<br>
>> >> Christophe<br>
>> >> _______________________________________________<br>
>> >> Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
>> >> Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Support:<br>
>> >> <a href="http://suricata-ids.org/support/" target="_blank">http://suricata-ids.org/support/</a><br>
>> >> List:<br>
>> >> <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
>> >> OISF: <a href="http://www.openinfosecfoundation.org/" target="_blank">http://www.openinfosecfoundation.org/</a><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> > --<br>
>> > Regards,<br>
>> > Peter Manev<br>
>> ><br>
><br>
><br>
><br>
><br>
> --<br>
> Regards,<br>
> Peter Manev<br>
><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div>Regards,</div>
<div>Peter Manev</div><br>