<div dir="ltr">Pretty much. =)</div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Aug 12, 2015 at 11:34 AM, Peter Manev <span dir="ltr"><<a href="mailto:petermanev@gmail.com" target="_blank">petermanev@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Wed, Aug 12, 2015 at 8:22 PM, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>> wrote:<br>
> 4x3.2Ghz Xeon cores<br>
><br>
<br>
That is a VM, correct?<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
> On Wed, Aug 12, 2015 at 8:32 AM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>> wrote:<br>
>><br>
>> On Tue, Aug 11, 2015 at 8:39 PM, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> wrote:<br>
>> > Drops to ~7-10% with that change. Sorry for the delayed response.<br>
>><br>
>><br>
>> I would suggest (for your test set up) - start with the defaults, then<br>
>> increase as needed (watch out for flow.emerg_mode_entered counters in<br>
>> the stats.log).<br>
>> How many CPUs do you have on the test VM ? (and what speed/type?)<br>
>><br>
>> Thanks<br>
>><br>
>> ><br>
>> > On Sat, Aug 1, 2015 at 10:24 AM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> On Fri, Jul 31, 2015 at 7:39 PM, Duane Howard<br>
>> >> <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> wrote:<br>
>> >> > Just build 2.1beta4 and it's the same thing.<br>
>> >> ><br>
>> >> > On Fri, Jul 31, 2015 at 10:22 AM, Duane Howard<br>
>> >> > <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> > wrote:<br>
>> >> >><br>
>> >> >> yaml sent off list.<br>
>> >><br>
>> >> From the privately shared yaml -<br>
>> >> flow:<br>
>> >> memcap: 1.5gb<br>
>> >> hash-size: 15728640<br>
>> >><br>
>> >> divide flow.memcap by 3 and flow.hash-size by 10 - reload and please<br>
>> >> let us know if any difference.<br>
>> >><br>
>> >> Thanks<br>
>> >><br>
>> >> >>This is the only machine I've tried on right now, and<br>
>> >> >> I can't use the other one that is similarly configured because it's<br>
>> >> >> being<br>
>> >> >> used for other stuff at the moment. I did pretty extensive<br>
>> >> >> experimentation<br>
>> >> >> with 2.0.6 a while back on hardware and didn't see this issue. I'll<br>
>> >> >> try<br>
>> >> >> installing 2.1 and playing with it later today to see if I can<br>
>> >> >> reproduce<br>
>> >> >> that way.<br>
>> >> >><br>
>> >> >> On Fri, Jul 31, 2015 at 6:47 AM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>><br>
>> >> >> wrote:<br>
>> >> >>><br>
>> >> >>> On Thu, Jul 30, 2015 at 11:27 PM, Duane Howard<br>
>> >> >>> <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> >>> wrote:<br>
>> >> >>> > strace only shows the process doing constant:<br>
>> >> >>> > nanosleep({0, 10000000}, NULL) = 0<br>
>> >> >>> > nanosleep({0, 10000000}, NULL) = 0<br>
>> >> >>> > nanosleep({0, 10000000}, NULL) = 0<br>
>> >> >>> > nanosleep({0, 10000000}, NULL) = 0<br>
>> >> >>> > nanosleep({0, 10000000}, NULL) = 0<br>
>> >> >>> > nanosleep({0, 10000000}, NULL) = 0<br>
>> >> >>> > ....<br>
>> >> >>> ><br>
>> >> >>><br>
>> >> >>> Hmm strange. I have not experianced that before.<br>
>> >> >>> Does this happen on any VM or just this one in particular?<br>
>> >> >>> Can you switch to 2.1beta4 and see if any difference. Please feel<br>
>> >> >>> free<br>
>> >> >>> to send me your suricata.yaml if you would like - I can see if this<br>
>> >> >>> is<br>
>> >> >>> enough to reproduce the issue.<br>
>> >> >>><br>
>> >> >>> thanks<br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> ><br>
>> >> >>> > On Thu, Jul 30, 2015 at 2:17 PM, Duane Howard<br>
>> >> >>> > <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> >>> > wrote:<br>
>> >> >>> >><br>
>> >> >>> >> I can reproduce. I ran it on a single low volume real NIC with<br>
>> >> >>> >> the<br>
>> >> >>> >> same<br>
>> >> >>> >> result. Nothing terribly interesting in suricata.log with -vv<br>
>> >> >>> >> enabled:<br>
>> >> >>> >> Here's the logs after rule and config loading:<br>
>> >> >>> >><br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Core dump size set to<br>
>> >> >>> >> unlimited.<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Unified2-alert initialized:<br>
>> >> >>> >> filename<br>
>> >> >>> >> suricata.u2, limit 128 MB<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Syslog output initialized<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Unable to find af-packet config<br>
>> >> >>> >> for<br>
>> >> >>> >> interface "eth0" or "default", using default value<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Going to use 1 thread(s)<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Enabling zero copy mode<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - RunModeIdsAFPWorkers<br>
>> >> >>> >> initialised<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Notice> - all 1 packet processing<br>
>> >> >>> >> threads,<br>
>> >> >>> >> 3<br>
>> >> >>> >> management threads initialized, engine started.<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Using interface 'eth0' via<br>
>> >> >>> >> socket<br>
>> >> >>> >> 7<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - All AFP capture threads are<br>
>> >> >>> >> running.<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Thread AFPacketeth01 using<br>
>> >> >>> >> socket<br>
>> >> >>> >> 7<br>
>> >> >>> >> 30/7/2015 -- 21:14:06 - <Info> - Starting to read on<br>
>> >> >>> >> AFPacketeth01<br>
>> >> >>> >> ^C30/7/2015 -- 21:15:27 - <Notice> - Signal Received. Stopping<br>
>> >> >>> >> engine.<br>
>> >> >>> >> 30/7/2015 -- 21:15:27 - <Info> - 0 new flows, 0 established<br>
>> >> >>> >> flows<br>
>> >> >>> >> were<br>
>> >> >>> >> timed out, 0 flows in closed state<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - time elapsed 81.925s<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - (AFPacketeth01) Kernel: Packets<br>
>> >> >>> >> 3372,<br>
>> >> >>> >> dropped 0<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - (AFPacketeth01) Packets 3345,<br>
>> >> >>> >> bytes<br>
>> >> >>> >> 1318862<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - Stream TCP processed 2723 TCP<br>
>> >> >>> >> packets<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - (AFPacketeth01) Alerts 0<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - Alert unified2 module wrote 0<br>
>> >> >>> >> alerts<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - host memory usage: 390144<br>
>> >> >>> >> bytes,<br>
>> >> >>> >> maximum:<br>
>> >> >>> >> 16777216<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Info> - cleaning up signature grouping<br>
>> >> >>> >> structure... complete<br>
>> >> >>> >> 30/7/2015 -- 21:15:28 - <Notice> - Stats for 'eth0': pkts:<br>
>> >> >>> >> 3372,<br>
>> >> >>> >> drop: 0<br>
>> >> >>> >> (0.00%), invalid chksum: 0<br>
>> >> >>> >><br>
>> >> >>> >><br>
>> >> >>> >> On Thu, Jul 30, 2015 at 2:02 PM, Peter Manev<br>
>> >> >>> >> <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>><br>
>> >> >>> >> wrote:<br>
>> >> >>> >>><br>
>> >> >>> >>><br>
>> >> >>> >>> On 30 jul 2015, at 21:50, Duane Howard<br>
>> >> >>> >>> <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> >>> >>> wrote:<br>
>> >> >>> >>><br>
>> >> >>> >>><br>
>> >> >>> >>> Can you reproduce this problem at will? Anything in verbose<br>
>> >> >>> >>> mode<br>
>> >> >>> >>> in<br>
>> >> >>> >>> Suricata.log?<br>
>> >> >>> >>><br>
>> >> >>> >>> Can you narrow it down to a relation of a single interface<br>
>> >> >>> >>> configuration<br>
>> >> >>> >>> (as you mention in your initial mail)?<br>
>> >> >>> >>><br>
>> >> >>> >>> Thanks<br>
>> >> >>> >>><br>
>> >> >>> >>><br>
>> >> >>> >>> This is Suricata version 2.0.8 RELEASE<br>
>> >> >>> >>> Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 PF_RING<br>
>> >> >>> >>> AF_PACKET<br>
>> >> >>> >>> HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1<br>
>> >> >>> >>> HAVE_HTP_URI_NORMALIZE_HOOK<br>
>> >> >>> >>> SIMD support: none<br>
>> >> >>> >>> Atomic intrisics: 1 2 4 8 byte(s)<br>
>> >> >>> >>> 64-bits, Little-endian architecture<br>
>> >> >>> >>> GCC version 4.6.3, C version 199901<br>
>> >> >>> >>> compiled with -fstack-protector<br>
>> >> >>> >>> compiled with _FORTIFY_SOURCE=2<br>
>> >> >>> >>> L1 cache line size (CLS)=64<br>
>> >> >>> >>> compiled with LibHTP v0.5.17, linked against LibHTP v0.5.17<br>
>> >> >>> >>> Suricata Configuration:<br>
>> >> >>> >>> AF_PACKET support: yes<br>
>> >> >>> >>> PF_RING support: yes<br>
>> >> >>> >>> NFQueue support: no<br>
>> >> >>> >>> NFLOG support: no<br>
>> >> >>> >>> IPFW support: no<br>
>> >> >>> >>> DAG enabled: no<br>
>> >> >>> >>> Napatech enabled: no<br>
>> >> >>> >>> Unix socket enabled: no<br>
>> >> >>> >>> Detection enabled: yes<br>
>> >> >>> >>><br>
>> >> >>> >>> libnss support: no<br>
>> >> >>> >>> libnspr support: no<br>
>> >> >>> >>> libjansson support: no<br>
>> >> >>> >>> Prelude support: no<br>
>> >> >>> >>> PCRE jit: no<br>
>> >> >>> >>> LUA support: no<br>
>> >> >>> >>> libluajit: no<br>
>> >> >>> >>> libgeoip: yes<br>
>> >> >>> >>> Non-bundled htp: yes<br>
>> >> >>> >>> Old barnyard2 support: no<br>
>> >> >>> >>> CUDA enabled: no<br>
>> >> >>> >>><br>
>> >> >>> >>> Suricatasc install: yes<br>
>> >> >>> >>><br>
>> >> >>> >>> Unit tests enabled: no<br>
>> >> >>> >>> Debug output enabled: no<br>
>> >> >>> >>> Debug validation enabled: no<br>
>> >> >>> >>> Profiling enabled: no<br>
>> >> >>> >>> Profiling locks enabled: no<br>
>> >> >>> >>> Coccinelle / spatch: no<br>
>> >> >>> >>><br>
>> >> >>> >>> Generic build parameters:<br>
>> >> >>> >>> Installation prefix (--prefix): /usr<br>
>> >> >>> >>> Configuration directory (--sysconfdir): /etc/suricata/<br>
>> >> >>> >>> Log directory (--localstatedir) : /var/log/suricata/<br>
>> >> >>> >>><br>
>> >> >>> >>> Host: x86_64-pc-linux-gnu<br>
>> >> >>> >>> GCC binary: gcc<br>
>> >> >>> >>> GCC Protect enabled: no<br>
>> >> >>> >>> GCC march native enabled: no<br>
>> >> >>> >>> GCC Profile enabled: no<br>
>> >> >>> >>><br>
>> >> >>> >>><br>
>> >> >>> >>> On Thu, Jul 30, 2015 at 1:41 PM, Peter Manev<br>
>> >> >>> >>> <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>><br>
>> >> >>> >>> wrote:<br>
>> >> >>> >>>><br>
>> >> >>> >>>><br>
>> >> >>> >>>><br>
>> >> >>> >>>> On 30 jul 2015, at 20:58, Duane Howard<br>
>> >> >>> >>>> <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> >>> >>>> wrote:<br>
>> >> >>> >>>><br>
>> >> >>> >>>> When closing suri it reports 0 packets for the interface I'm<br>
>> >> >>> >>>> mainly<br>
>> >> >>> >>>> concerned with. There's also nothing in fast or other logs to<br>
>> >> >>> >>>> indicate<br>
>> >> >>> >>>> traffic.<br>
>> >> >>> >>>><br>
>> >> >>> >>>> 30/7/2015 -- 19:58:41 - <Notice> - Stats for 'bond0': pkts:<br>
>> >> >>> >>>> 0,<br>
>> >> >>> >>>> drop: 0<br>
>> >> >>> >>>> (-nan%), invalid chksum: 0<br>
>> >> >>> >>>><br>
>> >> >>> >>>><br>
>> >> >>> >>>> What is the output of suricata --build-info ?<br>
>> >> >>> >>>><br>
>> >> >>> >>>> If you have perf tools installed have a look at what in<br>
>> >> >>> >>>> Suricata<br>
>> >> >>> >>>> uses<br>
>> >> >>> >>>> the most CPU -<br>
>> >> >>> >>>> perf top -p pidofsuri<br>
>> >> >>> >>>><br>
>> >> >>> >>>> Thanks<br>
>> >> >>> >>>><br>
>> >> >>> >>>><br>
>> >> >>> >>>> On Thu, Jul 30, 2015 at 12:57 PM, Alan Wanderley dos Santos<br>
>> >> >>> >>>> <<a href="mailto:alan.santos@rnp.br">alan.santos@rnp.br</a>> wrote:<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> Didi you see the fast.log and others logs?<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> Maybe there are some traffic (icmp or broadcast for example)<br>
>> >> >>> >>>>> coming<br>
>> >> >>> >>>>> to<br>
>> >> >>> >>>>> virtual machine, even little being data, can generated a lot<br>
>> >> >>> >>>>> of<br>
>> >> >>> >>>>> logs and<br>
>> >> >>> >>>>> degree the performance.<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> I had a similar situation on a VM of testing.<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> Just a shot into darkness rsrs<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> Regards,<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> -----------------------------------------------<br>
>> >> >>> >>>>> Alan Santos<br>
>> >> >>> >>>>> Analista de Segurança<br>
>> >> >>> >>>>> Centro de Atendimento a Incidentes de Segurança (CAIS)<br>
>> >> >>> >>>>> Rede Nacional de Ensino e Pesquisa (RNP)<br>
>> >> >>> >>>>> (19) 3787-3314 | <a href="mailto:alan.santos@rnp.br">alan.santos@rnp.br</a><br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> ________________________________<br>
>> >> >>> >>>>> De: "Duane Howard" <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>> >> >>> >>>>> Para: "oisf-users" <<a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a>><br>
>> >> >>> >>>>> Enviadas: Quinta-feira, 30 de julho de 2015 16:50:51<br>
>> >> >>> >>>>> Assunto: [Oisf-users] Suricata using 35% cpu with no load?<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> I've got a random virtual testing machine, and I'm seeing<br>
>> >> >>> >>>>> Suricata<br>
>> >> >>> >>>>> sitting at about 35% CPU load, even though there's absolutely<br>
>> >> >>> >>>>> no<br>
>> >> >>> >>>>> traffic<br>
>> >> >>> >>>>> heading to it at the moment. Is there an easy way to get<br>
>> >> >>> >>>>> Suricata<br>
>> >> >>> >>>>> to tell me<br>
>> >> >>> >>>>> what it's doing that would cause this? It occurs on real<br>
>> >> >>> >>>>> interfaces<br>
>> >> >>> >>>>> with low<br>
>> >> >>> >>>>> traffic, loopback, as well as bonds where there's no trafic.<br>
>> >> >>> >>>>> ./d<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> _______________________________________________<br>
>> >> >>> >>>>> Suricata IDS Users mailing list:<br>
>> >> >>> >>>>> <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
>> >> >>> >>>>> Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support:<br>
>> >> >>> >>>>> <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
>> >> >>> >>>>> List:<br>
>> >> >>> >>>>><br>
>> >> >>> >>>>><br>
>> >> >>> >>>>> <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
>> >> >>> >>>>> Suricata User Conference November 4 & 5 in Barcelona:<br>
>> >> >>> >>>>> <a href="http://oisfevents.net" rel="noreferrer" target="_blank">http://oisfevents.net</a><br>
>> >> >>> >>>><br>
>> >> >>> >>>><br>
>> >> >>> >>>> _______________________________________________<br>
>> >> >>> >>>> Suricata IDS Users mailing list:<br>
>> >> >>> >>>> <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
>> >> >>> >>>> Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support:<br>
>> >> >>> >>>> <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
>> >> >>> >>>> List:<br>
>> >> >>> >>>><br>
>> >> >>> >>>><br>
>> >> >>> >>>> <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
>> >> >>> >>>> Suricata User Conference November 4 & 5 in Barcelona:<br>
>> >> >>> >>>> <a href="http://oisfevents.net" rel="noreferrer" target="_blank">http://oisfevents.net</a><br>
>> >> >>> >>><br>
>> >> >>> >>><br>
>> >> >>> >><br>
>> >> >>> ><br>
>> >> >>><br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> --<br>
>> >> >>> Regards,<br>
>> >> >>> Peter Manev<br>
>> >> >><br>
>> >> >><br>
>> >> ><br>
>> >><br>
>> >><br>
>> >><br>
>> >> --<br>
>> >> Regards,<br>
>> >> Peter Manev<br>
>> ><br>
>> ><br>
>><br>
>><br>
>><br>
>> --<br>
>> Regards,<br>
>> Peter Manev<br>
><br>
><br>
<br>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Regards,<br>
Peter Manev<br>
</font></span></blockquote></div><br></div>