<div dir="ltr">Drops to ~7-10% with that change. Sorry for the delayed response.</div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Aug 1, 2015 at 10:24 AM, Peter Manev <span dir="ltr"><<a href="mailto:petermanev@gmail.com" target="_blank">petermanev@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, Jul 31, 2015 at 7:39 PM, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>> wrote:<br>
> Just build 2.1beta4 and it's the same thing.<br>
><br>
> On Fri, Jul 31, 2015 at 10:22 AM, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
> wrote:<br>
>><br>
>> yaml sent off list.<br>
<br>
</span>From the privately shared yaml -<br>
flow:<br>
memcap: 1.5gb<br>
hash-size: 15728640<br>
<br>
divide flow.memcap by 3 and flow.hash-size by 10 - reload and please<br>
let us know if any difference.<br>
<br>
Thanks<br>
<div class="HOEnZb"><div class="h5"><br>
>>This is the only machine I've tried on right now, and<br>
>> I can't use the other one that is similarly configured because it's being<br>
>> used for other stuff at the moment. I did pretty extensive experimentation<br>
>> with 2.0.6 a while back on hardware and didn't see this issue. I'll try<br>
>> installing 2.1 and playing with it later today to see if I can reproduce<br>
>> that way.<br>
>><br>
>> On Fri, Jul 31, 2015 at 6:47 AM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>> wrote:<br>
>>><br>
>>> On Thu, Jul 30, 2015 at 11:27 PM, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>>> wrote:<br>
>>> > strace only shows the process doing constant:<br>
>>> > nanosleep({0, 10000000}, NULL) = 0<br>
>>> > nanosleep({0, 10000000}, NULL) = 0<br>
>>> > nanosleep({0, 10000000}, NULL) = 0<br>
>>> > nanosleep({0, 10000000}, NULL) = 0<br>
>>> > nanosleep({0, 10000000}, NULL) = 0<br>
>>> > nanosleep({0, 10000000}, NULL) = 0<br>
>>> > ....<br>
>>> ><br>
>>><br>
>>> Hmm strange. I have not experianced that before.<br>
>>> Does this happen on any VM or just this one in particular?<br>
>>> Can you switch to 2.1beta4 and see if any difference. Please feel free<br>
>>> to send me your suricata.yaml if you would like - I can see if this is<br>
>>> enough to reproduce the issue.<br>
>>><br>
>>> thanks<br>
>>><br>
>>><br>
>>> ><br>
>>> > On Thu, Jul 30, 2015 at 2:17 PM, Duane Howard<br>
>>> > <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>>> > wrote:<br>
>>> >><br>
>>> >> I can reproduce. I ran it on a single low volume real NIC with the<br>
>>> >> same<br>
>>> >> result. Nothing terribly interesting in suricata.log with -vv enabled:<br>
>>> >> Here's the logs after rule and config loading:<br>
>>> >><br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Core dump size set to unlimited.<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Unified2-alert initialized: filename<br>
>>> >> suricata.u2, limit 128 MB<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Syslog output initialized<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Unable to find af-packet config for<br>
>>> >> interface "eth0" or "default", using default value<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Going to use 1 thread(s)<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Enabling zero copy mode<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - RunModeIdsAFPWorkers initialised<br>
>>> >> 30/7/2015 -- 21:14:06 - <Notice> - all 1 packet processing threads, 3<br>
>>> >> management threads initialized, engine started.<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Using interface 'eth0' via socket 7<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - All AFP capture threads are running.<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Thread AFPacketeth01 using socket 7<br>
>>> >> 30/7/2015 -- 21:14:06 - <Info> - Starting to read on AFPacketeth01<br>
>>> >> ^C30/7/2015 -- 21:15:27 - <Notice> - Signal Received. Stopping<br>
>>> >> engine.<br>
>>> >> 30/7/2015 -- 21:15:27 - <Info> - 0 new flows, 0 established flows were<br>
>>> >> timed out, 0 flows in closed state<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - time elapsed 81.925s<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - (AFPacketeth01) Kernel: Packets 3372,<br>
>>> >> dropped 0<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - (AFPacketeth01) Packets 3345, bytes<br>
>>> >> 1318862<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - Stream TCP processed 2723 TCP packets<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - (AFPacketeth01) Alerts 0<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - Alert unified2 module wrote 0 alerts<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - host memory usage: 390144 bytes,<br>
>>> >> maximum:<br>
>>> >> 16777216<br>
>>> >> 30/7/2015 -- 21:15:28 - <Info> - cleaning up signature grouping<br>
>>> >> structure... complete<br>
>>> >> 30/7/2015 -- 21:15:28 - <Notice> - Stats for 'eth0': pkts: 3372,<br>
>>> >> drop: 0<br>
>>> >> (0.00%), invalid chksum: 0<br>
>>> >><br>
>>> >><br>
>>> >> On Thu, Jul 30, 2015 at 2:02 PM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>><br>
>>> >> wrote:<br>
>>> >>><br>
>>> >>><br>
>>> >>> On 30 jul 2015, at 21:50, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>>> >>> wrote:<br>
>>> >>><br>
>>> >>><br>
>>> >>> Can you reproduce this problem at will? Anything in verbose mode in<br>
>>> >>> Suricata.log?<br>
>>> >>><br>
>>> >>> Can you narrow it down to a relation of a single interface<br>
>>> >>> configuration<br>
>>> >>> (as you mention in your initial mail)?<br>
>>> >>><br>
>>> >>> Thanks<br>
>>> >>><br>
>>> >>><br>
>>> >>> This is Suricata version 2.0.8 RELEASE<br>
>>> >>> Features: PCAP_SET_BUFF LIBPCAP_VERSION_MAJOR=1 PF_RING AF_PACKET<br>
>>> >>> HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK<br>
>>> >>> SIMD support: none<br>
>>> >>> Atomic intrisics: 1 2 4 8 byte(s)<br>
>>> >>> 64-bits, Little-endian architecture<br>
>>> >>> GCC version 4.6.3, C version 199901<br>
>>> >>> compiled with -fstack-protector<br>
>>> >>> compiled with _FORTIFY_SOURCE=2<br>
>>> >>> L1 cache line size (CLS)=64<br>
>>> >>> compiled with LibHTP v0.5.17, linked against LibHTP v0.5.17<br>
>>> >>> Suricata Configuration:<br>
>>> >>> AF_PACKET support: yes<br>
>>> >>> PF_RING support: yes<br>
>>> >>> NFQueue support: no<br>
>>> >>> NFLOG support: no<br>
>>> >>> IPFW support: no<br>
>>> >>> DAG enabled: no<br>
>>> >>> Napatech enabled: no<br>
>>> >>> Unix socket enabled: no<br>
>>> >>> Detection enabled: yes<br>
>>> >>><br>
>>> >>> libnss support: no<br>
>>> >>> libnspr support: no<br>
>>> >>> libjansson support: no<br>
>>> >>> Prelude support: no<br>
>>> >>> PCRE jit: no<br>
>>> >>> LUA support: no<br>
>>> >>> libluajit: no<br>
>>> >>> libgeoip: yes<br>
>>> >>> Non-bundled htp: yes<br>
>>> >>> Old barnyard2 support: no<br>
>>> >>> CUDA enabled: no<br>
>>> >>><br>
>>> >>> Suricatasc install: yes<br>
>>> >>><br>
>>> >>> Unit tests enabled: no<br>
>>> >>> Debug output enabled: no<br>
>>> >>> Debug validation enabled: no<br>
>>> >>> Profiling enabled: no<br>
>>> >>> Profiling locks enabled: no<br>
>>> >>> Coccinelle / spatch: no<br>
>>> >>><br>
>>> >>> Generic build parameters:<br>
>>> >>> Installation prefix (--prefix): /usr<br>
>>> >>> Configuration directory (--sysconfdir): /etc/suricata/<br>
>>> >>> Log directory (--localstatedir) : /var/log/suricata/<br>
>>> >>><br>
>>> >>> Host: x86_64-pc-linux-gnu<br>
>>> >>> GCC binary: gcc<br>
>>> >>> GCC Protect enabled: no<br>
>>> >>> GCC march native enabled: no<br>
>>> >>> GCC Profile enabled: no<br>
>>> >>><br>
>>> >>><br>
>>> >>> On Thu, Jul 30, 2015 at 1:41 PM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>><br>
>>> >>> wrote:<br>
>>> >>>><br>
>>> >>>><br>
>>> >>>><br>
>>> >>>> On 30 jul 2015, at 20:58, Duane Howard <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>>> >>>> wrote:<br>
>>> >>>><br>
>>> >>>> When closing suri it reports 0 packets for the interface I'm mainly<br>
>>> >>>> concerned with. There's also nothing in fast or other logs to<br>
>>> >>>> indicate<br>
>>> >>>> traffic.<br>
>>> >>>><br>
>>> >>>> 30/7/2015 -- 19:58:41 - <Notice> - Stats for 'bond0': pkts: 0,<br>
>>> >>>> drop: 0<br>
>>> >>>> (-nan%), invalid chksum: 0<br>
>>> >>>><br>
>>> >>>><br>
>>> >>>> What is the output of suricata --build-info ?<br>
>>> >>>><br>
>>> >>>> If you have perf tools installed have a look at what in Suricata<br>
>>> >>>> uses<br>
>>> >>>> the most CPU -<br>
>>> >>>> perf top -p pidofsuri<br>
>>> >>>><br>
>>> >>>> Thanks<br>
>>> >>>><br>
>>> >>>><br>
>>> >>>> On Thu, Jul 30, 2015 at 12:57 PM, Alan Wanderley dos Santos<br>
>>> >>>> <<a href="mailto:alan.santos@rnp.br">alan.santos@rnp.br</a>> wrote:<br>
>>> >>>>><br>
>>> >>>>> Didi you see the fast.log and others logs?<br>
>>> >>>>><br>
>>> >>>>> Maybe there are some traffic (icmp or broadcast for example) coming<br>
>>> >>>>> to<br>
>>> >>>>> virtual machine, even little being data, can generated a lot of<br>
>>> >>>>> logs and<br>
>>> >>>>> degree the performance.<br>
>>> >>>>><br>
>>> >>>>> I had a similar situation on a VM of testing.<br>
>>> >>>>><br>
>>> >>>>> Just a shot into darkness rsrs<br>
>>> >>>>><br>
>>> >>>>> Regards,<br>
>>> >>>>><br>
>>> >>>>> -----------------------------------------------<br>
>>> >>>>> Alan Santos<br>
>>> >>>>> Analista de Segurança<br>
>>> >>>>> Centro de Atendimento a Incidentes de Segurança (CAIS)<br>
>>> >>>>> Rede Nacional de Ensino e Pesquisa (RNP)<br>
>>> >>>>> (19) 3787-3314 | <a href="mailto:alan.santos@rnp.br">alan.santos@rnp.br</a><br>
>>> >>>>><br>
>>> >>>>> ________________________________<br>
>>> >>>>> De: "Duane Howard" <<a href="mailto:duane.security@gmail.com">duane.security@gmail.com</a>><br>
>>> >>>>> Para: "oisf-users" <<a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a>><br>
>>> >>>>> Enviadas: Quinta-feira, 30 de julho de 2015 16:50:51<br>
>>> >>>>> Assunto: [Oisf-users] Suricata using 35% cpu with no load?<br>
>>> >>>>><br>
>>> >>>>> I've got a random virtual testing machine, and I'm seeing Suricata<br>
>>> >>>>> sitting at about 35% CPU load, even though there's absolutely no<br>
>>> >>>>> traffic<br>
>>> >>>>> heading to it at the moment. Is there an easy way to get Suricata<br>
>>> >>>>> to tell me<br>
>>> >>>>> what it's doing that would cause this? It occurs on real interfaces<br>
>>> >>>>> with low<br>
>>> >>>>> traffic, loopback, as well as bonds where there's no trafic.<br>
>>> >>>>> ./d<br>
>>> >>>>><br>
>>> >>>>> _______________________________________________<br>
>>> >>>>> Suricata IDS Users mailing list:<br>
>>> >>>>> <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
>>> >>>>> Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support:<br>
>>> >>>>> <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
>>> >>>>> List:<br>
>>> >>>>> <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
>>> >>>>> Suricata User Conference November 4 & 5 in Barcelona:<br>
>>> >>>>> <a href="http://oisfevents.net" rel="noreferrer" target="_blank">http://oisfevents.net</a><br>
>>> >>>><br>
>>> >>>><br>
>>> >>>> _______________________________________________<br>
>>> >>>> Suricata IDS Users mailing list:<br>
>>> >>>> <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
>>> >>>> Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support:<br>
>>> >>>> <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
>>> >>>> List:<br>
>>> >>>> <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
>>> >>>> Suricata User Conference November 4 & 5 in Barcelona:<br>
>>> >>>> <a href="http://oisfevents.net" rel="noreferrer" target="_blank">http://oisfevents.net</a><br>
>>> >>><br>
>>> >>><br>
>>> >><br>
>>> ><br>
>>><br>
>>><br>
>>><br>
>>> --<br>
>>> Regards,<br>
>>> Peter Manev<br>
>><br>
>><br>
><br>
<br>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
Regards,<br>
Peter Manev<br>
</font></span></blockquote></div><br></div>