<p class="MsoNormal"><span lang="EN-US">Hi</span><span style="font-family:宋体">，</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">It seems that 1.3-dev works well for a
whole day. After it starts, the memory usage slowly increase. When the usage is
above 80%, it remains unchanged for a long time.</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">There isn’t any coredump file too. Everything
is all right. But I find that there are some logs as follows.</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p><p class="MsoNormal"><span lang="EN-US">...</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:00:09 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687208, ts.tv_usec:789402) flow_spare_q status(): 1408% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:01:30 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687290, ts.tv_usec:645263) flow_spare_q status(): 1263% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:02:34 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687354, ts.tv_usec:263409) flow_spare_q status(): 1181% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:03:11 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687391, ts.tv_usec:535048) flow_spare_q status(): 1388% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:03:46 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687426, ts.tv_usec:697482) flow_spare_q status(): 1093% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:04:06 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687446, ts.tv_usec:541106) flow_spare_q status(): 1010% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:04:38 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687478, ts.tv_usec:503235) flow_spare_q status(): 1406% flows at the queue</span></p>
<p class="MsoNormal"><span lang="EN-US">[2735] 5/12/2012
-- 14:05:19 - (flow-manager.c:517) <Info> (FlowManagerThread) -- Flow
emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec:
1354687518, ts.tv_usec:764832) flow_spare_q status(): 1184% flows at the queue</span></p><p class="MsoNormal">...</p><p class="MsoNormal"><br></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">I will run it for some days to see whether anything
else happens. </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Thanks.</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p><br><div class="gmail_quote">On Mon, Dec 3, 2012 at 10:54 PM, Victor Julien <span dir="ltr"><<a href="mailto:victor@inliniac.net" target="_blank">victor@inliniac.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 12/03/2012 03:38 AM, xbadou xbadou wrote:<br>
> Hi,<br>
><br>
> I use 'top' command, my memory is as follows:<br>
><br>
> Mem: 3080880k total, 356708k used, 2724172k free, 6452k buffers<br>
> Swap: 2650684k total, 0k used, 2650684k free, 83212k cached<br>
><br>
> When suricata starts, it used about 6.6% (~203MB). But it become larger<br>
> and larger.<br>
><br>
> The following is some log when suricata starts.<br>
><br>
> 3/12/2012 -- 08:44:50 - <Info> - AutoFP mode using default "Active<br>
> Packets" flow load balancer<br>
> 3/12/2012 -- 08:44:50 - <Info> - Use pid file /var/run/suricata.pid from<br>
> config file.<br>
> 3/12/2012 -- 08:44:50 - <Info> - preallocated 5000 packets. Total memory<br>
> 15440000<br>
> 3/12/2012 -- 08:44:50 - <Info> - allocated 131072 bytes of memory for<br>
> the host hash... 4096 buckets of size 32<br>
> 3/12/2012 -- 08:44:50 - <Info> - preallocated 1000 hosts of size 72<br>
> 3/12/2012 -- 08:44:50 - <Info> - host memory usage: 203072 bytes,<br>
> maximum: 16777216<br>
> 3/12/2012 -- 08:44:50 - <Info> - allocated 2097152 bytes of memory for<br>
> the flow hash... 65536 buckets of size 32<br>
> 3/12/2012 -- 08:44:50 - <Info> - preallocated 10000 flows of size 176<br>
> 3/12/2012 -- 08:44:50 - <Info> - flow memory usage: 3857152 bytes,<br>
> maximum: 33554432<br>
> 3/12/2012 -- 08:44:50 - <Info> - using magic-file /usr/share/file/magic<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_UNKNOWN_DECODE_EVENT(191)] - unknown decode event<br>
> "ipv6.ipv4_in_ipv6_too_small"<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert pkthdr<br>
> any any -> any any (msg:"SURICATA IPv4-in-IPv6 packet too short";<br>
> decode-event:ipv6.ipv4_in_ipv6_too_small; sid:2200082; rev:1;)" from<br>
> file /etc/suricata/rules/decoder-events.rules at line 93<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_UNKNOWN_DECODE_EVENT(191)] - unknown decode event<br>
> "ipv6.ipv4_in_ipv6_wrong_version"<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert pkthdr<br>
> any any -> any any (msg:"SURICATA IPv4-in-IPv6 invalid protocol";<br>
> decode-event:ipv6.ipv4_in_ipv6_wrong_version; sid:2200083; rev:1;)" from<br>
> file /etc/suricata/rules/decoder-events.rules at line 94<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_UNKNOWN_DECODE_EVENT(191)] - unknown decode event<br>
> "ipv6.ipv6_in_ipv6_too_small"<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert pkthdr<br>
> any any -> any any (msg:"SURICATA IPv6-in-IPv6 packet too short";<br>
> decode-event:ipv6.ipv6_in_ipv6_too_small; sid:2200084; rev:1;)" from<br>
> file /etc/suricata/rules/decoder-events.rules at line 96<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_UNKNOWN_DECODE_EVENT(191)] - unknown decode event<br>
> "ipv6.ipv6_in_ipv6_wrong_version"<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert pkthdr<br>
> any any -> any any (msg:"SURICATA IPv6-in-IPv6 invalid protocol";<br>
> decode-event:ipv6.ipv6_in_ipv6_wrong_version; sid:2200085; rev:1;)" from<br>
> file /etc/suricata/rules/decoder-events.rules at line 97<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_OPENING_RULE_FILE(41)] - opening rule file<br>
> /etc/suricata/rules/emerging-botcc.rules: No such file or directory.<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_OPENING_RULE_FILE(41)] - opening rule file<br>
> /etc/suricata/rules/emerging-ciarmy.rules: No such file or directory.<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_OPENING_RULE_FILE(41)] - opening rule file<br>
> /etc/suricata/rules/emerging-compromised.rules: No such file or directory.<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_OPENING_RULE_FILE(41)] - opening rule file<br>
> /etc/suricata/rules/emerging-drop.rules: No such file or directory.<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_OPENING_RULE_FILE(41)] - opening rule file<br>
> /etc/suricata/rules/emerging-dshield.rules: No such file or directory.<br>
> 3/12/2012 -- 08:44:53 - <Error> - [ERRCODE:<br>
> SC_ERR_OPENING_RULE_FILE(41)] - opening rule file<br>
> /etc/suricata/rules/emerging-tor.rules: No such file or directory.<br>
> 3/12/2012 -- 08:44:53 - <Info> - 41 rule files processed. 6106 rules<br>
> succesfully loaded, 4 rules failed<br>
> 3/12/2012 -- 08:44:54 - <Info> - 6114 signatures processed. 4 are<br>
> IP-only rules, 2880 are inspecting packet payload, 3885 inspect<br>
> application layer, 72 are decoder event only<br>
> 3/12/2012 -- 08:44:54 - <Info> - building signature grouping structure,<br>
> stage 1: adding signatures to signature source addresses... complete<br>
> 3/12/2012 -- 08:44:54 - <Info> - building signature grouping structure,<br>
> stage 2: building source address list... complete<br>
> 3/12/2012 -- 08:44:56 - <Info> - building signature grouping structure,<br>
> stage 3: building destination address lists... complete<br>
> 3/12/2012 -- 08:44:57 - <Warning> - [ERRCODE: SC_ERR_FOPEN(44)] - Error<br>
> opening file: "/etc/suricata//threshold.config": No such file or directory<br>
> 3/12/2012 -- 08:44:57 - <Info> - Core dump size is unlimited.<br>
> 3/12/2012 -- 08:44:57 - <Info> - fast output device (regular)<br>
> initialized: fast.log<br>
> 3/12/2012 -- 08:44:57 - <Info> - Unified2-alert initialized: filename<br>
> unified2.alert, limit 32 MB<br>
> 3/12/2012 -- 08:44:57 - <Info> - Using 1 live device(s).<br>
> 3/12/2012 -- 08:44:57 - <Info> - Unable to find pcap config for<br>
> interface wafbridge1, using default value<br>
> 3/12/2012 -- 08:44:57 - <Info> - using interface wafbridge1<br>
> 3/12/2012 -- 08:44:57 - <Info> - RunModeIdsPcapAutoFp initialised<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream "max-sessions": 262144<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream "prealloc-sessions": 32768<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream "memcap": 33554432<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream "midstream" session pickups:<br>
> disabled<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream "async-oneside": disabled<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream "checksum-validation": enabled<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream."inline": disabled<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream.reassembly "memcap": 67108864<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream.reassembly "depth": 1048576<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream.reassembly<br>
> "toserver-chunk-size": 2560<br>
> 3/12/2012 -- 08:44:57 - <Info> - stream.reassembly<br>
> "toclient-chunk-size": 2560<br>
> 3/12/2012 -- 08:44:57 - <Info> - all 7 packet processing threads, 3<br>
> management threads initialized, engine started.<br>
><br>
><br>
> My testing network is like this.<br>
><br>
> Working Network ------Suricata-------Internet<br>
><br>
> Working Network bandwidth is about 8~30Mbit/s. Each traffic we visit<br>
> Internet is checked by Suricata.<br>
<br>
</div></div>Could you try upgrading to our 1.3-dev branch here:<br>
<a href="https://github.com/inliniac/suricata/tree/master-1.3.x" target="_blank">https://github.com/inliniac/suricata/tree/master-1.3.x</a><br>
<br>
We fixed a memory leak in our flow engine that may be related to the<br>
issue you are having.<br>
<br>
Cheers,<br>
Victor<br>
<div class="im"><br>
<br>
><br>
> Thank you.<br>
><br>
><br>
><br>
> On Fri, Nov 30, 2012 at 6:44 PM, Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a><br>
</div><div class="im">> <mailto:<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>>> wrote:<br>
><br>
> Hi,<br>
><br>
> You mention that you have small traffic - how much memory does<br>
> Suricata use? how many rules do you load?<br>
><br>
> thank you<br>
><br>
> On Fri, Nov 30, 2012 at 3:50 AM, xbadou xbadou <<a href="mailto:xbadou@gmail.com">xbadou@gmail.com</a><br>
</div><div class="im">> <mailto:<a href="mailto:xbadou@gmail.com">xbadou@gmail.com</a>>> wrote:<br>
><br>
> Thank you very much.<br>
><br>
> But I want to known, whether I can do something to limit the max<br>
> memory usage of suricata. Because I just have very small<br>
> network traffic. I think 4 GB is maybe enough to me. I just<br>
> want suricata keep alive if it can't get more memory. Or<br>
> suricata do some memory clean jobs if it can't allocate more memory.<br>
><br>
> If suricata get a segfault very offen, I think I may need a<br>
> watchdog to watch this and restart it.<br>
><br>
><br>
> On Fri, Nov 30, 2012 at 10:26 AM, Marcos Rodriguez<br>
> <<a href="mailto:marcos.e.rodriguez@gmail.com">marcos.e.rodriguez@gmail.com</a><br>
</div><div class="im">> <mailto:<a href="mailto:marcos.e.rodriguez@gmail.com">marcos.e.rodriguez@gmail.com</a>>> wrote:<br>
><br>
><br>
><br>
> On Thu, Nov 29, 2012 at 8:23 PM, xbadou xbadou<br>
</div><div><div class="h5">> <<a href="mailto:xbadou@gmail.com">xbadou@gmail.com</a> <mailto:<a href="mailto:xbadou@gmail.com">xbadou@gmail.com</a>>> wrote:<br>
><br>
> Yes, I am running debian 5 with kernel 2.6.31.14<br>
> 32bit。 And the system ram size is 2GB*2.<br>
><br>
> So, if it is really this issue. How can I avoid this<br>
> coredump happen? Can I change some settings in the<br>
> suricata.yaml file?<br>
><br>
> Thanks.<br>
><br>
><br>
><br>
> At the bottom of the suricata.yaml file, you'll find this<br>
> section:<br>
><br>
> # Suricata core dump configuration. Limits the size of the<br>
> core dump file to<br>
> # approximately max-dump. The actual core dump size will be<br>
> a multiple of the<br>
> # page size. Core dumps that would be larger than max-dump<br>
> are truncated. On<br>
> # Linux, the actual core dump size may be a few pages larger<br>
> than max-dump.<br>
> # Setting max-dump to 0 disables core dumping.<br>
> # Setting max-dump to 'unlimited' will give the full core<br>
> dump file.<br>
> # On 32-bit Linux, a max-dump value >= ULONG_MAX may cause<br>
> the core dump size<br>
> # to be 'unlimited'.<br>
><br>
> coredump:<br>
> max-dump: unlimited<br>
><br>
> Change the max-dump to 0 to disable. :o)<br>
><br>
> marcos<br>
><br>
><br>
><br>
> _______________________________________________<br>
> Suricata IDS Devel mailing list:<br>
> <a href="mailto:oisf-devel@openinfosecfoundation.org">oisf-devel@openinfosecfoundation.org</a><br>
</div></div>> <mailto:<a href="mailto:oisf-devel@openinfosecfoundation.org">oisf-devel@openinfosecfoundation.org</a>><br>
<div class="im HOEnZb">> Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Participate:<br>
> <a href="http://suricata-ids.org/participate/" target="_blank">http://suricata-ids.org/participate/</a><br>
> List:<br>
> <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel</a><br>
> Redmine: <a href="https://redmine.openinfosecfoundation.org/" target="_blank">https://redmine.openinfosecfoundation.org/</a><br>
><br>
><br>
><br>
><br>
> --<br>
> Regards,<br>
> Peter Manev<br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Suricata IDS Devel mailing list: <a href="mailto:oisf-devel@openinfosecfoundation.org">oisf-devel@openinfosecfoundation.org</a><br>
> Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Participate: <a href="http://suricata-ids.org/participate/" target="_blank">http://suricata-ids.org/participate/</a><br>
> List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel</a><br>
> Redmine: <a href="https://redmine.openinfosecfoundation.org/" target="_blank">https://redmine.openinfosecfoundation.org/</a><br>
><br>
<br>
<br>
--<br>
</div><div class="im HOEnZb">---------------------------------------------<br>
Victor Julien<br>
<a href="http://www.inliniac.net/" target="_blank">http://www.inliniac.net/</a><br>
PGP: <a href="http://www.inliniac.net/victorjulien.asc" target="_blank">http://www.inliniac.net/victorjulien.asc</a><br>
---------------------------------------------<br>
<br>
</div><div class="HOEnZb"><div class="h5">_______________________________________________<br>
Suricata IDS Devel mailing list: <a href="mailto:oisf-devel@openinfosecfoundation.org">oisf-devel@openinfosecfoundation.org</a><br>
Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Participate: <a href="http://suricata-ids.org/participate/" target="_blank">http://suricata-ids.org/participate/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel</a><br>
Redmine: <a href="https://redmine.openinfosecfoundation.org/" target="_blank">https://redmine.openinfosecfoundation.org/</a></div></div></blockquote></div><br>