<div dir="ltr"><div>Thanks Victor/Cooper:</div><div><br></div><div>> If you have 48G of mem, I think you can use a lot more here. Like 16G or<br> something.</div><div><br></div><div>I used a large number like this initially, however, things eventually got chaotic and the application ran out of memory. This lead me to believe that it was allowing the cap on a per PF_RING thread basis, rather than as an aggregate (though I could be totally wrong :) )</div>
<div><br></div><div>Important to note that with 1.5gb cap I was at 55.8% mem utilization earlier over a ~12 hr period.</div><div><br></div><div> > As stream depth is 5mb, setting 12mb here doesn't really affect anything<br>
I think. The stream depth cuts stream tracking at 5mb regardless of the<br> setting here.</div><div><br></div><div>Noted. For some reason I thought it treated reassembly for that traffic differently if that setting was invoked. </div>
<div><br></div><div>> Also, if you have multiple vlans on the network, it may be worth trying<br> to disable.</div><div><br></div><div>We do, and I did do an earlier test run setting this just to false. To no avail however, as the issues continued. </div>
<div><br></div><div>> In 2.0.1 the stream engine should use less memory and clear memory<br> quicker. Could you try 2.0.1?</div><div><br></div><div>Just installed and am working with this now. Unfortunately, tcp.reassembly_memuse seems to continue to slowly increase. Below is some data taken from a recent test over the course of 35 minutes with the same data rates (only with 2.0.1). Reassem memcap is set to 3gb with this run. Stats.log output is every 10 seconds so I picked out every 3rd record on a random thread. </div>
<div><br></div><div>grep RxPFRp4p210 stats.log | grep reassembly_memuse | awk '!(NR%3)'<br>tcp.reassembly_memuse | RxPFRp4p210 | 154471336<br>tcp.reassembly_memuse | RxPFRp4p210 | 243540732<br>
tcp.reassembly_memuse | RxPFRp4p210 | 301571602<br>tcp.reassembly_memuse | RxPFRp4p210 | 314913802<br>tcp.reassembly_memuse | RxPFRp4p210 | 285059970<br>tcp.reassembly_memuse | RxPFRp4p210 | 272284065<br>
tcp.reassembly_memuse | RxPFRp4p210 | 254263605<br>tcp.reassembly_memuse | RxPFRp4p210 | 253255657<br>tcp.reassembly_memuse | RxPFRp4p210 | 272511595<br>tcp.reassembly_memuse | RxPFRp4p210 | 296499090<br>
tcp.reassembly_memuse | RxPFRp4p210 | 311359535<br>tcp.reassembly_memuse | RxPFRp4p210 | 283427074<br>tcp.reassembly_memuse | RxPFRp4p210 | 255654055<br>tcp.reassembly_memuse | RxPFRp4p210 | 270931311<br>
tcp.reassembly_memuse | RxPFRp4p210 | 299100487<br>tcp.reassembly_memuse | RxPFRp4p210 | 291949643<br>tcp.reassembly_memuse | RxPFRp4p210 | 293440732<br>tcp.reassembly_memuse | RxPFRp4p210 | 290335664<br>
tcp.reassembly_memuse | RxPFRp4p210 | 269864084<br>tcp.reassembly_memuse | RxPFRp4p210 | 267911989<br>tcp.reassembly_memuse | RxPFRp4p210 | 276945873<br>tcp.reassembly_memuse | RxPFRp4p210 | 283443241<br>
tcp.reassembly_memuse | RxPFRp4p210 | 345384994<br>tcp.reassembly_memuse | RxPFRp4p210 | 347958227<br>tcp.reassembly_memuse | RxPFRp4p210 | 330984680<br>tcp.reassembly_memuse | RxPFRp4p210 | 305309240<br>
tcp.reassembly_memuse | RxPFRp4p210 | 274026280<br>tcp.reassembly_memuse | RxPFRp4p210 | 328726738<br>tcp.reassembly_memuse | RxPFRp4p210 | 369907025<br>tcp.reassembly_memuse | RxPFRp4p210 | 352353094<br>
tcp.reassembly_memuse | RxPFRp4p210 | 333345579<br>tcp.reassembly_memuse | RxPFRp4p210 | 306374595<br>tcp.reassembly_memuse | RxPFRp4p210 | 277904734<br>tcp.reassembly_memuse | RxPFRp4p210 | 298247168<br>
tcp.reassembly_memuse | RxPFRp4p210 | 301306632<br>tcp.reassembly_memuse | RxPFRp4p210 | 266160428<br>tcp.reassembly_memuse | RxPFRp4p210 | 307500851<br>tcp.reassembly_memuse | RxPFRp4p210 | 307852499<br>
tcp.reassembly_memuse | RxPFRp4p210 | 290149127<br>tcp.reassembly_memuse | RxPFRp4p210 | 306533111<br>tcp.reassembly_memuse | RxPFRp4p210 | 291460952<br>tcp.reassembly_memuse | RxPFRp4p210 | 288518673<br>
tcp.reassembly_memuse | RxPFRp4p210 | 324852777<br>tcp.reassembly_memuse | RxPFRp4p210 | 302582777<br>tcp.reassembly_memuse | RxPFRp4p210 | 293625604<br>tcp.reassembly_memuse | RxPFRp4p210 | 307370770<br>
tcp.reassembly_memuse | RxPFRp4p210 | 311457210<br>tcp.reassembly_memuse | RxPFRp4p210 | 311108835<br>tcp.reassembly_memuse | RxPFRp4p210 | 320206099<br>tcp.reassembly_memuse | RxPFRp4p210 | 307685170<br>
tcp.reassembly_memuse | RxPFRp4p210 | 282411341<br>tcp.reassembly_memuse | RxPFRp4p210 | 288700701<br>tcp.reassembly_memuse | RxPFRp4p210 | 306848861<br>tcp.reassembly_memuse | RxPFRp4p210 | 281831829<br>
tcp.reassembly_memuse | RxPFRp4p210 | 292346555<br>tcp.reassembly_memuse | RxPFRp4p210 | 314350370<br>tcp.reassembly_memuse | RxPFRp4p210 | 286442727<br>tcp.reassembly_memuse | RxPFRp4p210 | 283934879<br>
tcp.reassembly_memuse | RxPFRp4p210 | 270730695<br>tcp.reassembly_memuse | RxPFRp4p210 | 269700704<br>tcp.reassembly_memuse | RxPFRp4p210 | 327235776<br>tcp.reassembly_memuse | RxPFRp4p210 | 326038648<br>
tcp.reassembly_memuse | RxPFRp4p210 | 359062072<br>tcp.reassembly_memuse | RxPFRp4p210 | 359215420<br>tcp.reassembly_memuse | RxPFRp4p210 | 371124232<br>tcp.reassembly_memuse | RxPFRp4p210 | 379226360<br>
tcp.reassembly_memuse | RxPFRp4p210 | 411893212<br>tcp.reassembly_memuse | RxPFRp4p210 | 429465136<br>tcp.reassembly_memuse | RxPFRp4p210 | 452973020<br>tcp.reassembly_memuse | RxPFRp4p210 | 464045120</div>
<div><br></div><div>Since reassembly memcap is not reached (yet) and pf_ring can't keep up, our number of free slots for pf_ring threads is continuously at 0, which has amplifying effects concerning packet loss.</div>
<div><br></div><div>grep capture.kernel_ stats.log | tail -n 10<br>capture.kernel_packets | RxPFRp4p212 | 57951223<br>capture.kernel_drops | RxPFRp4p212 | 5797512<br>capture.kernel_packets | RxPFRp4p213 | 70292808<br>
capture.kernel_drops | RxPFRp4p213 | 7525246<br>capture.kernel_packets | RxPFRp4p214 | 55188744<br>capture.kernel_drops | RxPFRp4p214 | 3877128<br>capture.kernel_packets | RxPFRp4p215 | 57682274<br>
capture.kernel_drops | RxPFRp4p215 | 5057610<br>capture.kernel_packets | RxPFRp4p216 | 55113863<br>capture.kernel_drops | RxPFRp4p216 | 4016017<br></div><div><br></div>
<div>> Are you stuck with PF_RING for some reason?</div><div><br></div><div>I am for the reasons Victor cited :)<br></div><div><br></div><div>Thanks all for the input so far. I am really hoping to get this working! If there is any more information I can give to help please let me know.</div>
<div><br></div><div>Thanks,</div><div>Jason</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, May 30, 2014 at 12:17 PM, Victor Julien <span dir="ltr"><<a href="mailto:lists@inliniac.net" target="_blank">lists@inliniac.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 05/30/2014 06:31 PM, Jason Batchelor wrote:<br>
> Hello,<br>
><br>
> I am having some issues with file extraction in Suricata, and after<br>
> attempting to do many optimizations and review of others experiences I<br>
> am still finding myself out of luck. Below is some verbose output of my<br>
> current configuration and some sample data after ~12 hours of running. I<br>
> have also included a smaller time frame with a few subtle changes for<br>
> consideration.<br>
><br>
> Under this configuration the rule I have, which is an IP based rule below...<br>
><br>
> alert http any any -> $MY_LOCAL_IP any (msg:"FILE PDF"; filemagic:"PDF<br>
> document"; filestore; sid:1; rev:1;)<br>
><br>
> Does not trigger at all when the reassembly mem cap is reached. Even<br>
> when it does (when reassembly memcap is below the threshold), I get<br>
> truncated PDFs. I have tried adjusting things like the reassembly<br>
> memcap, however, when I do, I very quickly run into a large amount of<br>
> packet loss because the number of free slots PF_RING can issue is not<br>
> able to keep up (details below). Additionally, reassembly mem cap seems<br>
> to slowly increase over time, eventually reaching its peak before the<br>
> number of free ring slots can finally keep up (presumably due to<br>
> segments being dropped).<br>
><br>
> I have struggled playing with time out values as well really to no avail<br>
> (details below).<br>
><br>
> When I turn http logging on, I do see the website that I go to being<br>
> properly logged fwiw.<br>
><br>
> I feel like I must be doing something wrong, or I am not seeing<br>
> something obvious. After reviewing many blogs and howtos, it seems folks<br>
> are able to do what I am trying to accomplish with the same (sometime<br>
> more) data rates and much less hardware.<br>
><br>
> I have tried the following:<br>
> - Increased min_num_slots to 65534 for PF_RING<br>
> - Tinkered with TCP timeout settings<br>
> - Adjusted reassembly memcap<br>
><br>
> Kindly take a look at the details I have listed below and let me know if<br>
> there is anything you can suggest. I am curious if I am just plain at<br>
> the limit of my hardware and need to consider upgrading and/or getting<br>
> PF_RING with DNA. Or, perhaps there are a few more items I should<br>
> consider within the application itself.<br>
><br>
> One final thing to consider, would tcp sequence randomization<br>
> significantly impact things? I would need to get in touch with the folks<br>
> responsible to see if we have this on but thought I would ask here as well!<br>
><br>
> Many thanks in advance for your time looking at this!<br>
><br>
> == Profile ==<br>
><br>
> CentOS 6.5 Linux<br>
> Kernel 2.6.32-431.11.2.el6.x86_64<br>
><br>
> Installed Suricata 2.0 and PF_RING 6.0.1 from source.<br>
><br>
> Machine sees ~400MB/s at peek load.<br>
><br>
> == Tuning ==<br>
><br>
> I've tuned the ixgbe NIC with the following settings...<br>
><br>
> ethtool -K p4p2 tso off<br>
> ethtool -K p4p2 gro off<br>
> ethtool -K p4p2 lro off<br>
> ethtool -K p4p2 gso off<br>
> ethtool -K p4p2 rx off<br>
> ethtool -K p4p2 tx off<br>
> ethtool -K p4p2 sg off<br>
> ethtool -K p4p2 rxvlan off<br>
> ethtool -K p4p2 txvlan off<br>
> ethtool -N p4p2 rx-flow-hash udp4 sdfn<br>
> ethtool -N p4p2 rx-flow-hash udp6 sdfn<br>
> ethtool -n p4p2 rx-flow-hash udp6<br>
> ethtool -n p4p2 rx-flow-hash udp4<br>
> ethtool -C p4p2 rx-usecs 1000<br>
> ethtool -C p4p2 adaptive-rx off<br>
><br>
> It is also using the latest driver available. I have also tried to<br>
> optimize things in the sysctl.conf<br>
><br>
> # -- 10gbe tuning from Intel ixgb driver README -- #<br>
><br>
> # turn off selective ACK and timestamps<br>
> net.ipv4.tcp_sack = 0<br>
> net.ipv4.tcp_timestamps = 0<br>
><br>
> # memory allocation min/pressure/max.<br>
> # read buffer, write buffer, and buffer space<br>
> net.ipv4.tcp_rmem = 10000000 10000000 10000000<br>
> net.ipv4.tcp_wmem = 10000000 10000000 10000000<br>
> net.ipv4.tcp_mem = 10000000 10000000 10000000<br>
><br>
> net.core.rmem_max = 524287<br>
> net.core.wmem_max = 524287<br>
> net.core.rmem_default = 524287<br>
> net.core.wmem_default = 524287<br>
> net.core.optmem_max = 524287<br>
> net.core.netdev_max_backlog = 300000<br>
><br>
> == Hardware Specs ==<br>
> CPU: Intel Xeon CPU @ 2.40Ghz x 32<br>
> RAM: 48G<br>
> NIC:<br>
> *-network:1<br>
> description: Ethernet interface<br>
> product: Ethernet 10G 2P X520 Adapter<br>
> vendor: Intel Corporation<br>
> physical id: 0.1<br>
> bus info: pci@0000:42:00.1<br>
> logical name: p4p2<br>
> version: 01<br>
> serial: a0:36:9f:07:ec:02<br>
> capacity: 1GB/s<br>
> width: 64 bits<br>
> clock: 33MHz<br>
> capabilities: pm msi msix pciexpress vpd bus_master cap_list rom<br>
> ethernet physical fibre 1000bt-fd autonegotiation<br>
> configuration: autonegotiation=on broadcast=yes driver=ixgbe<br>
> driverversion=3.21.2 duplex=full firmware=0x8000030d latency=0 link=yes<br>
> multicast=yes port=fibre promiscuous=yes<br>
> resources: irq:76 memory:d0f00000-d0f7ffff ioport:7ce0(size=32)<br>
> memory:d0ffc000-d0ffffff memory:d1100000-d117ffff(prefetchable)<br>
> memory:d1380000-d147ffff(prefetchable)<br>
> memory:d1480000-d157ffff(prefetchable)<br>
><br>
> == Suricata Config ==<br>
> Below are some details that may be relevant...<br>
><br>
> runmode: workers<br>
><br>
> host-mode: sniffer-only<br>
><br>
> default-packet-size: 9000<br>
><br>
> - file-store:<br>
> enabled: yes # set to yes to enable<br>
> log-dir: files # directory to store the files<br>
> force-magic: yes # force logging magic on all stored files<br>
> force-md5: yes # force logging of md5 checksums<br>
> waldo: file.waldo # waldo file to store the file_id across runs<br>
><br>
> defrag:<br>
> memcap: 512mb<br>
> hash-size: 65536<br>
> trackers: 65535 # number of defragmented flows to follow<br>
> max-frags: 65535 # number of fragments to keep (higher than trackers)<br>
> prealloc: yes<br>
> timeout: 30<br>
><br>
> flow:<br>
> memcap: 1gb<br>
> hash-size: 1048576<br>
> prealloc: 1048576<br>
> emergency-recovery: 30<br>
><br>
> flow-timeouts:<br>
> default:<br>
> new: 1<br>
> established: 5<br>
> closed: 0<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
> emergency-closed: 0<br>
> tcp:<br>
> new: 15<br>
> established: 100<br>
> closed: 5<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
> emergency-closed: 0<br>
> udp:<br>
> new: 5<br>
> established: 10<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
> icmp:<br>
> new: 1<br>
> established: 5<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
><br>
> stream:<br>
> memcap: 10gb<br>
<br>
</div></div>This is excessive, although it won't hurt.<br>
<div><br>
> checksum-validation: no # reject wrong csums<br>
> prealloc-sesions: 500000<br>
> midstream: false<br>
> asyn-oneside: false<br>
> inline: no # auto will use inline mode in IPS<br>
> mode, yes or no set it statically<br>
> reassembly:<br>
> memcap: 1.5gb<br>
<br>
</div>If you have 48G of mem, I think you can use a lot more here. Like 16G or<br>
something.<br>
<div><br>
> depth: 5mb<br>
> toserver-chunk-size: 2560<br>
> toclient-chunk-size: 2560<br>
> randomize-chunk-size: yes<br>
><br>
> host:<br>
> hash-size: 4096<br>
> prealloc: 1000<br>
> memcap: 16777216<br>
><br>
><br>
> pfring:<br>
> - interface: p4p2<br>
> threads: 16<br>
> cluster-id: 99<br>
> cluster-type: cluster_flow<br>
> checksum-checks: no<br>
> - interface: default<br>
><br>
> http:<br>
> enabled: yes<br>
> libhtp:<br>
> default-config:<br>
> personality: IDS<br>
><br>
> # Can be specified in kb, mb, gb. Just a number indicates<br>
> # it's in bytes.<br>
> request-body-limit: 12mb<br>
> response-body-limit: 12mb<br>
<br>
</div>As stream depth is 5mb, setting 12mb here doesn't really affect anything<br>
I think. The stream depth cuts stream tracking at 5mb regardless of the<br>
setting here.<br>
<div><div class="h5"><br>
><br>
> == ~12 hours (above config) =<br>
><br>
> top - 14:58:59 up 18:23, 3 users, load average: 6.44, 4.83, 4.32<br>
> Tasks: 664 total, 1 running, 663 sleeping, 0 stopped, 0 zombie<br>
> Cpu(s): 17.9%us, 0.1%sy, 0.0%ni, 80.3%id, 0.0%wa, 0.0%hi, 1.7%si,<br>
> 0.0%st<br>
> Mem: 49376004k total, 29289768k used, 20086236k free, 68340k buffers<br>
> Swap: 2621432k total, 0k used, 2621432k free, 820172k cached<br>
><br>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND<br>
> 17616 root 20 0 27.0g 26g 16g S 621.4 55.8 3532:51 Suricata-Main<br>
><br>
> watch 'cat /proc/net/pf_ring/*p4p2* | egrep "Num Free Slots|Tot<br>
> Packets|Tot Pkt Lost"'<br>
> ; First three threads...<br>
> Tot Packets : 627370957<br>
> Tot Pkt Lost : 3582014<br>
> Num Free Slots : 118705<br>
> Tot Packets : 676753767<br>
> Tot Pkt Lost : 5292092<br>
> Num Free Slots : 118745<br>
> Tot Packets : 665348839<br>
> Tot Pkt Lost : 3841911<br>
> Num Free Slots : 118677<br>
> ...<br>
><br>
> watch -n 10 'cat stats.log | egrep<br>
> "reassembly_memuse|segment_memcap_drop" | tail -n 32'<br>
> ; First three threads...<br>
> tcp.segment_memcap_drop | RxPFRp4p21 | 25782329<br>
> tcp.reassembly_memuse | RxPFRp4p21 | 1610612705<br>
> tcp.segment_memcap_drop | RxPFRp4p22 | 26161478<br>
> tcp.reassembly_memuse | RxPFRp4p22 | 1610612705<br>
> tcp.segment_memcap_drop | RxPFRp4p23 | 25813867<br>
> tcp.reassembly_memuse | RxPFRp4p23 | 1610612705<br>
><br>
> grep 'reassembly_gap' stats.log | tail -n 10<br>
> tcp.reassembly_gap | RxPFRp4p27 | 777366<br>
> tcp.reassembly_gap | RxPFRp4p28 | 774896<br>
> tcp.reassembly_gap | RxPFRp4p29 | 781761<br>
> tcp.reassembly_gap | RxPFRp4p210 | 776427<br>
> tcp.reassembly_gap | RxPFRp4p211 | 778734<br>
> tcp.reassembly_gap | RxPFRp4p212 | 773203<br>
> tcp.reassembly_gap | RxPFRp4p213 | 781125<br>
> tcp.reassembly_gap | RxPFRp4p214 | 776043<br>
> tcp.reassembly_gap | RxPFRp4p215 | 781790<br>
> tcp.reassembly_gap | RxPFRp4p216 | 783368<br>
><br>
> == PF RING ==<br>
><br>
> PF_RING Version : 6.0.1 ($Revision: exported$)<br>
> Total rings : 16<br>
><br>
> Standard (non DNA) Options<br>
> Ring slots : 65534<br>
> Slot version : 15<br>
> Capture TX : No [RX only]<br>
> IP Defragment : No<br>
> Socket Mode : Standard<br>
> Transparent mode : Yes [mode 0]<br>
> Total plugins : 0<br>
> Cluster Fragment Queue : 9175<br>
> Cluster Fragment Discard : 597999<br>
><br>
> == ~30 min (with changes) ==<br>
><br>
> FWIW, when I increase reassembly memcap and time outs to the following...<br>
><br>
> flow-timeouts:<br>
> default:<br>
> new: 5<br>
> established: 50<br>
> closed: 0<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
> emergency-closed: 0<br>
> tcp:<br>
> new: 15<br>
> established: 100<br>
> closed: 10<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
> emergency-closed: 0<br>
> udp:<br>
> new: 5<br>
> established: 50<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
> icmp:<br>
> new: 1<br>
> established: 5<br>
> emergency-new: 1<br>
> emergency-established: 1<br>
><br>
> reassembly:<br>
> memcap: 3gb<br>
> depth: 5mb<br>
><br>
> These are the results, note how there are no more free slots for<br>
> PF_RING. I believe this results in increased packet loss... which is<br>
> likely resulting in my truncated files that I receive when I pull a PDF.<br>
><br>
> watch 'cat /proc/net/pf_ring/*p4p2* | egrep "Num Free Slots|Tot<br>
> Packets|Tot Pkt Lost"'<br>
> ; First three threads...<br>
> Tot Packets : 80281541<br>
> Tot Pkt Lost : 44290194<br>
> Num Free Slots : 0<br>
> Tot Packets : 81926241<br>
> Tot Pkt Lost : 17412402<br>
> Num Free Slots : 0<br>
> Tot Packets : 80108557<br>
> Tot Pkt Lost : 14667061<br>
> Num Free Slots : 0<br>
><br>
> watch -n 10 'cat stats.log | egrep<br>
> "reassembly_memuse|segment_memcap_drop" | tail -n 32'<br>
> ; First three threads...<br>
> tcp.segment_memcap_drop | RxPFRp4p21 | 0<br>
> tcp.reassembly_memuse | RxPFRp4p21 | 1681598708<br>
> tcp.segment_memcap_drop | RxPFRp4p22 | 0<br>
> tcp.reassembly_memuse | RxPFRp4p22 | 1681626644<br>
> tcp.segment_memcap_drop | RxPFRp4p23 | 0<br>
> tcp.reassembly_memuse | RxPFRp4p23 | 1681597556<br>
> tcp.segment_memcap_drop | RxPFRp4p24 | 0<br>
> *** Important to note here, the reassembly memuse seems to steadily<br>
> increase overtime. After a few minutes of putting this in it has risen<br>
> to 2022140776 across. Makes me think things are not offloading<br>
> quickly... (timeout/depth issue?)<br>
><br>
> grep 'reassembly_gap' stats.log | tail -n 10<br>
> tcp.reassembly_gap | RxPFRp4p27 | 27603<br>
> tcp.reassembly_gap | RxPFRp4p28 | 26677<br>
> tcp.reassembly_gap | RxPFRp4p29 | 26869<br>
> tcp.reassembly_gap | RxPFRp4p210 | 25031<br>
> tcp.reassembly_gap | RxPFRp4p211 | 23988<br>
> tcp.reassembly_gap | RxPFRp4p212 | 23809<br>
> tcp.reassembly_gap | RxPFRp4p213 | 26420<br>
> tcp.reassembly_gap | RxPFRp4p214 | 25271<br>
> tcp.reassembly_gap | RxPFRp4p215 | 26285<br>
> tcp.reassembly_gap | RxPFRp4p216 | 26848<br>
<br>
</div></div>In 2.0.1 the stream engine should use less memory and clear memory<br>
quicker. Could you try 2.0.1?<br>
<br>
Also, if you have multiple vlans on the network, it may be worth trying<br>
to disable:<br>
<br>
vlan:<br>
use-for-tracking: true<br>
<br>
I think you've probably checked all or most things on them, but perhaps<br>
these diagrams here can be of some help here:<br>
<a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Self_Help_Diagrams" target="_blank">https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Self_Help_Diagrams</a><br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
---------------------------------------------<br>
Victor Julien<br>
<a href="http://www.inliniac.net/" target="_blank">http://www.inliniac.net/</a><br>
PGP: <a href="http://www.inliniac.net/victorjulien.asc" target="_blank">http://www.inliniac.net/victorjulien.asc</a><br>
---------------------------------------------<br>
<br>
_______________________________________________<br>
Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" target="_blank">http://suricata-ids.org/support/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
OISF: <a href="http://www.openinfosecfoundation.org/" target="_blank">http://www.openinfosecfoundation.org/</a><br>
</font></span></blockquote></div><br></div>