[Oisf-users] (no subject)

[Christophe Vandeplas]

Hi,


What kind of drop do you have?
- capture.kernel_drops
- tcp.segment_memcap_drop
- tcp.ssn_memcap_drop

Lower the number of threads in the af-packet section to the number of
cores your system has. (cat /proc/cpuinfo |  fgrep processor | wc -l )

Run suricata with no rules, and tweak the configuration, you should
have (almost) no packet drop before you activate rules.

After having made changes in the yaml configuration file I usually:
- stop suricata
- empty the logfiles
- start suricata
This way there's no risk of looking at older logs and misinterpreting
configuration changes.


If possible, link your stats.log to a monitoring tool to greate
graphs. This way you can correlate packet drops by suricata with other
events on the system. I've written an article about this :
http://christophe.vandeplas.com/2013/11/suricata-monitoring-with-zabbix-or-other.html
 But also other scripts exist.
Make sure you edit the suricata_stats.py script with the number of
threads configured in suricata.yaml


If your drops are capture.kernel_drops, then :
Have you read this article?
http://christophe.vandeplas.com/2013/11/suricata-capturekerneldrops-caused-by.html
Please do the first part "Confirmation of the problem" and see if you
also have the problem caused by the lack of NIC queues.
In a few words:
- start suricata
- as root, run  "top -H" and check how many AFPacketethXX threads are
generating load.
- if it's only one thread, then the problem has been pinpointed.
However working with cluster_flow should solve this problem. Make sure
you read the rest of the article then.


Kind regards
Christophe




On Sun, Jun 8, 2014 at 9:56 AM, X.qing <xqing.summer at gmail.com> wrote:
> It is really a great surprise for me to hear from you, first of all, thank
> you very much!
>
> i have changed cluster_cpu to cluster_flow as you command, and it works.
> however, it seems that only add threads does not help much in my system.
> After 1 hour's test, i still get about 50% drops .
> The basic configuration is followed by your a series of 4 articles about
> Suricata
> IDPS(http://pevma.blogspot.se/2013/12/suricata-and-grand-slam-of-open-source.html)
> and followed the case 4 in
> (http://pevma.blogspot.se/2014/05/playing-with-memory-consumption.html).
>
> These are the basic imformation about my system:
> Suricata version 2.0.1 with AF_PACKET, 22 threads
> CPU: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
> Kenel: Linux version 3.11.0-15-generic
> OS: Ubuntu 12.04.4
> RAM: 125G
> 72 rules loaded for test.
> Traffic size: 2-4Gps
>
> If it doesn't bother you too much, i have uploaded my configuration file in
> the attachment for you to check, i am eager to receive more suggestion from
> you.
>
> Thanks again.
> Best wishes.
>>
>>
>>
>>
>> 2014-06-05 15:27 GMT+08:00 Peter Manev <petermanev at gmail.com>:
>>
>>> The Reason the extra threads are not used is because you hvae set up
>>> the NIC affinity and it has only 16 irq-affinity threads.
>>>
>>> You can change the af-packet section form cluster_cpu to cluster_flow
>>> and they will be used, though i am not sure how it will affect
>>> performance in your case.
>>>
>>>
>>> The fact that you set up the yaml config just as it is on the blog
>>> does not guarantee you 0% drops.
>>>
>>> It depends on how much traffic do you inspect, how many rules do you
>>> load, what type oof traffic it is, ...
>>>
>>> Which blogpost did you follow in particular?
>>> What is the OS/kernel you are using?
>>> What amount of traffic do you inspect?
>>> How many rules do you load?
>>>
>>> thanks
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jun 5, 2014 at 6:21 AM, Blogger Contact Form
>>> <no-reply at blogger.com> wrote:
>>> > i've configured the suricata.yaml as you suggested above, but i still
>>> > get
>>> > about 60% drops. do u have any other suggestions?
>>> > i  intend to add the threads to improve the performance, so i only
>>> > change
>>> > the 'threads' in 'af-packet' to 22(the default is 16), but when i check
>>> > in
>>> > stats.log, the 17-22 packet is not used. do i missed to change any
>>> > other
>>> > parameter to change?
>>> > thanx :)
>>> >
>>> > Regards,
>>> > Tomato- | xqing.summer at gmail.com
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/