[Oisf-users] Tuning Suricata Inline IPS performance
Delta Yeh
delta.yeh at gmail.com
Thu Dec 15 00:20:57 EST 2011
Suricata IPS is nfq, IDS is pfring or pcap.
I want to know whether someone run suricata IPS in production , and what
is the performance of his setup.
2011/12/15 Hariharan Thantry <thantry at gmail.com>:
> Hi Delta,
>
>> When run suricata with pfring, with pfring bpf filter "tcp", I can
>> achive 700Mb(according to load runner result)
>>
>
> Does pfring allow you to run Suricata in IPS mode? I assume that the
> only socket option available for IPS is NFQUEUE.
>
> BTW, it might help to increase your ethernet MTU. I set mine for the
> 10G network at 16000. This results in lower # of packets, and hence
> better rates.
> Also, try to turn off autoneg/pause on your ethernet interface (using
> ethtool). You don't want your ethernet bridge interface back
> pressuring the send side ethernet interface.
> Inspite of all this, I get ~ 150 Mbps on a 10G link using NFQUEUE
> (single threaded, I'm trying an MT implementation). This is with
> bridge just doing an NF_ACCEPT on every packet,
>
> Fundamentally, I think this is a netlink_nfqueue socket library issue.
> I don't understand the tuning parameters there too well...will post if
> things get any better.
>
> Thanks,
> Hari
>
>
> On Wed, Dec 14, 2011 at 7:48 PM, Delta Yeh <delta.yeh at gmail.com> wrote:
>> Hi Eric,
>>
>> My box is 32 bit debian squeeze with 4G mem , dual core 2.7G, work as a bridge
>>
>> between test client and web server.
>>
>> In my tests , I didn't load any rule, and use load runner to test
>> http traffic only.
>>
>> For suricata nfq , I got similar result, about 1.5Mbps .
>>
>> I referered to
>> http://www.inliniac.net/blog/2008/01/23/improving-snort_inlines-nfq-performance.html
>> The command are:
>>
>> sysctl -w net.core.rmem_default=’8388608′
>> sysctl -w net.core.wmem_default=’8388608′
>>
>> sysctl -w net.ipv4.tcp_wmem=’1048576 4194304 16777216′
>> sysctl -w net.ipv4.tcp_rmem=’1048576 4194304 16777216′
>>
>> iptables -t mangle -F FORWARD
>> iptables -t mangle -I FORWARD -p tcp -s webserver -j NFQUEUE
>> --queue-balance 0:1
>>
>> iptables -t mangle -I FORWARD -p tcp -d webserver -j NFQUEUE
>> --queue-balance 0:1
>>
>> suricata.yaml max-pending-packets is 5000
>>
>> I run suricata with " -c /etc/suricata/suricata.yaml -q 0 -q 1
>> --runmode worker"
>>
>> I didn't test cpu affinity .
>>
>> When run suricata with pfring, with pfring bpf filter "tcp", I can
>> achive 700Mb(according to load runner result)
>>
>> During test, the suricata cpu is 70%, load runner send 860,000 request,
>> but suricata only record 310,000 request.
>>
>>
>> 2011/11/23 Eric Leblond <eric at regit.org>:
>>> Hello,
>>>
>>> Here's a list of things you can do to improve performance:
>>> * use multiple queues: you can then use the multithreading
>>> capability of your boxe and avoid some per-cpu lock.
>>> * Increase netfilter queue length: you will then be able to resist
>>> better to any burst effect
>>> * Uses NFQ in worker mode: this mode were one thread does all the
>>> work since capture to decision should be the most efficient for
>>> NFQ (currently not in official tree, see patch provided).
>>>
>>> Let's you've got a multicore system with 4 CPUs.
>>>
>>> On Netfilter side:
>>> iptables -A FORWARD -j NFQUEUE --queue-balance 0:3
>>>
>>> This will balance the packet between queue 0 to 3 with per-connection
>>> load-balancing. Please note that in this case your injection tool must
>>> be multi connection aware.
>>>
>>> Increase max_pending_packets in suricata.yaml to a decent value like
>>> 1000 (this will use "some" memory).
>>>
>>> Start suricata with:
>>> suricata -c suricata.yaml -q 0 -q 1 -q 2 -q 3
>>>
>>> If you are able to compile a suricata on your system, you can use
>>> current git tree, apply the attached patch and run:
>>> suricata -c suricata.yaml -q 0 -q 1 -q 2 -q 3 --runmode workers
>>>
>>> Next thing that can also be done is to work on cpu affinity to
>>> synchronize CPU used for capture and treatment in Suricata.
>>>
>>> Please let us know how things are improving with that.
>>>
>>> BR,
>>>
>>> On Mon, 2011-11-21 at 17:48 -0800, Hariharan Thantry wrote:
>>>> Hi Victor,
>>>>
>>>>
>>>> I think this is not necessarily because of Suricata itself, but by the
>>>> use of iptables/NFQUEUE in a purely bridged environment. (The Suricata
>>>> IPS does not have an IP address for the bridge). I used the very
>>>> simple NFQUEUE user space
>>>> handler http://www.netfilter.org/projects/libnetfilter_queue/doxygen/nfqnl__test_8c_source.html, stopped Suricata, and kept the following iptables entry
>>>>
>>>>
>>>> $ sudo iptables -A FORWARD -j NFQUEUE --queue-num 0
>>>>
>>>>
>>>> and used the above program (which just puts the packet back out) on my
>>>> bridge machine, and observed the same throughput speeds (~ 400 Kbps)
>>>> using iperf. (Only a single connection activated)
>>>>
>>>>
>>>> Interestingly, when I used ebtables, and its handler
>>>> (ulog) http://ebtables.sourceforge.net/examples/basic.html#ex_ulog,
>>>> with the ebtables FORWARD chain I observed near line rate speeds (>
>>>> 9Gbps)
>>>>
>>>>
>>>> $sudo ebtables -A FORWARD --ulog-nlgroup 1
>>>>
>>>>
>>>> The major difference that I can see between the two handlers, is that
>>>> in the case of NFQUEUE, the whole packet payload is actually copied
>>>> into user space, while for the test_ulog it isn't. I tried with the
>>>> NFQNL_COPY_META as well, and the speeds for that was ~ 2Mbps.
>>>>
>>>>
>>>> I know this isn't an iptables/ebtables forum, but wondering if anyone
>>>> can throw some light on this? I read this document
>>>> here: http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html, and this
>>>> figure
>>>> here http://ebtables.sourceforge.net/br_fw_ia/PacketFlow.png seems to
>>>> suggest that the bridged packets do indeed go through the iptables
>>>> filter table FORWARD chain...., so clearly, there is something that I
>>>> don't have a handle on. My CPU utilization is pretty low ( ~ 8%), so
>>>> that clearly isn't the issue here....
>>>>
>>>>
>>>> Thanks,
>>>> Hari
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Nov 21, 2011 at 10:37 AM, Victor Julien <victor at inliniac.net>
>>>> wrote:
>>>> On 11/21/2011 09:00 AM, Hariharan Thantry wrote:
>>>> > When I turn on Suricata (latest 1.1 release version), with
>>>> the defaults,
>>>> > the speeds range between 350kbps-1Mbps (using emerging
>>>> threats ruleset).
>>>>
>>>>
>>>> Those numbers are way to low. I run a 8k ruleset in nfq mode
>>>> on an Atom
>>>> N270 and it easily keeps up with 12mbit (which is my internet
>>>> connection). So on that hardware you should see much better
>>>> speeds.
>>>>
>>>> Do you see one of the threads hit 100% all the time?
>>>>
>>>> How many rules are you using? And are you using the specific
>>>> Suricata ET
>>>> version?
>>>>
>>>> --
>>>> ---------------------------------------------
>>>> Victor Julien
>>>> http://www.inliniac.net/
>>>> PGP: http://www.inliniac.net/victorjulien.asc
>>>> ---------------------------------------------
>>>>
>>>>
>>>> _______________________________________________
>>>> Oisf-users mailing list
>>>> Oisf-users at openinfosecfoundation.org
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Oisf-users mailing list
>>>> Oisf-users at openinfosecfoundation.org
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>> --
>>> Eric Leblond
>>> Blog: http://home.regit.org/
>>>
>>> _______________________________________________
>>> Oisf-users mailing list
>>> Oisf-users at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>> _______________________________________________
>> Oisf-users mailing list
>> Oisf-users at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
More information about the Oisf-users
mailing list