[Oisf-users] Suricata seperate Rx/Tx connection

mohammad kashif kashif.alig at gmail.com
Mon Nov 4 14:00:21 UTC 2019


Hi Michał

Thanks a lot for your detailed reply and drawing in ASCII !

I am testing with various options and I will let you know with the outcome.

Regards

Kashif

On Fri, Nov 1, 2019 at 9:38 PM Michał Purzyński <michalpurzynski1 at gmail.com>
wrote:

> I think we are all over-complicating things here and no one shares a
> complete solution ;)
>
>
> https://www.arista.com/en/um-eos/eos-section-20-4-tap-aggregation-traffic-steering
>
> If you have Arista in front of your cluster, you connect RX and TX from
> all your taps there, and then configure it creating bonding / aggregate
> interface on Arista (not on Linux) and connect those "bonded" ports to a
> number of network cards from the same or different sensors.
> Traffic will be load balanced per 2 or 3-tuple. You can then start
> Suricata with AF_Packet on all Ethernet interfaces
>
> OMG watch me drawing in ASCII ;)
>
> ---> (tap 1 RX) - |et1| | AR |
> ---> (tap 1 TX) - |et2| | IS   | | et41 | -> | Po10 | -> | sensor's first
> card |
> ---> (tap 2 RX) - |et3| | TA  | | et42 | -> | Po10 | -> | sensor's second
> card |
> ---> (tap 2 TX) - |et4| |        |
>
> Arista's configuration (and literally any packet broker will do, this one
> is cheap, Gigamon is expensive but makes a coffee while deduplicating
> packets and God knows what else. I deduplicated with a couple access-list,
> took me like one day)
>
> # conf t (hi, Cisco!!)
>
> tap aggregation
>    mode exclusive
>
> load-balance policies
>    load-balance fm6000 profile NSMConSymm
>       port-channel hash-seed 39
>       no fields mac
>       fields ip protocol dst-ip src-ip
>       distribution symmetric-hash mac-ip
>
> reboot here
>
> Create "output" aggregated links where you will connect your sensors, I
> have two (and I'm lying but that's for brevity)
>
> interface Port-Channel10
>    description Bro production
>    l2 mtu 9000
>    switchport mode tool
>    switchport tool group set to_bro
> !
> interface Port-Channel20
>    description Suricata production
>    ip access-group drop_noise_before_suricata out
>    l2 mtu 9000
>    switchport mode tool
>    switchport tool group set to_suricata
> !
>
> Then for each port where you have a tap connected
>
> interface Ethernet1
>    speed forced 10000full
>    l2 mtu 9000
>    ingress load-balance profile NSMConSymm
>    ip access-group drop_noise_from_taps in
>    service-policy type tapagg input from_taps
>    switchport mode tap
>
> Repeat as necessary
>
> Each sensor has two network cards here to deal with the "2x 10Gbit > 1x
> 10Gbit" problem and also for NUMA (optional)
>
> First interface on the sensor, will get all packets that are part of the
> flow between 1.2.3.4 <-> 23.24.25.26 for example
>
> interface Ethernet41
>    description nsm6:1a
>    channel-group 20 mode on
>    switchport mode tool
> !
>
> Second interface on the sensor, will get other 3-tuple hashed flows, etc
>
> interface Ethernet42
>    description nsm6:2a
>    channel-group 20 mode on
>    switchport mode tool
> !
>
> Going back to Arista's configuration, now let's glue it all together
>
> Class-map filters what's send to sensors, here we just send everything,
> while maintaining a configuration that lets me steer part of the traffic
> somewhere else, should I need to, like during DDoS.
>
> class-map type tapagg match-any match_from_taps
>    10 match ip access-group match_any
> !
> class-map type tapagg match-any match_noise
>    10 match ip access-group send_noise_to_null
> !
>
> Take traffic "labeled" as from_taps, try to match it against class
> match_noise, if it does, send to ports tagged with "to_null. Take remaining
> traffic, try to match against class match_from_taps, everything that
> matches is send to ports tagged to_bro and to_suricata
>
> policy-map type tapagg from_taps
>    10 class match_noise
>       set aggregation-group to_null
>    !
>    100 class match_from_taps
>       set aggregation-group group to_bro group to_suricata
> !
>
> On Fri, Nov 1, 2019 at 2:02 PM Nelson, Cooper <cnelson at ucsd.edu> wrote:
>
>> What packet capture method are you using?
>>
>>
>>
>> -Coop
>>
>>
>>
>> *From:* Amar <amar at countersnipe.com>
>> *Sent:* Friday, November 1, 2019 1:58 PM
>> *To:* Nelson, Cooper <cnelson at ucsd.edu>
>> *Cc:* mohammad kashif <kashif.alig at gmail.com>; Oisf-Users <
>> oisf-users at lists.openinfosecfoundation.org>
>> *Subject:* Re: [Oisf-users] Suricata seperate Rx/Tx connection
>>
>>
>>
>> CounterSnipe default setup bonds all interfaces into a single bond#(0)
>> and starts Suri with -i bond0 and it works fine.
>>
>>
>>
>> On Nov 1, 2019 at 10:49 PM, <Cooper Nelson <cnelson at ucsd.edu>> wrote:
>>
>> That would work with pcap, not sure how AF_PACKET handles bonded
>> interfaces.
>>
>>
>>
>> We use an Arista with two 10Gbit interfaces and pevma’s config.
>>
>>
>>
>> -Coop
>>
>>
>>
>> *From:* Amar <amar at countersnipe.com>
>> *Sent:* Friday, November 1, 2019 8:19 AM
>> *To:* mohammad kashif <kashif.alig at gmail.com>
>> *Cc:* Nelson, Cooper <cnelson at ucsd.edu>; Oisf-Users <
>> oisf-users at lists.openinfosecfoundation.org>
>> *Subject:* Re: [Oisf-users] Suricata seperate Rx/Tx connection
>>
>>
>>
>> Could bonding be the solution here. Bond eth1 and 2 and simply monitor
>> the bond.
>>
>>
>>
>> On Nov 1, 2019 at 4:08 PM, <mohammad kashif <kashif.alig at gmail.com>>
>> wrote:
>>
>>
>>
>>
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>
>> Conference: https://suricon.net
>> Trainings: https://suricata-ids.org/training/
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20191104/6e4fce87/attachment.html>


More information about the Oisf-users mailing list