[Oisf-users] Suricata seperate Rx/Tx connection

Fri Nov 1 21:38:10 UTC 2019

I think we are all over-complicating things here and no one shares a
complete solution ;)

https://www.arista.com/en/um-eos/eos-section-20-4-tap-aggregation-traffic-steering

If you have Arista in front of your cluster, you connect RX and TX from all
your taps there, and then configure it creating bonding / aggregate
interface on Arista (not on Linux) and connect those "bonded" ports to a
number of network cards from the same or different sensors.
Traffic will be load balanced per 2 or 3-tuple. You can then start Suricata
with AF_Packet on all Ethernet interfaces

OMG watch me drawing in ASCII ;)

---> (tap 1 RX) - |et1| | AR |
---> (tap 1 TX) - |et2| | IS   | | et41 | -> | Po10 | -> | sensor's first
card |
---> (tap 2 RX) - |et3| | TA  | | et42 | -> | Po10 | -> | sensor's second
card |
---> (tap 2 TX) - |et4| |        |

Arista's configuration (and literally any packet broker will do, this one
is cheap, Gigamon is expensive but makes a coffee while deduplicating
packets and God knows what else. I deduplicated with a couple access-list,
took me like one day)

# conf t (hi, Cisco!!)

tap aggregation
   mode exclusive

load-balance policies
   load-balance fm6000 profile NSMConSymm
      port-channel hash-seed 39
      no fields mac
      fields ip protocol dst-ip src-ip
      distribution symmetric-hash mac-ip

reboot here

Create "output" aggregated links where you will connect your sensors, I
have two (and I'm lying but that's for brevity)

interface Port-Channel10
   description Bro production
   l2 mtu 9000
   switchport mode tool
   switchport tool group set to_bro
!
interface Port-Channel20
   description Suricata production
   ip access-group drop_noise_before_suricata out
   l2 mtu 9000
   switchport mode tool
   switchport tool group set to_suricata
!

Then for each port where you have a tap connected

interface Ethernet1
   speed forced 10000full
   l2 mtu 9000
   ingress load-balance profile NSMConSymm
   ip access-group drop_noise_from_taps in
   service-policy type tapagg input from_taps
   switchport mode tap

Repeat as necessary

Each sensor has two network cards here to deal with the "2x 10Gbit > 1x
10Gbit" problem and also for NUMA (optional)

First interface on the sensor, will get all packets that are part of the
flow between 1.2.3.4 <-> 23.24.25.26 for example

interface Ethernet41
   description nsm6:1a
   channel-group 20 mode on
   switchport mode tool
!

Second interface on the sensor, will get other 3-tuple hashed flows, etc

interface Ethernet42
   description nsm6:2a
   channel-group 20 mode on
   switchport mode tool
!

Going back to Arista's configuration, now let's glue it all together

Class-map filters what's send to sensors, here we just send everything,
while maintaining a configuration that lets me steer part of the traffic
somewhere else, should I need to, like during DDoS.

class-map type tapagg match-any match_from_taps
   10 match ip access-group match_any
!
class-map type tapagg match-any match_noise
   10 match ip access-group send_noise_to_null
!

Take traffic "labeled" as from_taps, try to match it against class
match_noise, if it does, send to ports tagged with "to_null. Take remaining
traffic, try to match against class match_from_taps, everything that
matches is send to ports tagged to_bro and to_suricata

policy-map type tapagg from_taps
   10 class match_noise
      set aggregation-group to_null
   !
   100 class match_from_taps
      set aggregation-group group to_bro group to_suricata
!

On Fri, Nov 1, 2019 at 2:02 PM Nelson, Cooper <cnelson at ucsd.edu> wrote:

> What packet capture method are you using?
>
>
>
> -Coop
>
>
>
> *From:* Amar <amar at countersnipe.com>
> *Sent:* Friday, November 1, 2019 1:58 PM
> *To:* Nelson, Cooper <cnelson at ucsd.edu>
> *Cc:* mohammad kashif <kashif.alig at gmail.com>; Oisf-Users <
> oisf-users at lists.openinfosecfoundation.org>
> *Subject:* Re: [Oisf-users] Suricata seperate Rx/Tx connection
>
>
>
> CounterSnipe default setup bonds all interfaces into a single bond#(0) and
> starts Suri with -i bond0 and it works fine.
>
>
>
> On Nov 1, 2019 at 10:49 PM, <Cooper Nelson <cnelson at ucsd.edu>> wrote:
>
> That would work with pcap, not sure how AF_PACKET handles bonded
> interfaces.
>
>
>
> We use an Arista with two 10Gbit interfaces and pevma’s config.
>
>
>
> -Coop
>
>
>
> *From:* Amar <amar at countersnipe.com>
> *Sent:* Friday, November 1, 2019 8:19 AM
> *To:* mohammad kashif <kashif.alig at gmail.com>
> *Cc:* Nelson, Cooper <cnelson at ucsd.edu>; Oisf-Users <
> oisf-users at lists.openinfosecfoundation.org>
> *Subject:* Re: [Oisf-users] Suricata seperate Rx/Tx connection
>
>
>
> Could bonding be the solution here. Bond eth1 and 2 and simply monitor the
> bond.
>
>
>
> On Nov 1, 2019 at 4:08 PM, <mohammad kashif <kashif.alig at gmail.com>>
> wrote:
>
>
>
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20191101/02a6c908/attachment.html>