[Oisf-users] Suricata AF_PACKET 4 interfaces 2 subnets problem.
Jose Carlos Álvarez
jcalvarezvg at gmail.com
Wed Feb 10 10:38:55 UTC 2016
Hi all:
I'm testing suricata 2.1 Beta 4 on a server with 4 RJ45 interfaces; Suricata is configured in AF_PACKET IPS mode interfaces eth1-eth2 (pair 1) and eth3-eth4 (pair 2)
I am filtering simultaneously a subnet in pair 1 and other subnet in pair 2.
Traffic on pair 1 goes fine, but in pair 2 IMAP traffic doesn't goes through.
I have revised the logs and apparently no rules are dropping IMAP traffic. Any hints?
> On 09 Feb 2016, at 20:20, oisf-users-request at lists.openinfosecfoundation.org wrote:
>
> Send Oisf-users mailing list submissions to
> oisf-users at lists.openinfosecfoundation.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> or, via email, send a message with subject or body 'help' to
> oisf-users-request at lists.openinfosecfoundation.org
>
> You can reach the person managing the list at
> oisf-users-owner at lists.openinfosecfoundation.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Oisf-users digest..."
>
>
> Today's Topics:
>
> 1. Re: Alternatives to ET Pro? (Cooper F. Nelson)
> 2. Re: Alternatives to ET Pro? (Will Metcalf)
> 3. Re: Alternatives to ET Pro? (Will Metcalf)
> 4. Re: Alternatives to ET Pro? (Brandon Lattin)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 9 Feb 2016 09:34:57 -0800
> From: "Cooper F. Nelson" <cnelson at ucsd.edu>
> To: Brandon Lattin <latt0050 at umn.edu>, Will Metcalf
> <william.metcalf at gmail.com>
> Cc: oisf-users <oisf-users at lists.openinfosecfoundation.org>
> Subject: Re: [Oisf-users] Alternatives to ET Pro?
> Message-ID: <56BA2341.6020107 at ucsd.edu>
> Content-Type: text/plain; charset=utf-8
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> We are in the same boat and addressed our performance issues with some
> simple tuning.
>
> First off, consider filtering out top-talkers via bpf filters,
> particularly your ISP's CDN vlan (akamai and netflix). As well as local
> google/youtube caches if you have them. Filtering out HD video will
> double your performance, easily.
>
> You can find top-talkers on the command line with this script, just
> replace the '-i eth2' with your NIC.
>
>> #!/bin/bash
>>
>> sudo tcpdump -tnn -c 100000 -i eth2 | awk -F "." '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -nr | awk ' $1 > 100 '
>
>
> Also, as mentioned, consider migrating to either the 3.0 or the high
> performance branch:
>
>> https://github.com/inliniac/suricata/tree/dev-detect-grouping-v174
>
> The performance is way better. Building from source and setting your
> CFLAGS to '-O3' will also reduce packet drops by a few %.
>
> Will, you guys are doing a great job and suricata+ETPRO is easily the
> best value security product on the market!
>
> - -Coop
>
>> On 2/9/2016 8:56 AM, Brandon Lattin wrote:
>> We've been happy with ET for year, but we have noticed an increase in
>> the number of rules we've had to disable due to unreasonably high
>> percentages of CPU time. We do profile on a large sample set (~7Gbps for
>> 10 minutes) daily. I have no doubt that you guys do your best testing,
>> but we have a large network with a lot of students and researchers doing
>> some very weird things. We get that it's as much an art as it is a science.
>
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
>
> iQEcBAEBAgAGBQJWuiNBAAoJEKIFRYQsa8FWm1sH/2BnTyJmjYPzXe7luFozW3R5
> poRmCOhkX/E806j/ToQBBACAzH6rG0xacndXdvQpyhsMObs7mN36zFR5wAJfQfPn
> +JmfLL2OXQWRk3jr6gfHkkB/NhNPgAzhewRxpIA1DV/I0YuHYxRcTHWwK+0u99iR
> h6x41kSsllQk/UsvVacoR/h0Y8Di24CNtbwOqy0Bl35tFgPqvk2af1toAEVhHk6l
> Gjvqcsr5Xm2mSpsoxLvtBfLwPqpNU/ZXu95oqbVu/Bb8yzqLeQt3jgQyGhFp2tfn
> xbfHGj4gu0+N6QUxSXIQVLZFDPfLTZkoswSjc/ate1TC4rvB7y2CwaNY4gpooDA=
> =A4vF
> -----END PGP SIGNATURE-----
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 9 Feb 2016 12:45:54 -0600
> From: Will Metcalf <william.metcalf at gmail.com>
> To: Jeff H <jeff61225 at gmail.com>
> Cc: oisf-users <oisf-users at lists.openinfosecfoundation.org>
> Subject: Re: [Oisf-users] Alternatives to ET Pro?
> Message-ID:
> <CAO0nrJbji4ZmAbt08rzk7KexscrE3TTX6zK_9FiqExNt4BYm=Q at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> If i recall correctly it was around May/June 2015. If the first you are
> hearing of it is at renewal time, we apparently did a very poor job of
> communicating the increase to our customers. For this I am truly sorry. If
> we did not send out a price increase notification prior to the increase
> going into effect shame on us. I will try to run down what went wrong and
> report back.
>
> Regards,
>
> Will
>
>> On Tue, Feb 9, 2016 at 10:55 AM, Jeff H <jeff61225 at gmail.com> wrote:
>>
>> On Tue, Feb 9, 2016 at 8:41 AM, Will Metcalf <william.metcalf at gmail.com>
>> wrote:
>>
>>> BTW we have no plans to raise prices that I'm aware of :).
>>>
>>> Regards,
>>>
>>> Will
>>
>> When was the last price increase? I was just looking and noticed the price
>> is now $750 for a year of ET Pro. The last subscription I purchased was in
>> May, 2015 and it was $500.
>>
>> Jeff
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160209/ac34abe9/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 9 Feb 2016 12:49:58 -0600
> From: Will Metcalf <william.metcalf at gmail.com>
> To: Brandon Lattin <latt0050 at umn.edu>
> Cc: oisf-users <oisf-users at lists.openinfosecfoundation.org>
> Subject: Re: [Oisf-users] Alternatives to ET Pro?
> Message-ID:
> <CAO0nrJYQ6hLyWur8R7jG=SpfPRmQjR+JSTkW=HhZHh2c0Kv8tQ at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
>> We've been happy with ET for year, but we have noticed an increase in the
> number of rules we've had to disable due to unreasonably high percentages
> of CPU time. We do profile on a large sample set (~7Gbps for 10 minutes)
> daily.
>
> Would you be willing to share these stats off list? I would love to take a
> stab at trying to optimize the worst offenders, additionally I don't think
> there is any super sensitive information in the rule perf stats.
>
> Regards,
>
> Will
>
>> On Tue, Feb 9, 2016 at 10:56 AM, Brandon Lattin <latt0050 at umn.edu> wrote:
>>
>> We've been happy with ET for year, but we have noticed an increase in the
>> number of rules we've had to disable due to unreasonably high percentages
>> of CPU time. We do profile on a large sample set (~7Gbps for 10 minutes)
>> daily. I have no doubt that you guys do your best testing, but we have a
>> large network with a lot of students and researchers doing some very weird
>> things. We get that it's as much an art as it is a science.
>>
>> As for the price whispers I'm hearing from legal, I guess we'll let the
>> lawyers figure it out.
>>
>> On Tue, Feb 9, 2016 at 10:41 AM, Will Metcalf <william.metcalf at gmail.com>
>> wrote:
>>
>>> I'm sorry that you feel our rule quality has declined. We test rules for
>>> perf/fp's before they go out each day on a live sensor network and with a
>>> collection of QA pcaps. That said the straight poop is every network is a
>>> snowflake and determining performance impact of a rule in a specific
>>> network without a feedback loop is impossible, it's an educated guess at
>>> best. As a PRO customer you are more than welcome to open a support ticket.
>>> With additional info about your environment we can help try to tune the
>>> rule with you. Alternatively you can simply disable it, or look at using
>>> Lua to detect a multi-byte encoded xor'd executable although I doubt this
>>> will be any cheaper perf wise. Additionally I would be weary of relying on
>>> suri rule perf stats outside of single threaded mode during short runs to
>>> sample rule perf. In my experience they tend to be unreliable even with the
>>> same rules/networks across runs. Victor can correct me if I'm wrong but
>>> afaik they are unreliable in these modes as they include lock wait time
>>> which should level out over long runs. BTW we have no plans to raise
>>> prices that I'm aware of :).
>>>
>>> Regards,
>>>
>>> Will
>>>
>>>> On Tue, Feb 9, 2016 at 9:36 AM, Brandon Lattin <latt0050 at umn.edu> wrote:
>>>>
>>>> I'm sure some of you are aware that Proofpoint has acquired Emerging
>>>> Threats.
>>>>
>>>> We've seen a decline (perhaps anecdotal) in rule quality - to the tune
>>>> of a single new rule (2815810) taking 49% of total CPU time. Additionally,
>>>> it would appear they are planning on raising prices.
>>>>
>>>> I'm curious if anyone is using an alternative to the ET Pro set.
>>>>
>>>> Thanks!
>>>>
>>>> --
>>>> Brandon Lattin
>>>> Security Analyst
>>>> University of Minnesota - University Information Security
>>>> Office: 612-626-6672
>>>>
>>>> _______________________________________________
>>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>>> Site: http://suricata-ids.org | Support:
>>>> http://suricata-ids.org/support/
>>>> List:
>>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>> Suricata User Conference November 9-11 in Washington, DC:
>>>> http://oisfevents.net
>>
>>
>> --
>> Brandon Lattin
>> Security Analyst
>> University of Minnesota - University Information Security
>> Office: 612-626-6672
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160209/92c20d4a/attachment-0001.html>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 9 Feb 2016 13:20:06 -0600
> From: Brandon Lattin <latt0050 at umn.edu>
> To: Will Metcalf <william.metcalf at gmail.com>
> Cc: oisf-users <oisf-users at lists.openinfosecfoundation.org>
> Subject: Re: [Oisf-users] Alternatives to ET Pro?
> Message-ID:
> <CAJWRZ8TkjK-KeDTiTHDOZnaO+3yhp3DRvmCY+ZtyOAL2xqyvGA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Absolutely. Let me chat with one of my guys and we'll work out an automated
> method.
>
> We can modify our testing frequency and window if it's helpful as well.
>
> I'll get back to you within the week with something.
>
> On Tue, Feb 9, 2016 at 12:49 PM, Will Metcalf <william.metcalf at gmail.com>
> wrote:
>
>>> We've been happy with ET for year, but we have noticed an increase in the
>> number of rules we've had to disable due to unreasonably high percentages
>> of CPU time. We do profile on a large sample set (~7Gbps for 10 minutes)
>> daily.
>>
>> Would you be willing to share these stats off list? I would love to take
>> a stab at trying to optimize the worst offenders, additionally I don't
>> think there is any super sensitive information in the rule perf stats.
>>
>> Regards,
>>
>> Will
>>
>>> On Tue, Feb 9, 2016 at 10:56 AM, Brandon Lattin <latt0050 at umn.edu> wrote:
>>>
>>> We've been happy with ET for year, but we have noticed an increase in the
>>> number of rules we've had to disable due to unreasonably high percentages
>>> of CPU time. We do profile on a large sample set (~7Gbps for 10 minutes)
>>> daily. I have no doubt that you guys do your best testing, but we have a
>>> large network with a lot of students and researchers doing some very weird
>>> things. We get that it's as much an art as it is a science.
>>>
>>> As for the price whispers I'm hearing from legal, I guess we'll let the
>>> lawyers figure it out.
>>>
>>> On Tue, Feb 9, 2016 at 10:41 AM, Will Metcalf <william.metcalf at gmail.com>
>>> wrote:
>>>
>>>> I'm sorry that you feel our rule quality has declined. We test rules for
>>>> perf/fp's before they go out each day on a live sensor network and with a
>>>> collection of QA pcaps. That said the straight poop is every network is a
>>>> snowflake and determining performance impact of a rule in a specific
>>>> network without a feedback loop is impossible, it's an educated guess at
>>>> best. As a PRO customer you are more than welcome to open a support ticket.
>>>> With additional info about your environment we can help try to tune the
>>>> rule with you. Alternatively you can simply disable it, or look at using
>>>> Lua to detect a multi-byte encoded xor'd executable although I doubt this
>>>> will be any cheaper perf wise. Additionally I would be weary of relying on
>>>> suri rule perf stats outside of single threaded mode during short runs to
>>>> sample rule perf. In my experience they tend to be unreliable even with the
>>>> same rules/networks across runs. Victor can correct me if I'm wrong but
>>>> afaik they are unreliable in these modes as they include lock wait time
>>>> which should level out over long runs. BTW we have no plans to raise
>>>> prices that I'm aware of :).
>>>>
>>>> Regards,
>>>>
>>>> Will
>>>>
>>>>> On Tue, Feb 9, 2016 at 9:36 AM, Brandon Lattin <latt0050 at umn.edu> wrote:
>>>>>
>>>>> I'm sure some of you are aware that Proofpoint has acquired Emerging
>>>>> Threats.
>>>>>
>>>>> We've seen a decline (perhaps anecdotal) in rule quality - to the tune
>>>>> of a single new rule (2815810) taking 49% of total CPU time. Additionally,
>>>>> it would appear they are planning on raising prices.
>>>>>
>>>>> I'm curious if anyone is using an alternative to the ET Pro set.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> --
>>>>> Brandon Lattin
>>>>> Security Analyst
>>>>> University of Minnesota - University Information Security
>>>>> Office: 612-626-6672
>>>>>
>>>>> _______________________________________________
>>>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>>>> Site: http://suricata-ids.org | Support:
>>>>> http://suricata-ids.org/support/
>>>>> List:
>>>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>> Suricata User Conference November 9-11 in Washington, DC:
>>>>> http://oisfevents.net
>>>
>>>
>>> --
>>> Brandon Lattin
>>> Security Analyst
>>> University of Minnesota - University Information Security
>>> Office: 612-626-6672
>
>
> --
> Brandon Lattin
> Security Analyst
> University of Minnesota - University Information Security
> Office: 612-626-6672
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160209/3f85f850/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at lists.openinfosecfoundation.org
> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
>
> ------------------------------
>
> End of Oisf-users Digest, Vol 75, Issue 15
> ******************************************
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6024 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160210/972256e5/attachment-0001.bin>
More information about the Oisf-users
mailing list