[Oisf-users] EXTERNAL: Re: autofp vs workers - updated comparison?

[Rasmor, Zachary R]

Hello Eric,

Thanks for the reply, this definitely sounds interesting. One question...My understanding was that it is necessary for all packets belonging to a given flow to be processed by the same worker thread. Am I mistaken? And if my understanding is correct, how does this work with the rollover option when a packet is sent to a different socket?

Regards,
Zach

________________________
Zach Rasmor
Email: zachary.r.rasmor at lmco.com
Office: 301.240.6116

-----Original Message-----
From: Eric Leblond [mailto:eric at regit.org] 
Sent: Monday, July 20, 2015 12:36 PM
To: Rasmor, Zachary R
Cc: oisf-users at lists.openinfosecfoundation.org
Subject: EXTERNAL: Re: [Oisf-users] autofp vs workers - updated comparison?

Hi,

Le 20 juil. 2015 5:53 PM, "Rasmor, Zachary R" <zachary.r.rasmor at lmco.com> a écrit :
>
> Hello,
>
>  
>
> I was hoping for some updated insight regarding the comparison between the autofp and workers runmodes. I saw some of discussion on this thread comparing the two from 2-3 years ago, but I think a lot has changed since then. I’ve also noticed posts expressing performance concerns with autofp because of contention due to locking, but I’m not sure how up-to-date those are, either.
>
>  
>
> The conventional wisdom from users on this thread, as well as from the Suricata training I attended in Virgina this year, seem to suggest that ‘workers’ is the preferred runmode. However, my testing has shown various circumstances where I’m dropping packets in workers mode due to 1 or 2 (out of 16) threads pegged at ~99-100% CPU. Also, any costly Lua add-ons raise the likelihood of holding up the pipeline and dropping packets in workers mode. The load balancing aspects of autofp make it a more appealing and logical option, in my mind.

Yes but this is without considering the great work by some guys at Google ;) They have implemented an option in af_packet called rollover that send the packet to another socket in case of contention. I've proposed a PR implementing that mode in Suricata. It has shown some dramatic improvement regarding packet loss in workers mode. I hope Victor will merge that soon and that it will be available in next beta.

Side note: lua is not that slow compare to some regular expressions.

>
> I’d appreciate any up-to-date insight anyone has. I’ve noticed some of the autofp related optimizations in 2.1beta4, but I’d like to better understand how much autofp has evolved in the past couple years.

Recent feedback I've got seem to show workers mode is still better.

BR,
--
Eric

>  
>
> Thanks,
>
> Zach
>
>  
>
> ________________________
>
> Zach Rasmor
>
> Senior Software Engineer
>
> Lockheed Martin CIRT
>
> 700 N Frederick Ave | Gaithersburg, MD 20879
>
> Email: zachary.r.rasmor at lmco.com
>
> Office: 301.240.6116
>
>  
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 11767 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20150721/dd44112e/attachment-0002.bin>