[Oisf-users] Suricata at 10G, why don't use thread local storage instead of shared linked list to manage flow data?

[Maxim]

Hi all, 
I use perf to monitor suricata, and found that in my case (the 10G scenario) the function FlowGetFlowFromHash in flow-hash.c consumes most CPU times due to list traversing and pthread_lock_lock/unlock. I wonder to know why don't we use thread local storage to get rid of this to reduce contention? Seems that if we use the worker runmode, and use multiple threads to process captured packets, these threads share the same flow management list. My proposal is that if we bind threads to separate cores, and bind different NIC IRQs to these cores correspondingly, these threads should use thread local storage linked list or per-thread linked list to manage their own flow data, and let the NIC to issue that packets belonging to the same flow are sent to corresponding threads, what do you think? I think hardware flow function is good than the software-implemented one, am I right? Thanks.






At 2017-02-02 22:33:27, "Collyer, Jeffrey W. (jwc3f)" <jwc3f at virginia.edu> wrote:

Here is the stream section.  Everything that wasn’t commented out.  I think its just the defaults, so that may need some tuning.


stream:
  memcap: 64mb
  checksum-validation: yes      # reject wrong csums
  inline: auto                  # auto will use inline mode in IPS mode, yes or no set it statically
  reassembly:
    memcap: 256mb
    depth: 1mb                  # reassemble 1mb into a stream
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes




Jeffrey Collyer
Information Security Engineer
University of Virginia




On Feb 1, 2017, at 4:14 PM, Peter Manev <petermanev at gmail.com> wrote:


On Wed, Feb 1, 2017 at 3:04 PM, Collyer, Jeffrey W. (jwc3f)
<jwc3f at virginia.edu> wrote:
Sure,



Yes it does not look right....

Can you please share your stream and reassembly section from your
suricata.yaml as well ?
( https://redmine.openinfosecfoundation.org/projects/suricata/repository/revisions/master/entry/suricata.yaml.in#L1197
)



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20170203/d6cbec57/attachment-0002.html>