[Oisf-users] dev-detect-grouping-v174, only 2 cores being used?

Sun Mar 20 16:14:37 UTC 2016

On Fri, Mar 11, 2016 at 8:15 AM, Barkley, Joey
<Joey.Barkley at ingramcontent.com> wrote:
> ok, that seemed to do the trick. I've reduced my ruleset to around 6500 for now from the 19K I was trying before, but this branch with those settings seem pretty good. Previously I was loading up about 1500 rules with 16 cores and using about 45GB RAM. Now I'm running 6500 on 8 cores (really 6, 2 are for mgmt) and using around 13GB RAM. Startup time is drastically reduced as well. Seconds instead of minutes.
>
> All 6 processing cores seem to be working normally. mgmt cores are doing some work but not a lot. I'll continue to tweak my yaml file to see if I can max out performance with these lower memory requirements. If I continue to increase the groups I assume it will increase the memory usage, correct?

Yeah - but the increase observed so far  during live test is just minimal.
 I doubt you are going to need more than 600-800 groups - have a look
at your suricata.log when started in verbose mode ("-v") and you will
see how many groups are used for TCP/UDP and OTHER ...but as I
mentioned - during the testing I  have not seen it use more than 600
groups (per proto).

>
> Thanks for the help and all the hard work!
>
> ________________________________________
> From: Peter Manev <petermanev at gmail.com>
> Sent: Wednesday, March 9, 2016 7:41 PM
> To: Barkley, Joey
> Cc: oisf-users at lists.openinfosecfoundation.org
> Subject: Re: [Oisf-users] dev-detect-grouping-v174, only 2 cores being used?
>
> On Tue, Mar 1, 2016 at 5:01 PM, Barkley, Joey
> <Joey.Barkley at ingramcontent.com> wrote:
>> Responses inline below...
>>
>> ________________________________________
>> From: Peter Manev <petermanev at gmail.com>
>> Sent: Tuesday, March 1, 2016 12:47 AM
>> To: Barkley, Joey
>> Cc: oisf-users at lists.openinfosecfoundation.org
>> Subject: Re: [Oisf-users] dev-detect-grouping-v174, only 2 cores being used?
>>
>> On Mon, Feb 29, 2016 at 10:37 PM, Barkley, Joey
>> <Joey.Barkley at ingramcontent.com> wrote:
>>> All,
>>>
>>>
>>> I've done some tweaking to my test instance but can't seem to get it running
>>> properly. Here is what I did:
>>>
>>>
>>> 1) Took the dev-detect-grouping-v174 branch and merged master (as of this
>>> morning, 2016-02-29) into it.
>>
>> I would suggest do it step by step - in order to avoid excessive
>> troubleshooting if needed.
>> So start with just the dev-detect-grouping-v174 branch - but if you
>> start with that I would recommend the latest branch -
>> dev-detect-grouping-v178 branch -
>> https://github.com/inliniac/suricata/tree/dev-detect-grouping-v178
>>
>> OK. I will go to just the v178 branch and retest and tell you what happens.
>>
>
> You might want to try the latest update on that branch -
> https://github.com/inliniac/suricata/tree/dev-detect-grouping-v185
>
>>>
>>> 2) Built Suricata and used my normal config file, but made the required
>>> changes in the "detect" section.
>>
>> What changes are those exactly? Can you share that section of the suricata.yaml?
>>
>> detect:
>>     profile: medium
>>     custom-values:
>>         toclient-groups: 3
>>         toserver-groups: 25
>>     sgh-mpm-context: auto
>>     inspection-recursion-limit: 3000
>>
>> As mentioned below, I also tried 30 & 250 just to see what happens. Same result.
>
> Try profile "custom" with  800 groups on both client/server.
>
>>>
>>>     a. I tried the default (profile medium, toclient 3, toserver 25) but
>>> then also changed to 30 and 250 just to test. Same results with both.
>>>
>>
>> How many rules do you load?(or are you trying with no rules as a test)
>>
>> Right now I think I've got just under 13K rules. I can cut that back if needed.
>>
>>> 3) I have 8 threads set, and I have management cpu set to 0,2 and detect cpu
>>> set to 4-14 (even number cores).
>>>
>>> 4) management cpu set is exclusive and high, so is detect cpu set
>>>
>>>
>>> Suricata starts up very quickly (few seconds) and consumes very little RAM.
>>> However, I get cpu 0 with a very small use %, and cpu's 4 & 14 pegged at
>>> 100%. kernel_drops are extremely high (compared to my working config).
>>>
>>
>> This is - cpu's 4 and 14 are only pegged - not 4 through 14 (even
>> numbers only), is that correct?
>>
>> That's correct. Basically the first and last CPUs I've specified in the detect settings are pegged. None of the others have ANY usage displayed in htop at all.
>
> For the purpose of narrowing the problem - can you try the default  -
> aka switch off the CPU affinity and see if any difference?
>
>>
>>>
>>> I know I've got a lot of variables in this setup, but does anyone see
>>> anything obviously wrong with how I've set things up? Should I stop
>>> separating out the management CPU set and just run them on the CPUs that the
>>> detect threads run on?
>>>
>>>
>>> Thanks,
>>>
>>> Joey Barkley
>>>
>>>
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>> Suricata User Conference November 9-11 in Washington, DC:
>>> http://oisfevents.net
>>
>>
>>
>> --
>> Regards,
>> Peter Manev
>>
>
>
>
> --
> Regards,
> Peter Manev
>

-- 
Regards,
Peter Manev