[Oisf-users] high cpu loads running multiple suricata instances

[Aki Heikkinen]

I'll try to combine a bit longer answer to many questions my email 
brought up:

Suricata is running inline as IPS, on a multirouting linux 
firewall/router which has many internal networks behind it (vlans over 
trunked gigabit connections to core switches, WANs directly connected to 
two separate dual fast ethernet cards). Mainly just cleaning up LANs <-> 
WANs, open WLAN <-> WANs and WANs <-> DMZs traffic, also some internal 
traffic is screened. Iow, very different configs, hard to see how this 
could be achieved with single suricata running.

Setup was originally made for multiple snort_inline instances, and 
suricata is now running as a drop-in-replacement until can make some 
time for larger system revamp (maybe next summer). Suricata instances 
are not tied to cores and are taxing CPUs heavily which is not ideal, 
but end-user experience is better - pages from internet and DMZs load 
quicker. Internal non-nfqueued traffic is flowing without hickups, 
kernel scheduler should give precedence over userspace applications anyway.

I realize it's not intended way to use suricata but it sure works. Could 
tune it though as the basic setup seems solid.

Still at loss why idle instances (not a single packet through nfqueue) 
eat up CPU cycles so viciously.

Yours,

Aki





On 07/20/2010 04:20 PM, Victor Julien wrote:
> What made you decide to run multiple instances? Suricata is
> multi-threaded so it will use all your cores with a single instance as
> well. That should reduce the overhead of the "management" threads, most
> importantly the flow manager.
>
> Cheers,
> victor
>
> Aki Heikkinen wrote:
>    
>> Hi,
>>
>> We're running 7 to 10 suricata 1.0 instances on old 2,7GHz quad
>> processor xeon server (with hyperthreading so OS sees 8 cores), used to
>> run snort_inline on same hardware (snort_inline instances attached to
>> different cores). Suricata instances suck up cpu cycles. I can
>> understand this for high traffic-volume instances which has hundreds of
>> UDP connections passing through but half of the instances are mainly
>> idling.
>>
>> top - 15:45:14 up 76 days,  7:41,  4 users,  load average: 5.09, 5.40, 5.37
>> Tasks: 135 total,   4 running, 131 sleeping,   0 stopped,   0 zombie
>> Cpu(s):  8.5%us, 21.4%sy,  0.0%ni, 70.0%id,  0.0%wa,  0.0%hi,  0.0%si,
>> 0.0%st
>> Mem:   3961684k total,  3712484k used,   249200k free,   245996k buffers
>> Swap:  7815612k total,   282032k used,  7533580k free,   593384k cached
>>
>>
>> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>>
>> 23732 suricata  20   0  369m 160m 1588 S   55  4.2  10368:21 suricata
>> 23728 suricata  20   0  455m 228m 1484 S   53  5.9  10367:21 suricata
>> 14068 suricata  20   0  378m 225m 1600 S   49  5.8  27:37.22 suricata
>> 23734 suricata  20   0  832m 545m 1640 R   49 14.1  13311:46 suricata
>> 23730 suricata  20   0  408m 237m 1540 S   48  6.1  10373:09 suricata
>> 23726 suricata  20   0  477m 310m 1612 S   46  8.0  10383:05 suricata
>> 23724 suricata  20   0  789m 589m 1620 S   44 15.2  11698:04 suricata
>>
>> Any tips would be appreciated howto tune suricata to better use
>> resources on this hardware. Current config is pretty much suricatas
>> default with most ET signatures enabed.
>>
>> Hopefully suricata manual will be out soon! :)
>>
>> Yours,
>>
>> Aki Heikkinen
>>
>> _______________________________________________
>> Oisf-users mailing list
>> Oisf-users at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>      
>
>    


-- 

  --------------------------------------------------------------------------
   Aki Heikkinen
  Järjestelmäasiantuntija


  KuusiSolutions
  +358 40 7653740
  www.kuusisolutions.fi

  -------------------------------------------------------------------------
  -
  TÄRKEÄÄ! Tämä sähköposti on tarkoitettu vain nimetyille
  vastaanottajille. Viesti voi sisältää luottamuksellista ja
  tekijänoikeudellisesti suojattua tietoa. Viestin luvaton eteenpäin
  lähettäminen ja muu käyttö on kiellettyä. Jos olet vastaanottanut tämän
  viestin vahingossa, pyydämme sinua lähettämään sen takaisin meille ja
  tuhoamaan kaikki kopiot. Kiitos!
  -------------------------------------------------------------------------