[Oisf-users] Occasional burst of packet loss

[Yasha Zislin]

When you say " If the packets all have the same src/dst ports/IPs,
> then they are all going to be handled by the same thread", do you mean that all of these four (src IP, dst IP, src Port and dst Port) have to be the same for one thread to be utilized? what if one of these four is different, is it still the same thread? for example, ping sweep or port scan.
Good question about jumbo frames. I am not 100% sure but I would think there are some. In suricata configuration have default-packet-size: set to 1522.
Yes, I am monitoring flows internally as well but my flow buffers are pretty high. Here is a stat info on that:flow_mgr.closed_pruned    | FlowManagerThread         | 271691732flow_mgr.new_pruned       | FlowManagerThread         | 59854939flow_mgr.est_pruned       | FlowManagerThread         | 35647800flow.memuse               | FlowManagerThread         | 11828262632flow.spare                | FlowManagerThread         | 40005878
Besides last two entries, I am not sure how to read it. 
I should not have local to local flows since local to local traffic doesnt try to use this firewall.
I did packet profiling with Suricata and it is about 99% HTTP(s).
I guess, I am trying to figure out if there is a way to reduce packet loss and improve performance while being attacked by either DDOS or something else.
Thanks.
> Date: Mon, 3 Nov 2014 11:18:59 -0800
> From: cnelson at ucsd.edu
> To: coolyasha at hotmail.com; oisf-users at lists.openinfosecfoundation.org
> Subject: Re: [Oisf-users] Occasional burst of packet loss
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> It doesn't even have to be a DOS attack.  Any single high-volume flow
> can peg a CPU as the individual packets within the flow are tied to a
> single core.
> 
> So, for example, our ISP has a /24 dedicated to CDN servers (like Akamai
> and Netflix) and I've seen many cases where a single IP conversation to
> this block causes a DOS condition.  Since we are a gigabit network, its
> not uncommon for a big download (like an Apple update) to average
> 500Mbit/second.  If the packets all have the same src/dst ports/IPs,
> then they are all going to be handled by the same thread.
> 
> Re: packet loss on the internal interface.  Are you monitoring internal
> flows?  Do you have jumbo frames enabled?  Local <-> Local IP flows are
> also an issue as of course they can be extremely high volume.
> Especially for well-tuned protocols like NFS.
> 
> - -Coop
> 
> On 11/3/2014 10:09 AM, Yasha Zislin wrote:
> > Coop,
> > 
> > That makes sense. So you are saying that if there is a DOS attack to one
> > host, only one thread would be utilized for inspection? It wouldnt just
> > spread out across all detection threads?
> > 
> > Also, I did look at other threads and some have less
> > capture.kernel_packets and some have MORE. These with higher values have
> > no packet loss.
> > 
> > Here is another twist to the story.
> > So these two SPAN ports that I monitor are before and after border
> > firewall. Packet loss occurs only on internal interface. I would think
> > that the firewall has high chance of stopping DOS attack.
> > 
> > Thanks for the info.
> > 
> 
> - -- 
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> 
> iQEcBAEBAgAGBQJUV9UjAAoJEKIFRYQsa8FWiL8H/0jSuWDDKDdwR+2mtBNC82kt
> fdB1Q4iWRLjMwS2rjNw99e65ekAr3aowUI4IBU06pZbfW+jnfz7Q/0W7tcim/9BQ
> RAbQqbGI93fc5J/k2MAeYveQRh3O8v9xY7IWlHIGclH+w3JWo7O/vi0i2FzKYKW5
> dp27tKHNM7kSt/n4vfk+C17p8LVK//aYWEVkNekZHJDdbEwEAdEfFp0VPus2CGFH
> Q5n04oqPyzhb17B2Ct4YDP6hCsm4K2/tSW+szxZv3AMZZ9n6fYzXZjftvprovIYZ
> dOCbVbhc6Tl+nvgOIoWam9vOUinZcm/vR3wlLzI41Xmiul9GL+k/LeMcAU0LuDY=
> =QzIF
> -----END PGP SIGNATURE-----
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20141103/f7e05d9f/attachment-0002.html>