[Oisf-users] HTTP Sessions and resource estimation

[Cooper F. Nelson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Say you have lots of customers watching streaming video over HTTP (like
Netflix).  Each video stream will ultimately consume 20MB of memory
before suricata stops tracking it and releases the memory.

I also do a bit of performance analysis for HTTP proxy/cache design and
it turns out that the vast majority (over 99%) of HTTP objects are under
1 MB in size, so you really aren't getting much from tracking past that.
 I understand that TCP connections are often left open and recycled, but
most 'interesting' packets from a network security perspective are going
to be within the first MB of new flows.  In fact, most of the ET HTTP
sigs (other than the WEB_CLIENT sigs) will only trigger against the
first few packets, if at all.

- -Coop

On 3/19/2015 10:54 AM, Yasha Zislin wrote:
> Can you explain what it is? and how it affect memory utilization?
> 
> Thanks.
> 
>> Date: Thu, 19 Mar 2015 09:29:12 -0700
>> From: cnelson at ucsd.edu
>> To: coolyasha at hotmail.com; oisf-users at lists.openinfosecfoundation.org
>> Subject: Re: [Oisf-users] HTTP Sessions and resource estimation
>>
> I think that is too high a stream reassembly depth. Try 1mb instead.
> 
> On 3/19/2015 7:19 AM, Yasha Zislin wrote:
> 
>> My stream reassembly depth is set to 20mb. I forget why it is so high,
>> but I've made it to minimize packet loss.
> 
>> I am monitoring two span ports (about 1gig each) and my 40 logical
>> CPUs/140 gigs of RAM server is using 95% of RAM.
>> I thought Suricata was able to handle 10 gig feeds. Just trying to
>> understand what I am doing wrong.
> 
>> Thanks.
> 
> 
> 

- -- 
Cooper Nelson
Network Security Analyst
UCSD ACT Security Team
cnelson at ucsd.edu x41042
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)

iQEcBAEBAgAGBQJVCxPYAAoJEKIFRYQsa8FWfEwH/2hgwdAkdj3eL+DarFDweMLC
n7EWdYzpgBG3J01Fod8l8zk7lrv5GpzkHHq9seZq9AnCStfky1zYku0dor8JibvN
4XTKdqMlLludYgkRqXpH9P09qp7dhZru+qNtFHEKT9M2Vb3LYbE9iAAF58HjcRMk
ffWAMKh7ojJLfIRH8M8zHpnyj0/+LDfJR6Rze5D/gKSI3SD17bO6ztO+hMhJnbq2
bKEGPOF4MHGyG/EVMV0LU+wivUa9QkJAPP3FlgAJE5YKfPULve5ZZGYF5kKt87T7
j9iJMIP2V8TCtlK4bPMDm17lzn4b1VAMN/eFGTspvEKkcEM1Y1PC+VXdgp/utPU=
=m6pY
-----END PGP SIGNATURE-----