[Oisf-users] Questions about stats and packet drops

[Jose Vila]

Thanks Cooper for your reply.

I've added more cores, reducing the drop rate below 2%. Can't add BPF
filters as the network is heterogeneous and I want to catch as much traffic
as possible, despite its src/dst port (I have detected some webservices in
weird ports).

I still have the same questions I posted in my first mail:

* What does exactly "tcp.reassembly_memuse" mean and in which units it is
measured? If it's measured in bytes I'm getting more than 18 Exabytes of
memory usage !!!

* I believe "tcp.segment_memcap_drop" means packets received by suricata
(thus counted in "capture.kernel_packets") but couldn't get to the (stream
or reassembly?) processor for further treatment. Which processor is the
right one? How can I reduce its value?

* I believe "tcp.stream_depth_reached" gets incremented each time the
"stream.reassembly.depth" is reached, but no packets are dropped here, they
are passed to other processors for further inspection without being
reassembled. Is this right?

* What does exactly "tcp.reassembly_gap" mean?

Thank you very much,

Regards,

Jose Vila.


On Sun, Jan 4, 2015 at 4:57 PM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Couple things you could try.
>
> 1.  Use all available cores (12 workers threads).
>
> 2.  Use a bpf filter to only monitor ports 80 and 53
>
> On 12/24/2014 12:37 AM, Jose Vila wrote:
> > Hi,
> >
> > I'm playing around with Suricata, and want to reduce the number of drops.
> >
> > I have 1000Mbits/s traffic and a server with 12 cores and 12GB of RAM.
> > The objective of this sensor is to get HTTP and DNS logging and it only
> > has a bunch of very simple rules for file extraction.
> >
> > I'm using PF_RING, and recently switched to "workers" runmode, which
> > reduced my packer drop rate (capture.kernel_drop statistic) to around
> > 5-6% with 6 worker threads.
> >
> > My memcaps are:
> > defrag.memcap = 32mb
> > flow.memcap = 256mb
> > stream.memcap = 7gb
> > stream.reassembly.memcap = 3gb
> > stream.reassembly.depth = 8mb
> >
>
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
>
> iQEcBAEBAgAGBQJUqWMAAAoJEKIFRYQsa8FWX30IAKTZvJbYsQLmMAXnr7z+yWhl
> FfcXyBkwOB5SddbAQUoBEWunqAjU2VNAVyh8w/gf5kK8mGYA87iIdGYxfz1XqNK2
> TEKqgHeYkAjCCQxtiUtYwrSHoul5slMt5HKvJg2JtVP6QchT6SwJ/srnL2n6+PSM
> FB5q3pr4oQpqwGiwQTwQlcWYVFWOpnMXKy9w9tenbDpGmx78YJZhoZ1z7cxIbAEu
> LyIImTu4Iou61a7i7b1o0LQiwxLViW7Ouw3QthIcl5OnKXIzD0xL3VGSuZLP/RY0
> uv9lA1sYdZDtRsBVS1skEc/cX3akmrADbY73Inc8em4rq9Gao0F+4Cs50LUeDJc=
> =W2mu
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20150107/2a0c801d/attachment-0002.html>