[Oisf-users] Hunting down performance bugs - ioctls

[Mark Ashley]

Yup, removing --enable-debug made a fair bit of a difference. I don't think
I need to use it in anger any more now that suricata is ported and running.

What about the --enable-profiling and --enable-profiling-locks options? Do
they have a significant hit on performance?

I discovered via their profiling functionality that removing the ioctl
issue dropped my packet loss from 67% to 64%, and now removing
--enable-debug dropped it further to between 14% to 33%, depending which
thread you ask. It's almost usable on this platform!

I'll shuffle more threads onto the busy interface and see how much I can
improve on that. Nope, going from 4 to 6 threads raised the drops from the
previous 14%-33% up to 30%-65%. Going back to one thread only on the busy
interface, after the initial boot up drops, the drops are almost zero and
the thread is ticking away at up to 100K packets/second. That leads me to
believe the locking isn't working so well on this system, I'm not sure they
are actually using atomics.

Another issue which hits the O.S. scheduler is top(1M) is pretty bad on
resources, it increases my dropped packets by 20% - 50%. Using prstat(1M)
is much nicer on the system. It's the way it's written.

ta,
Mark.



On Wed, Dec 18, 2013 at 12:57 AM, Victor Julien <lists at inliniac.net> wrote:

> On 12/17/2013 02:42 PM, Mark Ashley wrote:
> >               suricata`PrintList+0x4e4
> >               suricata`StreamTcpReassembleAppLayer+0x301
> >               suricata`StreamTcpReassembleHandleSegmentUpdateACK+0x17b
> >               suricata`StreamTcpReassembleHandleSegment+0x1a0
> >               suricata`HandleEstablishedPacketToServer+0x1888
> >               suricata`StreamTcpPacketStateEstablished+0x20c6
> >               suricata`StreamTcpPacket+0x82a
> >               suricata`StreamTcp+0x461
> >               suricata`TmThreadsSlotVarRun+0xada
> >               suricata`TmThreadsSlotProcessPkt+0x61
> >               suricata`PcapCallbackLoop+0xd0b
> >               libpcap.so.1.5.2`pcap_read_bpf+0x13d
> >              2843
>
> PrintList above suggests you've compiled with --enable-debug. This will
> be slow no matter what you do. The PrintList above for example iterates
> the TCP segment list multiple times per packet I think, or maybe just on
> segment insert. Anyhow: slow.
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20131218/95ca7175/attachment-0002.html>