[Oisf-users] Packets stucked in Nfqueue when running inline
Eric Leblond
eric at regit.org
Wed Aug 17 14:53:50 UTC 2011
Hi again,
On Wed, 2011-08-17 at 16:28 +0200, Eric Leblond wrote:
> Hello,
>
> On Tue, 2011-08-16 at 14:30 -0500, Fernando Ortiz wrote:
> > Sorry the late of the answer. I got a server to make more test in
> > production again.
>
> No problem, I was on holiday :P
>
> > I patched Suricata. I still have the same problem with packets stucked
>
> Bad news.
>
> If you have some time, could you test the attached patch.
This is not necessary to test this patch: I've continued to study the
problem. There is an issue with the code pointed out by the patch but
this can not explain the problem.
BR,
--
Eric
>
> If there is some warning message containing:
> "lost its mind"
> Then, we may be really near to have found the problem.
>
> BR,
> >
> >
> > This output is after running suricata for about 12 hours with a
> > throughput around 15 mbps
> > Colas:
> > 1 10607 200 2 65535 0 0 51706878 1
> > 2 -4206 188 2 65535 0 0 51706879 1
> >
> >
> > pkts: 103405470 Alertas: 115222
> > sessions: 617097 ssn_memcap_drop: 0
> > segment_memcap_drop: 0
> >
> >
> > I had de Log Level in "info" which as I understand it logs error,
> > warning and info. I restart suricata with Log Level = "warn" but I
> > still don't see any nfqueue warning. Only several of these two
> > entries.
> >
> >
> > <Error> (HTPHandleResponseData) -- [ERRCODE: SC_ERR_ALPARSER(59)] -
> > Error in parsing HTTP server response: [1] [htp_response.c] [360]
> > Invalid C-L field in response
> >
> >
> > <Warning> (SSLv2Decode) -- [ERRCODE: SC_ERR_ALPARSER(59)] -
> > SSLV2_MT_ERROR msg_type recived. Error encountered in establishing
> > the sslv2 session, may be version
> >
> >
> > I am running in Arch Linux, Kernel 2.6.32-lts. As anybody has comment
> > about this issue, it may look that is a problem at my side. At least
> > maybe with kernel or distro.
> >
> > 2011/6/30 Dave Remien <dave.remien at gmail.com>
> >
> >
> > On Thu, Jun 30, 2011 at 2:53 PM, Eric Leblond <eric at regit.org>
> > wrote:
> > Hi,
> >
> > Could you check by running Suricata with debug at
> > warning level ?
> >
> > SC_LOG_LEVEL=warn /path/to/suricata
> > $MY_BEAUTIFUL_CUSTOM_PARAMS
> >
> > Following a related discussion on netfilter-devel
> > mailing list, it
> > appears that the problem could be due to a verdict
> > failure. The problem
> > is that this case is detected and a message is
> > displayed but only if
> > WARN or more log level is set.
> >
> > I've tried to reproduce the problem here (single
> > powerful laptop and I
> > did not manage to make it).
> >
> >
> >
> > How many CPUs, at a question? Lots??
> >
> > As suggested on Netfilter devel mailing list, I've
> > modified the code to
> > try to reissue verdict. I attach the patch to the
> > mail. I've tested this
> > patch and it seems to work fine (at least when the
> > problem does not
> > occur).
> >
> >
> >
> > As it turns out, you can issue verdict on any packet in (or
> > not in 8-) the nfqueue. I once had code that issued verdict on
> > hundreds of K packets going backwards, in an attempt to clear
> > the queue. It'll be interesting to see if the queue gets
> > cleared with Eric's patch.
> >
> >
> > Cheers!
> >
> >
> > Dave
> >
> >
> > Could you test it ?
> >
> > On Wed, 2011-06-22 at 02:50 -0500, Fernando Ortiz
> > wrote:
> > >
> >
> >
> > > 2011/6/21 Dave Remien <dave.remien at gmail.com>
> > > That's all new enough that the old "stuck
> > packet" problem
> > > shouldn't be reappearing (was a problem up
> > until about 2.6.21
> > > or 22).
> > >
> > >
> > > Could you try running two instances of
> > Suricata, one on each
> > > queue, rather than a single instance on two
> > queues?
> > >
> > >
> > >
> > >
> > > I ran two instances of Suricata at a time packets
> > were getting
> > > stucked. I let them run for a quarter of hour, zero
> > packets stucked.
> > >
> > >
> > > Just for be sure I load balanced traffic across 4
> > queues. I ran 3
> > > instances of Suricata
> > >
> > >
> > > suricata -c /etc/suricata/suricata.yaml -q1 -q2 -D
> > > suricata -c /etc/suricata/suricata.yaml -q4 -D
> > > suricata -c /etc/suricata/suricata.yaml -q3 -D
> > >
> > >
> > > ips2 ~]# cat /proc/net/netfilter/nfnetlink_queue
> > > 1 3147 37 2 65535 0 0 325684 1
> > > 2 -4292 28 2 65535 0 0 325686 1
> > > 3 3692 0 2 65535 0 0 112386 1
> > > 4 3706 0 2 65535 0 0 112387 1
> > >
> > >
> > > That was interesting.
> >
> > > _______________________________________________
> > > Oisf-users mailing list
> > > Oisf-users at openinfosecfoundation.org
> > >
> > http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> >
> > --
> >
> >
> > Eric Leblond
> > Blog: http://home.regit.org/
> >
> >
> >
> >
> >
> > --
> > "Of course, someone who knows more about this will correct me
> > if I'm
> > wrong, and someone who knows less will correct me if I'm
> > right."
> > David Palmer (palmer at tybalt.caltech.edu)
> >
> >
> >
> >
> >
> >
> > --
> > Fernando Ortiz
> > Twitter: http://twitter.com/FernandOrtizF
> >
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
--
Eric Leblond
Blog: http://home.regit.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20110817/dacd0909/attachment.sig>
More information about the Oisf-users
mailing list