[Oisf-users] analyzing http parsing errors

[Christophe Vandeplas]

Hello,

As said in a previous mail I have many many HTTP errors "Unable to
match response to request" like the following error:
After analysis I notice that only 20-30% of the whole TCP session is
analyzed and all further HTTP requests in the same TCP session are
simply ignored by Suricata.

I was able to find the place and partial origin of the error by adding
some breakpoints, but I can't really understand what's happening in
the code.

Some info about my pcap: The pcap file contains many http requests in
one TCP session.

The engine steps at the first request, then the response, then the
second request and so on.
Just after a gzipped response (decoded correctly) the next packet
encountered by the engine is not the next request from the pcap, but
it's the response. It seems like the engine is simply skipping this
http request. But as I don't really understand the code yet I don't
understand the cause and can't locate the bug.

I'm motivated to debug this problem and help find the real origin of
the problem but I really need some guidance/help.

Thanks for the assistance.


On Wed, Dec 14, 2011 at 1:47 PM, Christophe Vandeplas
<christophe at vandeplas.com> wrote:
> Hello,
>
> As I have loads and loads of HTTP (and smtp) parsing errors on my Suricata
> instance I wanted to analyze why they occur and try debugging/solving the
> issue myself. However I'm having a weird behavior with Suricata once I
> enable --enable-debug.
>
> I compiled Suricata from the git master repo.
>
> I load a PCAP file that throws HTTP parsing errors and get the following
> output.
> I get the same output I run this Suricata in gdb.
> Pcap file contains a single tcp session in 82kB
>
> [6659] 14/12/2011 -- 11:18:10 - (source-pcap-file.c:212) <Info>
> (ReceivePcapFileThreadInit) -- reading pcap file
> ../proxytraff-error-parsing.pcap
> [2059] 14/12/2011 -- 11:18:10 - (tm-threads.c:1810) <Info>
> (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 3 management
> threads initialized, engine started.
> [6659] 14/12/2011 -- 11:18:10 - (app-layer-htp.c:550) <Error>
> (HTPHandleResponseData) -- [ERRCODE: SC_ERR_ALPARSER(59)] - Error in parsing
> HTTP server response: [1] [htp_response.c] [677] Unable to match response to
> request
> [6659] 14/12/2011 -- 11:18:10 - (app-layer-parser.c:977) <Error>
> (AppLayerParse) -- [ERRCODE: SC_ERR_ALPARSER(59)] - Error occured in parsing
> "http" app layer protocol, using network protocol 6, source IP address
> 10.80.96.37, destination IP address 10.7.108.10, src port 63272 and dst port
> 8080
> [6659] 14/12/2011 -- 11:18:10 - (source-pcap-file.c:189) <Info>
> (ReceivePcapFileLoop) -- pcap file end of file reached (pcap err code 0)
>
>
> When I compile ./configure --enable-debug , and load exactly the same PCAP I
> get the following output:
> (also the same with ./configure  --enable-debug --enable-debug-validation )
>
> [6659] 14/12/2011 -- 11:24:37 - (source-pcap-file.c:212) <Info>
> (ReceivePcapFileThreadInit) -- reading pcap file
> ../proxytraff-error-parsing.pcap
> [2059] 14/12/2011 -- 11:24:37 - (tm-threads.c:1810) <Info>
> (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 3 management
> threads initialized, engine started.
> Bus error: 10 (core dumped)
>
>
> Running that DEBUG enabled Suricata in gdb I get  (After a first breakpoint
> I 'continue'd )
> [3091] 14/12/2011 -- 11:39:33 - (tm-threads.c:1810) <Info>
> (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 3 management
> threads initialized, engine started.
>
> Program received signal EXC_BAD_ACCESS, Could not access memory.
> Reason: KERN_PROTECTION_FAILURE at address: 0x00000001029920f8
> [Switching to process 93462 thread 0x1a03]
> 0x00000001001b5205 in ?? ()
>
> (gdb) bt
> #0  0x00000001001b5205 in ?? ()
> #1  0x00000001001acfb7 in ?? ()
> #2  0x00000001001a5336 in ?? ()
> #3  0x000000010018911a in ?? ()
> #4  0x00000001001906c9 in ?? ()
> #5  0x000000010016ddd7 in ?? ()
> #6  0x000000010017b9ce in ?? ()
> #7  0x00000001001667c0 in ?? ()
> #8  0x000000010014fe72 in ?? ()
> #9  0x00000001000140f3 in ?? ()
> #10 0x000000010034872c in pcap_offline_read ()
> #11 0x0000000100013814 in ?? ()
> #12 0x000000010014e92b in ?? ()
> #13 0x00007fff883548bf in _pthread_start ()
> #14 0x00007fff88357b75 in thread_start ()
>
> It's weird that I don't get resolved functions in the backtrace, no?
> Any advice what I should do next?
>
> Thanks
> Christophe