[Oisf-users] analyzing http parsing errors

Thu Dec 15 20:34:36 UTC 2011

>>The more thorough response will come form the coders but a few
>>questions if
>>I may:
>>1. the stream is just HTTP - correct?

It's a single TCP session I extracted from a bigger pcap.
The file is 81K big and shows no errors in Wireshark. (no tcp or
whatever errors)

>>2. Does it occur only when gzip is around

I extracted another session (28K) that gave a similar problem, and
this time it's not just after a gzip.
Sorry for the wrong lead.
However it's right after two responses containing JavaScript.
But not sure if that's related.

>>3. how do you compile  Suri and do you have chksms enabled or not in
>>yaml - disabled by default, but just asking?

To run (and get the many http errors) the standard ./autogen.sh &&
./configure && make

To debug: Code from git repo (2 days ago), activated debug flags., I
did need to manually remove all -O2 to -O0 to prevent a bug from
occuring. (couldn't get those -O2 gone using export with CFLAGS.)
(from a clean slate)
./autogen.sh
./configure --enable-debug --enable-debug-validation
Then went into the libhtp directory and
./configure --enable-htp-debug
make

> 4. Could the part of the pcap file triggering the behaviour be sent privately to me or to another developer?

Eric,
I'll send you the two pcap files, please treat them with good care and
do not share them without my prior consent.

Peter,
As Eric is part of the official development team (cfr website) I
prefer to send it to him atm.
Thanks for understanding.

>>On Thu, Dec 15, 2011 at 8:49 PM, Christophe Vandeplas <
>>christophe at vandeplas.com> wrote:
>>
>>> Hello,
>>>
>>> As said in a previous mail I have many many HTTP errors "Unable to
>>> match response to request" like the following error:
>>> After analysis I notice that only 20-30% of the whole TCP session is
>>> analyzed and all further HTTP requests in the same TCP session are
>>> simply ignored by Suricata.
>>>
>>> I was able to find the place and partial origin of the error by
>>adding
>>> some breakpoints, but I can't really understand what's happening in
>>> the code.
>>>
>>> Some info about my pcap: The pcap file contains many http requests in
>>> one TCP session.
>>>
>>> The engine steps at the first request, then the response, then the
>>> second request and so on.
>>> Just after a gzipped response (decoded correctly) the next packet
>>> encountered by the engine is not the next request from the pcap, but
>>> it's the response. It seems like the engine is simply skipping this
>>> http request. But as I don't really understand the code yet I don't
>>> understand the cause and can't locate the bug.
>>>
>>> I'm motivated to debug this problem and help find the real origin of
>>> the problem but I really need some guidance/help.
>>>
>>> Thanks for the assistance.
>>>
>>>
>>> On Wed, Dec 14, 2011 at 1:47 PM, Christophe Vandeplas
>>> <christophe at vandeplas.com> wrote:
>>> > Hello,
>>> >
>>> > As I have loads and loads of HTTP (and smtp) parsing errors on my
>>> Suricata
>>> > instance I wanted to analyze why they occur and try
>>debugging/solving the
>>> > issue myself. However I'm having a weird behavior with Suricata
>>once I
>>> > enable --enable-debug.
>>> >
>>> > I compiled Suricata from the git master repo.
>>> >
>>> > I load a PCAP file that throws HTTP parsing errors and get the
>>following
>>> > output.
>>> > I get the same output I run this Suricata in gdb.
>>> > Pcap file contains a single tcp session in 82kB
>>> >
>>> > [6659] 14/12/2011 -- 11:18:10 - (source-pcap-file.c:212) <Info>
>>> > (ReceivePcapFileThreadInit) -- reading pcap file
>>> > ../proxytraff-error-parsing.pcap
>>> > [2059] 14/12/2011 -- 11:18:10 - (tm-threads.c:1810) <Info>
>>> > (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 3
>>> management
>>> > threads initialized, engine started.
>>> > [6659] 14/12/2011 -- 11:18:10 - (app-layer-htp.c:550) <Error>
>>> > (HTPHandleResponseData) -- [ERRCODE: SC_ERR_ALPARSER(59)] - Error
>>in
>>> parsing
>>> > HTTP server response: [1] [htp_response.c] [677] Unable to match
>>> response to
>>> > request
>>> > [6659] 14/12/2011 -- 11:18:10 - (app-layer-parser.c:977) <Error>
>>> > (AppLayerParse) -- [ERRCODE: SC_ERR_ALPARSER(59)] - Error occured
>>in
>>> parsing
>>> > "http" app layer protocol, using network protocol 6, source IP
>>address
>>> > 10.80.96.37, destination IP address 10.7.108.10, src port 63272 and
>>dst
>>> port
>>> > 8080
>>> > [6659] 14/12/2011 -- 11:18:10 - (source-pcap-file.c:189) <Info>
>>> > (ReceivePcapFileLoop) -- pcap file end of file reached (pcap err
>>code 0)
>>> >
>>> >
>>> > When I compile ./configure --enable-debug , and load exactly the
>>same
>>> PCAP I
>>> > get the following output:
>>> > (also the same with ./configure  --enable-debug
>>> --enable-debug-validation )
>>> >
>>> > [6659] 14/12/2011 -- 11:24:37 - (source-pcap-file.c:212) <Info>
>>> > (ReceivePcapFileThreadInit) -- reading pcap file
>>> > ../proxytraff-error-parsing.pcap
>>> > [2059] 14/12/2011 -- 11:24:37 - (tm-threads.c:1810) <Info>
>>> > (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 3
>>> management
>>> > threads initialized, engine started.
>>> > Bus error: 10 (core dumped)
>>> >
>>> >
>>> > Running that DEBUG enabled Suricata in gdb I get  (After a first
>>> breakpoint
>>> > I 'continue'd )
>>> > [3091] 14/12/2011 -- 11:39:33 - (tm-threads.c:1810) <Info>
>>> > (TmThreadWaitOnThreadInit) -- all 5 packet processing threads, 3
>>> management
>>> > threads initialized, engine started.
>>> >
>>> > Program received signal EXC_BAD_ACCESS, Could not access memory.
>>> > Reason: KERN_PROTECTION_FAILURE at address: 0x00000001029920f8
>>> > [Switching to process 93462 thread 0x1a03]
>>> > 0x00000001001b5205 in ?? ()
>>> >
>>> > (gdb) bt
>>> > #0  0x00000001001b5205 in ?? ()
>>> > #1  0x00000001001acfb7 in ?? ()
>>> > #2  0x00000001001a5336 in ?? ()
>>> > #3  0x000000010018911a in ?? ()
>>> > #4  0x00000001001906c9 in ?? ()
>>> > #5  0x000000010016ddd7 in ?? ()
>>> > #6  0x000000010017b9ce in ?? ()
>>> > #7  0x00000001001667c0 in ?? ()
>>> > #8  0x000000010014fe72 in ?? ()
>>> > #9  0x00000001000140f3 in ?? ()
>>> > #10 0x000000010034872c in pcap_offline_read ()
>>> > #11 0x0000000100013814 in ?? ()
>>> > #12 0x000000010014e92b in ?? ()
>>> > #13 0x00007fff883548bf in _pthread_start ()
>>> > #14 0x00007fff88357b75 in thread_start ()
>>> >
>>> > It's weird that I don't get resolved functions in the backtrace,
>>no?
>>> > Any advice what I should do next?
>>> >
>>> > Thanks
>>> > Christophe
>>> _______________________________________________
>>> Oisf-users mailing list
>>> Oisf-users at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>
>>
>>
>>--
>>Peter Manev
>>_______________________________________________
>>Oisf-users mailing list
>>Oisf-users at openinfosecfoundation.org
>>http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> --
> Envoyé de mon téléphone Android avec K-9 Mail. Excusez la brièveté.