[Oisf-devel] Packet Array in 1.0.x
Victor Julien
victor at inliniac.net
Mon Sep 6 14:55:05 UTC 2010
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Joel Ebrahimi wrote:
> Hey Victor,
>
> In the first call to pcap_dispatch, when this statement:
>
> (pcap_max_read_packets < packet_q_len) ? *pcap_max_read_packets :
> packet_q_len
>
> pcap_max_read_packet is 0, so 0 is passed in to cnt for pcap_dispatch. A
> 0 or a -1 here causes the all the packets received in the buffer to be
> processed.
Right. In source-pcap.c we make sure we never get to that condition by
entering a wait loop that sets packet_q_len based on the number of
available packets in the packet pool. As long as it's 0 it will remain
in that loop.
Did you implement a similar protection?
> In my test cast I can see PcapCallback called 10's or thousands or
> times. It appear that depending on the pcap file Im sending that either
> memory is exhausted here in PcapCallback and the process is killed or if
> it returns then the variable ptv->array_idx is set to something
> extremely high (like 15000) and when the for loop happens after the
> pcap_dispatch, then once cnt goes passed 256 we are accessing outside
> our memory space and I get the segfault.
Problem is that it will be hard to set the array size to be big enough.
So I think choosing and then enforcing a limit is inevitable.
Cheers,
Victor
>
> // Joel
>
>> -----Original Message-----
>> From: Victor Julien [mailto:victor at inliniac.net]
>> Sent: Monday, August 23, 2010 2:01 AM
>> To: Joel Ebrahimi
>> Cc: Will Metcalf; oisf-devel at openinfosecfoundation.org
>> Subject: Re: [Oisf-devel] Packet Array in 1.0.x
>>
> Hi Joel,
>
> For the pcap module we pass a variable to pcap_dispatch that makes sure
> we read a max number of packets equal to or lower than 256, based on
>> the
> amount of available packets in the PacketPool. If none are available we
> wait. See lines 180 to 200 in source-pcap.c
>
> Cheers,
> Victor
>
> Joel Ebrahimi wrote:
>>>> Sure. As I said though its not always a segfault but I can see all
>> the
>>>> memory being used up on the system. Here is the segfault backtrace:
>>>>
>>>> #0 0x10012150 in PacketEnqueue (q=0x1080eb70, p=0x2a9) at
>>>> packet-queue.c:142
>>>> #1 0x10012f98 in ReceivePcap (tv=0x1080ea78, p=0x10265f70,
>>>> data=0x107faef8, pq=0x1080eb10, postpq=0x1080eb70)
>>>> at source-pcap.c:219
>>>> #2 0x10104f44 in TmThreadsSlot1 (td=0x1080ea78) at tm-threads.c:371
>>>> #3 0x0ff0d994 in start_thread () from /lib/libpthread.so.0
>>>> #4 0x0ff0d994 in start_thread () from /lib/libpthread.so.0
>>>> Previous frame inner to this frame (corrupt stack?)
>>>>
>>>> Really the problem seems to get ugly in the ReceivePcap function
>> here:
>>>> for (cnt = 0; cnt < ptv->array_idx; cnt++) {
>>>> Packet *pp = ptv->array[cnt];
>>>>
>>>> /* enqueue all but the first in the postpq, the first
>>>> * pkt is handled by the tv "out handler" */
>>>> if (cnt > 0) {
>>>> PacketEnqueue(postpq, pp);
>>>> }
>>>> }
>>>>
>>>> The problem being that array_idx keeps incrementing. So in that
>> specific
>>>> backtrace case above array_idx is 6538 and when this section of code
>> is
>>>> hit, once count exceeds 255 we are outside the range of memory and as
>>>> you can see from the backtrace that in the function PacketEnqueue
>> where
>>>> p's address is wrong.
>>>>
>>>> I know the array stuff is new in the 1.0.x release and looking it
>> over I
>>>> don't see a case that ensures it stays within the max 256 allocated
>> or I
>>>> don't see a case where array_idx ever gets reset.
>>>>
>>>> // Joel
>>>>
>>>>> -----Original Message-----
>>>>> From: Will Metcalf [mailto:william.metcalf at gmail.com]
>>>>> Sent: Thursday, August 19, 2010 4:36 PM
>>>>> To: Joel Ebrahimi
>>>>> Cc: oisf-devel at openinfosecfoundation.org
>>>>> Subject: Re: [Oisf-devel] Packet Array in 1.0.x
>>>>>
>>>>> Can you give us the output of a backtrace Joel? You can send
>> off-list
>>>>> if you so desire.
>>>>>
>>>>> Regards,
>>>>>
>>>>> Will
>>>>>
>>>>> On Thu, Aug 19, 2010 at 5:19 PM, Joel Ebrahimi <jebrahimi at bivio.net>
>>>> wrote:
>>>>>> I'm seeing multiple problems with the array that was added to
>>>>>> PcapThreadVars_.
>>>>>>
>>>>>> I either exhaust memory completely or segfault. I think they both
>>>> stem
>>>>>> from the same issue with is that the array seems to keep filling
>> and
>>>>>> eventually there is an access outside of PCAP_FILE_MAX_PKTS (256)
>>>> which
>>>>>> causes the segfaults.
>>>>>>
>>>>>> Is the array supposed to hold a finite number of past packets ie.
>> 256
>>>> or
>>>>>> whatever PCAP_FILE_MAX_PKTS is set to? If so then I suppose a
>> modulus
>>>>>> operator or some kind of reset is needed.
>>>>>>
>>>>>> // Joel
>>>>>>
>>>>>>
>>>>>>
>>>>>> Joel Ebrahimi
>>>>>> Solutions Architect
>>>>>> Bivio Networks
>>>>>> jebrahimi at bivio.net
>>>>>>
>>>>>> _______________________________________________
>>>>>> Oisf-devel mailing list
>>>>>> Oisf-devel at openinfosecfoundation.org
>>>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>>>
>>>> _______________________________________________
>>>> Oisf-devel mailing list
>>>> Oisf-devel at openinfosecfoundation.org
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>
- --
- ---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
- ---------------------------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkyFAMkACgkQiSMBBAuniMd6iQCeMytxiSc1DigDWNQd9qH+BVC0
GMwAn3TfL3rN8uvEGJZcXo3fU2kY0wk+
=dBGn
-----END PGP SIGNATURE-----
More information about the Oisf-devel
mailing list