[Oisf-devel] TCP Reassembly

Sat Jul 4 07:24:32 UTC 2015

On 03-07-15 22:48, Teryl Taylor wrote:
> Hi Anoop and Edward,
>
> Thanks for the response.   I find that the issue with a lot of the tcp
> reassembly libraries out there is that they throw out the reassembled
> stream  if a packet is missing.    For any form of security analysis,
> this isn't ideal because there could still be important information in
> the part of the stream that is collected.  I'd like to find a library
> that maybe fills in missing packets with zeros, and times out if it
> doesn't get a FIN after a time period.   I figure the assemblers for
> suricata, snort, and or Bro are likely more robust because they want to
> analyze the streams; however, they seem to be tightly coupled to their
> overall platforms which isn't useful when you want to build a
> lightweight application.
>
> Edward, This is a good idea as libnids doesn't seem to have been
> supported in years.  Libnids was built using the linux stack, which is
> good for reliability, but it doesn't always handle those cases that a
> security person would be interested in.  Would be nice for a simple and
> robust API.

Not a library, but Suricata in the current beta version, can output 
streaming data from 2 sources: tcp stream reassembly and HTTP bodies 
after normalizaton/decompression/dechunking.

There are 2 ways to get to this data:
1. 2 dedicated outputs (see your yaml for tcp-data and http-body-data)

2. Lua support. This gives you flexibility over the output. You can 
stream the data into files or pretty much do anything with it that Lua 
lets you do.

It might be worth checking out if this can work for your use case.

Cheers,
Victor

>
> Cheers,
>
> Teryl
>
> On Fri, Jul 3, 2015 at 5:39 AM, Edward Fjellskål
> <edwardfjellskaal at gmail.com <mailto:edwardfjellskaal at gmail.com>> wrote:
>
>     -----BEGIN PGP SIGNED MESSAGE-----
>     Hash: SHA1
>
>     I was also hoping there would be something like this out there,
>     like a updated version of libnids, but that also has IPv6.
>
>     Ive for long dreamt of coding this my self, but failed my tries so
>     far, and I dont have the time to spend on it.
>
>     Maybe one could crowdsource someone to update libnids etc? or start
>     over?
>
>     E
>
>
>     On 07/02/2015 07:49 PM, Anoop Saldanha wrote:
>     > On Sun, Jun 21, 2015 at 4:16 AM, Teryl Taylor
>     > <teryl.taylor at gmail.com <mailto:teryl.taylor at gmail.com>> wrote:
>     >> Hi everyone,
>     >>
>     >> I'm looking for a stable and fairly reliable TCP reassembler.
>     >> I've been playing around with libnids, libtins, and libntoh and
>     >> all work well, but they don't seem to work on some of the pcaps
>     >> I'm testing on, whereas wireshark does.   I was curious if
>     >> suricata's tcp reassembly is modular enough to use on it's own
>     >> and, if so,  is there any example code or test code, that would
>     >> be good to look at to get a feel for how I could integrate it?
>     >> Would the reassembly engine be a good option? Or does anyone have
>     >> an alternative suggestion?
>     >>
>     >
>     > What's the purpose?  Want to use/convert it for termination, or
>     > it's just for non-termination re-assembly?
>     >
>     -----BEGIN PGP SIGNATURE-----
>     Version: GnuPG v1
>
>     iQEcBAEBAgAGBQJVllhLAAoJEAf3kNGaI009eZ0H/ietKW9QDZZW8sSLIHaarc5K
>     0JAjpS/P7JnWI1BgEQp64lqI3Oop6MoxGs8p5TTzlh9IXei1OrWaCI3PBYjBLA1e
>     fz6q53DIR40k3dWFuRpaTvnjPkfAezA2Tv1FO150ZZP4G9/ZFkQVldGg9Oo290Au
>     IE15OjZ3VlY265mWSOE1726hkrbhCHET34Qfr+9oz/OjOU0+n+xb284PJ8YFTRHF
>     REUg1EoZu3JYEZ0p101/qVk6lqlCpvDelMeZ+sOPB8XCfu4CMaMY/kcHOF7WWX6k
>     08vMMXoWIIrkjBZPlBDEnN6kSMLgS4awNSb71azhPYF1OkD6BswTb9x/hM7fP0A=
>     =lLle
>     -----END PGP SIGNATURE-----
>     _______________________________________________
>     Suricata IDS Devel mailing list:
>     oisf-devel at openinfosecfoundation.org
>     <mailto:oisf-devel at openinfosecfoundation.org>
>     Site: http://suricata-ids.org | Participate:
>     http://suricata-ids.org/participate/
>     List:
>     https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>     Redmine: https://redmine.openinfosecfoundation.org/
>     Developer Training in Copenhagen Sept 14-18:
>     http://suricata-ids.org/training/
>
>
>
>
> _______________________________________________
> Suricata IDS Devel mailing list: oisf-devel at openinfosecfoundation.org
> Site: http://suricata-ids.org | Participate: http://suricata-ids.org/participate/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
> Redmine: https://redmine.openinfosecfoundation.org/
> Developer Training in Copenhagen Sept 14-18: http://suricata-ids.org/training/
>

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------

-------------- next part --------------
A non-text attachment was scrubbed...
Name: http-body-data.lua
Type: text/x-lua
Size: 1194 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20150704/93d6be8e/attachment-0006.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tcp-data.lua
Type: text/x-lua
Size: 1174 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20150704/93d6be8e/attachment-0007.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tcp-data2file.lua
Type: text/x-lua
Size: 733 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20150704/93d6be8e/attachment-0008.bin>