[Oisf-users] [Emerging-Sigs] Suricata filemagic issue leading to FN on 2009419 and probably others

[Victor Julien]

On 03/24/2012 01:06 AM, Rodrigo Montoro(Sp0oKeR) wrote:
> Why not a file into "etc/" and a configuration somewhere in suricata
> config as snort has for unicode.map file ?
> 
> I think using the suggested magic file network admins will make sure
> they cover in the correct way as the idea of unicode normalization for
> different languages.

This could work, however there are several file formats to support.
Martins libmagic used a format 5, mine a 7. So assuming there are at
least 7 versions right now. Even if only 3 are more or less recent, it
would mean ET and/or OISF would have to support 3 sets of files. Not
impossible, but it adds a burden, especially for QA I think.

Cheers,
Victor

> Regards,
> 
> On Fri, Mar 23, 2012 at 4:14 PM, Will Metcalf
> <wmetcalf at emergingthreatspro.com> wrote:
>> We are rolling these back to normal rules in the Suricata  rule sets.  This
>> will happen in today's push.
>>
>> Regards,
>>
>> Will
>>
>> On Fri, Mar 23, 2012 at 1:57 PM, Martin Holste <mcholste at gmail.com> wrote:
>>>
>>> Given how many different possible versions of the library there may be
>>> (FreeBSD, Solaris, etc.), my bet is that packaging the library with
>>> Suricata will probably lead to the fewest installation problems.
>>>
>>> On Fri, Mar 23, 2012 at 12:56 PM, Kyle Creyts <kyle.creyts at gmail.com>
>>> wrote:
>>>> Also, could just make it a requirement, unless you're distributing bins
>>>> only.
>>>>
>>>> On Mar 23, 2012 1:22 PM, "Victor Julien" <victor at inliniac.net> wrote:
>>>>>
>>>>> ET recently started using Suricata's filemagic keyword to determine
>>>>> certain file types in HTTP. Martin and I identified a serious issue
>>>>> with
>>>>> the concept. The problem is that for the file classification Suricata
>>>>> relies on libmagic and it's file definitions. It turns out that there
>>>>> is
>>>>> some variance between libmagic versions.
>>>>>
>>>>> For example and Window exec we played with, on my system (Ubuntu 11.10,
>>>>> libmagic1 5.04-5ubuntu3) returns:
>>>>>
>>>>> "PE32 executable for MS Windows (GUI) Intel 80386 32-bit"
>>>>>
>>>>> However, on Martin's SUSE install it returns:
>>>>>
>>>>> "MS-DOS executable PE  for MS Windows (GUI) Intel 80386 32-bit"
>>>>>
>>>>> This made SID 2000419 False Negative for Martin.
>>>>>
>>>>> We have tried loading the more recent Ubuntu magic definitions in
>>>>> Suricata on the SUSE system, but this failed to work as the format is
>>>>> different. So distributing a set of magic definitions with ET is not
>>>>> feasible.
>>>>>
>>>>> One option would be to have several rules, one for each version of the
>>>>> magic definition, but at this point I don't know how many variations
>>>>> exist. This is probably a maintenance nightmare anyway.
>>>>>
>>>>> Another option would be to make the match more generic, but this may
>>>>> still FN with unknown variations and may FP if it's too broad.
>>>>>
>>>>> So I think at this point it's best to revert the filemagic rules to
>>>>> their originals.
>>>>>
>>>>> In the future we may consider distributing libmagic with Suricata, like
>>>>> we do with libhtp, so that we know for sure that everyone runs the same
>>>>> version. This may not sit well with distributions shipping Suricata
>>>>> though.
>>>>>
>>>>> Ideas / comments are welcome.
>>>>>
>>>>> --
>>>>> ---------------------------------------------
>>>>> Victor Julien
>>>>> http://www.inliniac.net/
>>>>> PGP: http://www.inliniac.net/victorjulien.asc
>>>>> ---------------------------------------------
>>>>>
>>>>> _______________________________________________
>>>>> Emerging-sigs mailing list
>>>>> Emerging-sigs at lists.emergingthreats.net
>>>>> http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
>>>>>
>>>>> Support Emerging Threats! Subscribe to Emerging Threats Pro
>>>>> http://www.emergingthreatspro.com
>>>>> The ONLY place to get complete premium rulesets for Snort 2.4.0 through
>>>>> Current!
>>>>
>>>>
>>>> _______________________________________________
>>>> Emerging-sigs mailing list
>>>> Emerging-sigs at lists.emergingthreats.net
>>>> http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
>>>>
>>>> Support Emerging Threats! Subscribe to Emerging Threats Pro
>>>> http://www.emergingthreatspro.com
>>>> The ONLY place to get complete premium rulesets for Snort 2.4.0 through
>>>> Current!
>>> _______________________________________________
>>> Emerging-sigs mailing list
>>> Emerging-sigs at lists.emergingthreats.net
>>> http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
>>>
>>> Support Emerging Threats! Subscribe to Emerging Threats Pro
>>> http://www.emergingthreatspro.com
>>> The ONLY place to get complete premium rulesets for Snort 2.4.0 through
>>> Current!
>>
>>
>>
>> _______________________________________________
>> Emerging-sigs mailing list
>> Emerging-sigs at lists.emergingthreats.net
>> http://lists.emergingthreats.net/mailman/listinfo/emerging-sigs
>>
>> Support Emerging Threats! Subscribe to Emerging Threats Pro
>> http://www.emergingthreatspro.com
>> The ONLY place to get complete premium rulesets for Snort 2.4.0 through
>> Current!
> 
> 
> 


-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------