[Discussion] Binary Signature Detection

Sun Jan 25 22:48:53 UTC 2009

My long term goals for jsunpack are to make a Snort preprocessor, but
there are obvious performance problems. I am trying to find shorter term
solutions for now and I hope to link jsunpack with detection by IDS
engines - If anyone is interested in helping achieve this goal or
volunteering JavaScript URLs or Snort hits, please contact me off list.

Blake

Josh Smith wrote:
> Martin,
>
> Thank you for the praise.  It's taken me longer than I would have
> wanted to work on this subject, as I'm still an undergrad at RIT.  I
> do work for iDefense as an Intern, and Blake (who I believe is on this
> mailing list as well) is presenting at Shmoocon on his Javascript
> de-obfuscation website and engine (http://jsunpack.jeek.org/dec/go).
> We have actually been discussing automatically making rules based upon
> user submissions to his site.
>
> -Josh
>
> On Sun, Jan 25, 2009 at 2:56 PM, Martin Holste <mcholste at gmail.com> wrote:
>   
>> I want to take a second to really commend Josh on these signatures because I
>> really think that this was a worthwhile endeavor and has really defined the
>> scope of what needs to be detected.  That said, I think that now that we
>> know how many signatures it would take, we need to find a different way of
>> doing it.  Does a standard MZ executable signature miss on any of these PEiD
>> signatures?  I would think that by definition, all packed exe's must also
>> have at least part of the standard MZ PE headers.  If that is confirmed,
>> then I would posit the following:
>>
>> There are too many ways to pack an exe to fit into an efficient rule set.
>> Polymorphic packers can easily evade existing PEiD's (see ShadowServer.org's
>> packer stats).
>> I've not had reliable results with the PEiD Snort sigs because of signature
>> overlap and false positives.
>>
>> Therefore, using the standard (and existing) exe signature which canonically
>> identifes all exe's downloaded and feeds these exe's into an
>> application-layer exe analysis framework is preferable to attempting to make
>> Snort detect this on the wire.  This can easily be accomplished (and to some
>> degree already is) with the Bro framework, (insofar that it can auto-extract
>> exe's and test them against things like known malware hashes).  If it's not
>> already done, it would be trivial to run an extracted exe through PEiD.
>> This would be far easier to maintain than an entire ruleset of Snort sigs.
>> In fact, Snort could do this as well with the "session" rule modifier and
>> Jason Brevenik's SnortUnified Perl module tailing the unified output.  This
>> would also be the perfect spot to hook into a sandbox queue.  Imagine a
>> framework where a rule like this could be written: "alert if a binary is
>> extracted which when executed contacts a domain not on this whitelist."  If
>> the binary hashes are cached and submitted to a central repository, over
>> time, a larger existing library can be created which would save a lot of
>> back-end work.
>>
>> There's a big hole in this, though, and that is a relatively new phenemenon
>> I've been coming across lately which consists of executable scripts encoded
>> in Javascript payloads, where the Javascript, after being unobfuscated, will
>> save a compressed .js file and run it using wscript.exe.  This is used to
>> replace the standard phase one downloader Trojan.  Occasionally, I've seen
>> it do phase two activities as well, like registry modification.  So, while
>> exe headers are easy to identify in a network stream, very application-level
>> Javascript implementations like this are nearly impossible to identify.  I
>> think Javascript behavior profiling and a signature language that describes
>> it is the next frontier for analysis, and I would love to see someone with
>> Josh's abilities and ambition try to attempt something in that problem
>> space.
>>
>> To whet your appetites, check out the simply amazing work UC Santa Barbara
>> has done with wepawet (wepawet.iseclab.org).  It profiles Javascript and
>> Flash in a sandbox and shows the deobfuscated and run-time output.  Check
>> out the samples there for a look at how detailed it gets.
>>
>> --Martin
>>
>> On Sun, Jan 25, 2009 at 11:03 AM, Josh Smith <famousjs at gmail.com> wrote:
>>     
>>> David,
>>>
>>> Well I originally converted the database file they offer on the PEiD
>>> website, and that was about 1500 signatures.  Now I've just been
>>> collecting database files from around the internet.  The one I
>>> originally did may be dated, but still applies to quite a few binary
>>> signatures.
>>>
>>> -Josh
>>>
>>> On Sun, Jan 25, 2009 at 11:58 AM, David Glosser <david.glosser at gmail.com>
>>> wrote:
>>>       
>>>> wow! is there any way to have a smaller list of "active" sigs? (or would
>>>> that "smaller" list still be too large for most snort installations)?
>>>>
>>>>
>>>>
>>>> On Sun, Jan 25, 2009 at 11:38 AM, Josh Smith <famousjs at gmail.com> wrote:
>>>>         
>>>>> I have been working on converting the PEiD database of binary packer
>>>>> signatures straight to snort signatures.  I've been refining my
>>>>> signatures with other members from Emerging Threats, and have over
>>>>> 10,000 snort signatures for packers.  I was told this may be a good
>>>>> topic to bring up (binary packer detection) for OISF.
>>>>>
>>>>> -Josh
>>>>> _______________________________________________
>>>>> Discussion mailing list
>>>>> Discussion at openinfosecfoundation.org
>>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>>>>>           
>>>>         
>>> _______________________________________________
>>> Discussion mailing list
>>> Discussion at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>>>       
>>     
> _______________________________________________
> Discussion mailing list
> Discussion at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>   