From shadowbq at gmail.com  Sun Sep 20 18:16:37 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Sun, 20 Sep 2009 18:16:37 -0400
Subject: [Oisf-wg-ruleslanguage] Intrusion Detection Message Exchange Format
	(IDMEF)
Message-ID: <d6c2aca00909201516n131eb6f4pd65b0bdf1ad686dd@mail.gmail.com>

Just an Idea(tm) : => Intrusion Detection Message Exchange Format (IDMEF)
http://www.ietf.org/rfc/rfc4765.txt

Please let us not forget about ietf efforts and the like. This rfc is
for the alerting output, but it could go a long way into writing an
xml language dtd.

I understand that snort rules are "user standard" and arcsight CEF is
almost the same for the output data.. but let's look at mitre and
other "documentation" before we run out and write YAML, or XML.

FYI: One reason that snort rules are so hard to deal with is there is
NO?! bison/yacc grammar definition file. We must clearly define and
use strict versioning for what ever grammar file is defined.

------------------
Example:
(this information should be in a rule checker and public DTD[if using
xml] definition page. DTD should also be bundled with superIDS source
for offline use.)

superIDS v.01 compact with rule_def 0.1 0.2
superIDS v.02 compact with rule_def 0.1 0.2 0.3
superIDS v.04 compact with rule_def 0.1 0.2 0.3
++ (warning: rule_def 0.1 use depricated)

superIDS v.11 compact with rule_def 0.2 0.3
++ (error: rule_def 0.1 use obsolete)

---------------

I want to validate the rules configuration (XML) not run a binary
snort -T !!! There are really good reasons for this.. pushing a rule
file to a globally distributed ids implementation is time consuming
and never as easy as it sounds.

---------------

If I have to open up the snort source one more time Marty to count the
splits, cases, and loops I will go mad...lol

Thoughts on:
YAML - easy, but requires strict tabs and spaces (no like..)
XML - Can be easy, but can be overly complicated like the IDMEF example..

We could support multiple input types.. simple xsd/xsl to transform
xml to yaml.. just an idea. (works when there is a clearly defined
language..
Examples:
http://www.yaml.org/xml.html
http://www.unfitforprint.com/articles/2005/09/23/yaml2xml-in-33-lines-or-your-money-back

~~~~ shadowbq

From shadowbq at gmail.com  Sun Sep 20 20:02:15 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Sun, 20 Sep 2009 20:02:15 -0400
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
Message-ID: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>

My personal opinion :=> "lets please not do this.." (sniffle)

---
FYI: There are some decent articles on encrypting xml data..

http://dotnetslackers.com/articles/xml/XMLEncryption.aspx

---

IDEAS:

If the organization as whole wants to encrypt some of the rules so
that we can get "pre-release patches" this would be my route:

Create an CA and X.509 certificate chain for the root signing of
Asymmetric encryption of the rules.

Have anyone/company who wants to have access to utilize encrypted
rules register for a certificate with the OSIF organization
Certificate Authority.

Anyone without a certificate can utilize the /rule file/ without
error, but the system just skips the encrypted rules.

Allow multiple X.509 chains for rule decryption, such that an
organization can create its own encryption cert and have "propriety
rules" mixed with "org subscription rules" and "org community rules".

This is never going to be trivial....

Having something like this.. rough sketch of an xml encrypted rule file...

The below uses RSA to decode off a keychain named OSIF

Reference: http://www.w3.org/TR/xmlenc-core/

<?xml version="1.0" standalone="no"?>
<rules>
  <rule>
    <title>OSIF Pre-Release $soft GDI overflow </title>
    <vulnerability>
      <title>$soft GDI overflow </title>
      <para>This space can have a more detailed description.</para>
      <para>This space can have additional description.</para>
      <releasedate>01-01-2039</releasedate>
      <url>https://www.osif.org/rules/reference/[$sid]</url>
      <author>OSIF blah....</author>
	  <status>Limited Public Release</status>
    </vulnerability>
    <tracking>
      <version>1.0</version>
      <classtype>bad-unknown</classtype>
      <priority>1</priority>
	  <id>112273645111111</id>
    </tracking>
    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
xmlns="http://www.w3.org/2001/04/xmlenc#">
	  <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
	  <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
		<KeyName>osif</KeyName>
	  </KeyInfo>
	  <CipherData>
		<CipherValue>
		<![CDATA[
		1892379812730981273091287301298731290837019283709
		1238712098379182736897126398761293876192876912876
		ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
		98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
		]]>
		</CipherValue>
	  </CipherData>
    </EncryptedData>
  </rule>
</rules>

~~~~ shadowbq

From brectanu at gmail.com  Mon Sep 21 02:28:14 2009
From: brectanu at gmail.com (Brian Rectanus)
Date: Sun, 20 Sep 2009 23:28:14 -0700
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
In-Reply-To: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>
References: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>
Message-ID: <b61a4e530909202328w611e491et64193b826afbf1d@mail.gmail.com>

I agree we should not do it.  What does it give us, really?  The end
user that gets these rules can still decrypt it and send it to
whomever they wish, so the rules are not "protected" other than from
someone that should not have had access to download them in the first
place and that should just be protected in another manner (ie do not
let unauthorized users download them).  The same could be handled (and
better, I think) through a legal document/NDA.

-B

On Sun, Sep 20, 2009 at 5:02 PM, Scott MacGregor <shadowbq at gmail.com> wrote:
> My personal opinion :=> "lets please not do this.." (sniffle)
>
> ---
> FYI: There are some decent articles on encrypting xml data..
>
> http://dotnetslackers.com/articles/xml/XMLEncryption.aspx
>
> ---
>
> IDEAS:
>
> If the organization as whole wants to encrypt some of the rules so
> that we can get "pre-release patches" this would be my route:
>
> Create an CA and X.509 certificate chain for the root signing of
> Asymmetric encryption of the rules.
>
> Have anyone/company who wants to have access to utilize encrypted
> rules register for a certificate with the OSIF organization
> Certificate Authority.
>
> Anyone without a certificate can utilize the /rule file/ without
> error, but the system just skips the encrypted rules.
>
> Allow multiple X.509 chains for rule decryption, such that an
> organization can create its own encryption cert and have "propriety
> rules" mixed with "org subscription rules" and "org community rules".
>
> This is never going to be trivial....
>
> Having something like this.. rough sketch of an xml encrypted rule file...
>
> The below uses RSA to decode off a keychain named OSIF
>
> Reference: http://www.w3.org/TR/xmlenc-core/
>
> <?xml version="1.0" standalone="no"?>
> <rules>
> ?<rule>
> ? ?<title>OSIF Pre-Release $soft GDI overflow </title>
> ? ?<vulnerability>
> ? ? ?<title>$soft GDI overflow </title>
> ? ? ?<para>This space can have a more detailed description.</para>
> ? ? ?<para>This space can have additional description.</para>
> ? ? ?<releasedate>01-01-2039</releasedate>
> ? ? ?<url>https://www.osif.org/rules/reference/[$sid]</url>
> ? ? ?<author>OSIF blah....</author>
> ? ? ? ? ?<status>Limited Public Release</status>
> ? ?</vulnerability>
> ? ?<tracking>
> ? ? ?<version>1.0</version>
> ? ? ?<classtype>bad-unknown</classtype>
> ? ? ?<priority>1</priority>
> ? ? ? ? ?<id>112273645111111</id>
> ? ?</tracking>
> ? ?<EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
> xmlns="http://www.w3.org/2001/04/xmlenc#">
> ? ? ? ? ?<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
> ? ? ? ? ?<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
> ? ? ? ? ? ? ? ?<KeyName>osif</KeyName>
> ? ? ? ? ?</KeyInfo>
> ? ? ? ? ?<CipherData>
> ? ? ? ? ? ? ? ?<CipherValue>
> ? ? ? ? ? ? ? ?<![CDATA[
> ? ? ? ? ? ? ? ?1892379812730981273091287301298731290837019283709
> ? ? ? ? ? ? ? ?1238712098379182736897126398761293876192876912876
> ? ? ? ? ? ? ? ?ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
> ? ? ? ? ? ? ? ?98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
> ? ? ? ? ? ? ? ?]]>
> ? ? ? ? ? ? ? ?</CipherValue>
> ? ? ? ? ?</CipherData>
> ? ?</EncryptedData>
> ?</rule>
> </rules>
>
> ~~~~ shadowbq
> _______________________________________________
> Oisf-wg-ruleslanguage mailing list
> Oisf-wg-ruleslanguage at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>

From shadowbq at gmail.com  Mon Sep 21 10:23:11 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Mon, 21 Sep 2009 10:23:11 -0400
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
In-Reply-To: <b61a4e530909202328w611e491et64193b826afbf1d@mail.gmail.com>
References: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>
	<b61a4e530909202328w611e491et64193b826afbf1d@mail.gmail.com>
Message-ID: <d6c2aca00909210723p41c8b6e9ncdbcba0d2b132075@mail.gmail.com>

No matter how you do this.. the end user /engineer will be able to
decrypt the rule off the box. Its not ok to run a "black box" on my
network, and it will never be ok.. That being said, I would recommend
digital signing as an authorship verification.. much later in
development.

NDA will be required for the certificate issuance in any manner if we
end up down this route.

Just an Idea(tm).. If we came up with our own way of distributing
rules.. requiring certs (X509), a SSL server, common NDA template, a
common web retrieval API with front end example app (rubyrails app/php
app/py egg) etc.. open source and document the exchange process so
that $soft, redhat, my own company could all be rule "sources"
..maybe... oinkmaster++ .. NDA with each rule source server etc...

If OSIF wanted to continue down the encryption in the rule file: = >
It would take a significant effort to a accomplish rule distributions
that are signed by each individuals public key for rule issuance. This
could cause bursting high volume requests that require a lot of
encryption processing on the OSIF servers on "new rule days/pre-patch
Mondays".

Reminder: A significant problem not faced with other applications is
that IDS are often place out-of-band and are not capable of
communicating back to a online website for bi-directional
communications during operation.



On Mon, Sep 21, 2009 at 2:28 AM, Brian Rectanus <brectanu at gmail.com> wrote:
> I agree we should not do it. ?What does it give us, really? ?The end
> user that gets these rules can still decrypt it and send it to
> whomever they wish, so the rules are not "protected" other than from
> someone that should not have had access to download them in the first
> place and that should just be protected in another manner (ie do not
> let unauthorized users download them). ?The same could be handled (and
> better, I think) through a legal document/NDA.
>
> -B
>
> On Sun, Sep 20, 2009 at 5:02 PM, Scott MacGregor <shadowbq at gmail.com> wrote:
>> My personal opinion :=> "lets please not do this.." (sniffle)
>>
>> ---
>> FYI: There are some decent articles on encrypting xml data..
>>
>> http://dotnetslackers.com/articles/xml/XMLEncryption.aspx
>>
>> ---
>>
>> IDEAS:
>>
>> If the organization as whole wants to encrypt some of the rules so
>> that we can get "pre-release patches" this would be my route:
>>
>> Create an CA and X.509 certificate chain for the root signing of
>> Asymmetric encryption of the rules.
>>
>> Have anyone/company who wants to have access to utilize encrypted
>> rules register for a certificate with the OSIF organization
>> Certificate Authority.
>>
>> Anyone without a certificate can utilize the /rule file/ without
>> error, but the system just skips the encrypted rules.
>>
>> Allow multiple X.509 chains for rule decryption, such that an
>> organization can create its own encryption cert and have "propriety
>> rules" mixed with "org subscription rules" and "org community rules".
>>
>> This is never going to be trivial....
>>
>> Having something like this.. rough sketch of an xml encrypted rule file...
>>
>> The below uses RSA to decode off a keychain named OSIF
>>
>> Reference: http://www.w3.org/TR/xmlenc-core/
>>
>> <?xml version="1.0" standalone="no"?>
>> <rules>
>> ?<rule>
>> ? ?<title>OSIF Pre-Release $soft GDI overflow </title>
>> ? ?<vulnerability>
>> ? ? ?<title>$soft GDI overflow </title>
>> ? ? ?<para>This space can have a more detailed description.</para>
>> ? ? ?<para>This space can have additional description.</para>
>> ? ? ?<releasedate>01-01-2039</releasedate>
>> ? ? ?<url>https://www.osif.org/rules/reference/[$sid]</url>
>> ? ? ?<author>OSIF blah....</author>
>> ? ? ? ? ?<status>Limited Public Release</status>
>> ? ?</vulnerability>
>> ? ?<tracking>
>> ? ? ?<version>1.0</version>
>> ? ? ?<classtype>bad-unknown</classtype>
>> ? ? ?<priority>1</priority>
>> ? ? ? ? ?<id>112273645111111</id>
>> ? ?</tracking>
>> ? ?<EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
>> xmlns="http://www.w3.org/2001/04/xmlenc#">
>> ? ? ? ? ?<EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
>> ? ? ? ? ?<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
>> ? ? ? ? ? ? ? ?<KeyName>osif</KeyName>
>> ? ? ? ? ?</KeyInfo>
>> ? ? ? ? ?<CipherData>
>> ? ? ? ? ? ? ? ?<CipherValue>
>> ? ? ? ? ? ? ? ?<![CDATA[
>> ? ? ? ? ? ? ? ?1892379812730981273091287301298731290837019283709
>> ? ? ? ? ? ? ? ?1238712098379182736897126398761293876192876912876
>> ? ? ? ? ? ? ? ?ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
>> ? ? ? ? ? ? ? ?98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
>> ? ? ? ? ? ? ? ?]]>
>> ? ? ? ? ? ? ? ?</CipherValue>
>> ? ? ? ? ?</CipherData>
>> ? ?</EncryptedData>
>> ?</rule>
>> </rules>
>>
>> ~~~~ shadowbq
>> _______________________________________________
>> Oisf-wg-ruleslanguage mailing list
>> Oisf-wg-ruleslanguage at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>>
> _______________________________________________
> Oisf-wg-ruleslanguage mailing list
> Oisf-wg-ruleslanguage at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>

From shadowbq at gmail.com  Mon Sep 21 11:35:30 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Mon, 21 Sep 2009 11:35:30 -0400
Subject: [Oisf-wg-ruleslanguage] Supporting languages / Getting things done..
Message-ID: <d6c2aca00909210835l58cf570eq57c61d2365b277fb@mail.gmail.com>

What languages to support as external scripts that can feed
information back to a rule (i.e. a function for a rule to call). Perl,
Ruby, Python? All?

YAML..as markup. (so many bindings)

Scripting languages can be utilized to create the YAML files "on the
fly". (RUBY PERL PYTHON) This is the important thing about the way the
rules are loaded, they need to be able to be loaded via a listening
API services. The API starts, I run script, walla my ruby code loaded
a new rule configuration. This way we can write XML, YAML, ruby,
python, java whatever is in our hearts.. as long as the output of the
scripts creates YAML to be loaded by API.

ON Self Contained Rules:..

Many people feel rule configurations need to be self contained..

This can be solved with reverse distribution scripts.

ON: "seperate the matching and actions pieces"

> This allows users to reuse/inherit matching and lets users build their
> own logic for a sequence of things to do when matches occur.  What do
> you all think?

Good idea

ON : Relational Trees / Stackable Rule files.

Good Idea

Reverse Distribution Example / Fully Expanded Rules.. (For debugging
and redistribution.)

Take an array of signature ids, put it in a text file, and run it
through a (ruby) script that loads all the YAML rules files. It would
output individual distributional rules filling all the variables with
actual values and action modules combined.


Remember :

From jdickey at seven-sigma.com  Tue Sep 22 23:36:08 2009
From: jdickey at seven-sigma.com (Jeff Dickey)
Date: Wed, 23 Sep 2009 11:36:08 +0800
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
In-Reply-To: <d6c2aca00909210723p41c8b6e9ncdbcba0d2b132075@mail.gmail.com>
Message-ID: <C6DFB8A8.1D9D5%jdickey@seven-sigma.com>

I'm late to the party, but I'll put my two rupiah in.... What I seem to be
hearing is what I've believed for some time, that the "traditional" X509 and
Friends toolchain is a heavier burden than we may want to deal with
(technically/financially/other areas?)... Has anybody given any serious
thought to making something like GPG work for signing/authentication?

Jeff

(who also thinks we *so* need Google Wave for these discussions....)


On 21/9/09 22:23 , "Scott MacGregor" <shadowbq at gmail.com> wrote:

> No matter how you do this.. the end user /engineer will be able to
> decrypt the rule off the box. Its not ok to run a "black box" on my
> network, and it will never be ok.. That being said, I would recommend
> digital signing as an authorship verification.. much later in
> development.
> 
> NDA will be required for the certificate issuance in any manner if we
> end up down this route.
> 
> Just an Idea(tm).. If we came up with our own way of distributing
> rules.. requiring certs (X509), a SSL server, common NDA template, a
> common web retrieval API with front end example app (rubyrails app/php
> app/py egg) etc.. open source and document the exchange process so
> that $soft, redhat, my own company could all be rule "sources"
> ..maybe... oinkmaster++ .. NDA with each rule source server etc...
> 
> If OSIF wanted to continue down the encryption in the rule file: = >
> It would take a significant effort to a accomplish rule distributions
> that are signed by each individuals public key for rule issuance. This
> could cause bursting high volume requests that require a lot of
> encryption processing on the OSIF servers on "new rule days/pre-patch
> Mondays".
> 
> Reminder: A significant problem not faced with other applications is
> that IDS are often place out-of-band and are not capable of
> communicating back to a online website for bi-directional
> communications during operation.
> 
> 
> 
> On Mon, Sep 21, 2009 at 2:28 AM, Brian Rectanus <brectanu at gmail.com> wrote:
>> I agree we should not do it. ?What does it give us, really? ?The end
>> user that gets these rules can still decrypt it and send it to
>> whomever they wish, so the rules are not "protected" other than from
>> someone that should not have had access to download them in the first
>> place and that should just be protected in another manner (ie do not
>> let unauthorized users download them). ?The same could be handled (and
>> better, I think) through a legal document/NDA.
>> 
>> -B
>> 
>> On Sun, Sep 20, 2009 at 5:02 PM, Scott MacGregor <shadowbq at gmail.com> wrote:
>>> My personal opinion :=> "lets please not do this.." (sniffle)
>>> 
>>> ---
>>> FYI: There are some decent articles on encrypting xml data..
>>> 
>>> http://dotnetslackers.com/articles/xml/XMLEncryption.aspx
>>> 
>>> ---
>>> 
>>> IDEAS:
>>> 
>>> If the organization as whole wants to encrypt some of the rules so
>>> that we can get "pre-release patches" this would be my route:
>>> 
>>> Create an CA and X.509 certificate chain for the root signing of
>>> Asymmetric encryption of the rules.
>>> 
>>> Have anyone/company who wants to have access to utilize encrypted
>>> rules register for a certificate with the OSIF organization
>>> Certificate Authority.
>>> 
>>> Anyone without a certificate can utilize the /rule file/ without
>>> error, but the system just skips the encrypted rules.
>>> 
>>> Allow multiple X.509 chains for rule decryption, such that an
>>> organization can create its own encryption cert and have "propriety
>>> rules" mixed with "org subscription rules" and "org community rules".
>>> 
>>> This is never going to be trivial....
>>> 
>>> Having something like this.. rough sketch of an xml encrypted rule file...
>>> 
>>> The below uses RSA to decode off a keychain named OSIF
>>> 
>>> Reference: http://www.w3.org/TR/xmlenc-core/
>>> 
>>> <?xml version="1.0" standalone="no"?>
>>> <rules>
>>> ?<rule>
>>> ? ?<title>OSIF Pre-Release $soft GDI overflow </title>
>>> ? ?<vulnerability>
>>> ? ? ?<title>$soft GDI overflow </title>
>>> ? ? ?<para>This space can have a more detailed description.</para>
>>> ? ? ?<para>This space can have additional description.</para>
>>> ? ? ?<releasedate>01-01-2039</releasedate>
>>> ? ? ?<url>https://www.osif.org/rules/reference/[$sid]</url>
>>> ? ? ?<author>OSIF blah....</author>
>>> ? ? ? ? ?<status>Limited Public Release</status>
>>> ? ?</vulnerability>
>>> ? ?<tracking>
>>> ? ? ?<version>1.0</version>
>>> ? ? ?<classtype>bad-unknown</classtype>
>>> ? ? ?<priority>1</priority>
>>> ? ? ? ? ?<id>112273645111111</id>
>>> ? ?</tracking>
>>> ? ?<EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
>>> xmlns="http://www.w3.org/2001/04/xmlenc#">
>>> ? ? ? ? ?<EncryptionMethod
>>> Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
>>> ? ? ? ? ?<KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
>>> ? ? ? ? ? ? ? ?<KeyName>osif</KeyName>
>>> ? ? ? ? ?</KeyInfo>
>>> ? ? ? ? ?<CipherData>
>>> ? ? ? ? ? ? ? ?<CipherValue>
>>> ? ? ? ? ? ? ? ?<![CDATA[
>>> ? ? ? ? ? ? ? ?1892379812730981273091287301298731290837019283709
>>> ? ? ? ? ? ? ? ?1238712098379182736897126398761293876192876912876
>>> ? ? ? ? ? ? ? ?ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
>>> ? ? ? ? ? ? ? ?98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
>>> ? ? ? ? ? ? ? ?]]>
>>> ? ? ? ? ? ? ? ?</CipherValue>
>>> ? ? ? ? ?</CipherData>
>>> ? ?</EncryptedData>
>>> ?</rule>
>>> </rules>
>>> 
>>> ~~~~ shadowbq
>>> _______________________________________________
>>> Oisf-wg-ruleslanguage mailing list
>>> Oisf-wg-ruleslanguage at openinfosecfoundation.org
>>> 
http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguag>>>
e
>>> 
>> _______________________________________________
>> Oisf-wg-ruleslanguage mailing list
>> Oisf-wg-ruleslanguage at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>> 
> _______________________________________________
> Oisf-wg-ruleslanguage mailing list
> Oisf-wg-ruleslanguage at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage

-- 
Jeff Dickey         http://archlever.blogspot.com/
Email:              jdickey at seven-sigma.com
Phone/SMS:          +65 8333 4403
Skype:              jeff_dickey
LinkedIn:           jdickey
Yahoo! IM:          jeff_dickey
MSN IM:             jeff_dickey at hotmail.com (for IM only, please)
ICQ:                8053918
QQ:                 30302349
GnuPG key:
    Fingerprint     D367 FB97 4E59 BEC0 8EBC  D8E3 3BD4 7D4C DFE0 6488
    Valid from      01 July 2009 to 31 December 2009
    Download from   http://tr.im/qqQa



From shadowbq at gmail.com  Sun Sep 20 22:16:37 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Sun, 20 Sep 2009 18:16:37 -0400
Subject: [Oisf-wg-ruleslanguage] Intrusion Detection Message Exchange Format
	(IDMEF)
Message-ID: <d6c2aca00909201516n131eb6f4pd65b0bdf1ad686dd@mail.gmail.com>

Just an Idea(tm) : => Intrusion Detection Message Exchange Format (IDMEF)
http://www.ietf.org/rfc/rfc4765.txt

Please let us not forget about ietf efforts and the like. This rfc is
for the alerting output, but it could go a long way into writing an
xml language dtd.

I understand that snort rules are "user standard" and arcsight CEF is
almost the same for the output data.. but let's look at mitre and
other "documentation" before we run out and write YAML, or XML.

FYI: One reason that snort rules are so hard to deal with is there is
NO?! bison/yacc grammar definition file. We must clearly define and
use strict versioning for what ever grammar file is defined.

------------------
Example:
(this information should be in a rule checker and public DTD[if using
xml] definition page. DTD should also be bundled with superIDS source
for offline use.)

superIDS v.01 compact with rule_def 0.1 0.2
superIDS v.02 compact with rule_def 0.1 0.2 0.3
superIDS v.04 compact with rule_def 0.1 0.2 0.3
++ (warning: rule_def 0.1 use depricated)

superIDS v.11 compact with rule_def 0.2 0.3
++ (error: rule_def 0.1 use obsolete)

---------------

I want to validate the rules configuration (XML) not run a binary
snort -T !!! There are really good reasons for this.. pushing a rule
file to a globally distributed ids implementation is time consuming
and never as easy as it sounds.

---------------

If I have to open up the snort source one more time Marty to count the
splits, cases, and loops I will go mad...lol

Thoughts on:
YAML - easy, but requires strict tabs and spaces (no like..)
XML - Can be easy, but can be overly complicated like the IDMEF example..

We could support multiple input types.. simple xsd/xsl to transform
xml to yaml.. just an idea. (works when there is a clearly defined
language..
Examples:
http://www.yaml.org/xml.html
http://www.unfitforprint.com/articles/2005/09/23/yaml2xml-in-33-lines-or-your-money-back

~~~~ shadowbq


From shadowbq at gmail.com  Mon Sep 21 00:02:15 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Sun, 20 Sep 2009 20:02:15 -0400
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
Message-ID: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>

My personal opinion :=> "lets please not do this.." (sniffle)

---
FYI: There are some decent articles on encrypting xml data..

http://dotnetslackers.com/articles/xml/XMLEncryption.aspx

---

IDEAS:

If the organization as whole wants to encrypt some of the rules so
that we can get "pre-release patches" this would be my route:

Create an CA and X.509 certificate chain for the root signing of
Asymmetric encryption of the rules.

Have anyone/company who wants to have access to utilize encrypted
rules register for a certificate with the OSIF organization
Certificate Authority.

Anyone without a certificate can utilize the /rule file/ without
error, but the system just skips the encrypted rules.

Allow multiple X.509 chains for rule decryption, such that an
organization can create its own encryption cert and have "propriety
rules" mixed with "org subscription rules" and "org community rules".

This is never going to be trivial....

Having something like this.. rough sketch of an xml encrypted rule file...

The below uses RSA to decode off a keychain named OSIF

Reference: http://www.w3.org/TR/xmlenc-core/

<?xml version="1.0" standalone="no"?>
<rules>
  <rule>
    <title>OSIF Pre-Release $soft GDI overflow </title>
    <vulnerability>
      <title>$soft GDI overflow </title>
      <para>This space can have a more detailed description.</para>
      <para>This space can have additional description.</para>
      <releasedate>01-01-2039</releasedate>
      <url>https://www.osif.org/rules/reference/[$sid]</url>
      <author>OSIF blah....</author>
	  <status>Limited Public Release</status>
    </vulnerability>
    <tracking>
      <version>1.0</version>
      <classtype>bad-unknown</classtype>
      <priority>1</priority>
	  <id>112273645111111</id>
    </tracking>
    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
xmlns="http://www.w3.org/2001/04/xmlenc#">
	  <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
	  <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
		<KeyName>osif</KeyName>
	  </KeyInfo>
	  <CipherData>
		<CipherValue>
		<![CDATA[
		1892379812730981273091287301298731290837019283709
		1238712098379182736897126398761293876192876912876
		ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
		98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
		]]>
		</CipherValue>
	  </CipherData>
    </EncryptedData>
  </rule>
</rules>

~~~~ shadowbq


From brectanu at gmail.com  Mon Sep 21 06:28:14 2009
From: brectanu at gmail.com (Brian Rectanus)
Date: Sun, 20 Sep 2009 23:28:14 -0700
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
In-Reply-To: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>
References: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>
Message-ID: <b61a4e530909202328w611e491et64193b826afbf1d@mail.gmail.com>

I agree we should not do it.  What does it give us, really?  The end
user that gets these rules can still decrypt it and send it to
whomever they wish, so the rules are not "protected" other than from
someone that should not have had access to download them in the first
place and that should just be protected in another manner (ie do not
let unauthorized users download them).  The same could be handled (and
better, I think) through a legal document/NDA.

-B

On Sun, Sep 20, 2009 at 5:02 PM, Scott MacGregor <shadowbq at gmail.com> wrote:
> My personal opinion :=> "lets please not do this.." (sniffle)
>
> ---
> FYI: There are some decent articles on encrypting xml data..
>
> http://dotnetslackers.com/articles/xml/XMLEncryption.aspx
>
> ---
>
> IDEAS:
>
> If the organization as whole wants to encrypt some of the rules so
> that we can get "pre-release patches" this would be my route:
>
> Create an CA and X.509 certificate chain for the root signing of
> Asymmetric encryption of the rules.
>
> Have anyone/company who wants to have access to utilize encrypted
> rules register for a certificate with the OSIF organization
> Certificate Authority.
>
> Anyone without a certificate can utilize the /rule file/ without
> error, but the system just skips the encrypted rules.
>
> Allow multiple X.509 chains for rule decryption, such that an
> organization can create its own encryption cert and have "propriety
> rules" mixed with "org subscription rules" and "org community rules".
>
> This is never going to be trivial....
>
> Having something like this.. rough sketch of an xml encrypted rule file...
>
> The below uses RSA to decode off a keychain named OSIF
>
> Reference: http://www.w3.org/TR/xmlenc-core/
>
> <?xml version="1.0" standalone="no"?>
> <rules>
>  <rule>
>    <title>OSIF Pre-Release $soft GDI overflow </title>
>    <vulnerability>
>      <title>$soft GDI overflow </title>
>      <para>This space can have a more detailed description.</para>
>      <para>This space can have additional description.</para>
>      <releasedate>01-01-2039</releasedate>
>      <url>https://www.osif.org/rules/reference/[$sid]</url>
>      <author>OSIF blah....</author>
>          <status>Limited Public Release</status>
>    </vulnerability>
>    <tracking>
>      <version>1.0</version>
>      <classtype>bad-unknown</classtype>
>      <priority>1</priority>
>          <id>112273645111111</id>
>    </tracking>
>    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
> xmlns="http://www.w3.org/2001/04/xmlenc#">
>          <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
>          <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
>                <KeyName>osif</KeyName>
>          </KeyInfo>
>          <CipherData>
>                <CipherValue>
>                <![CDATA[
>                1892379812730981273091287301298731290837019283709
>                1238712098379182736897126398761293876192876912876
>                ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
>                98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
>                ]]>
>                </CipherValue>
>          </CipherData>
>    </EncryptedData>
>  </rule>
> </rules>
>
> ~~~~ shadowbq
> _______________________________________________
> Oisf-wg-ruleslanguage mailing list
> Oisf-wg-ruleslanguage at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>


From shadowbq at gmail.com  Mon Sep 21 14:23:11 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Mon, 21 Sep 2009 10:23:11 -0400
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
In-Reply-To: <b61a4e530909202328w611e491et64193b826afbf1d@mail.gmail.com>
References: <d6c2aca00909201702t744bb073oe6b200cc110dbae4@mail.gmail.com>
	<b61a4e530909202328w611e491et64193b826afbf1d@mail.gmail.com>
Message-ID: <d6c2aca00909210723p41c8b6e9ncdbcba0d2b132075@mail.gmail.com>

No matter how you do this.. the end user /engineer will be able to
decrypt the rule off the box. Its not ok to run a "black box" on my
network, and it will never be ok.. That being said, I would recommend
digital signing as an authorship verification.. much later in
development.

NDA will be required for the certificate issuance in any manner if we
end up down this route.

Just an Idea(tm).. If we came up with our own way of distributing
rules.. requiring certs (X509), a SSL server, common NDA template, a
common web retrieval API with front end example app (rubyrails app/php
app/py egg) etc.. open source and document the exchange process so
that $soft, redhat, my own company could all be rule "sources"
..maybe... oinkmaster++ .. NDA with each rule source server etc...

If OSIF wanted to continue down the encryption in the rule file: = >
It would take a significant effort to a accomplish rule distributions
that are signed by each individuals public key for rule issuance. This
could cause bursting high volume requests that require a lot of
encryption processing on the OSIF servers on "new rule days/pre-patch
Mondays".

Reminder: A significant problem not faced with other applications is
that IDS are often place out-of-band and are not capable of
communicating back to a online website for bi-directional
communications during operation.



On Mon, Sep 21, 2009 at 2:28 AM, Brian Rectanus <brectanu at gmail.com> wrote:
> I agree we should not do it.  What does it give us, really?  The end
> user that gets these rules can still decrypt it and send it to
> whomever they wish, so the rules are not "protected" other than from
> someone that should not have had access to download them in the first
> place and that should just be protected in another manner (ie do not
> let unauthorized users download them).  The same could be handled (and
> better, I think) through a legal document/NDA.
>
> -B
>
> On Sun, Sep 20, 2009 at 5:02 PM, Scott MacGregor <shadowbq at gmail.com> wrote:
>> My personal opinion :=> "lets please not do this.." (sniffle)
>>
>> ---
>> FYI: There are some decent articles on encrypting xml data..
>>
>> http://dotnetslackers.com/articles/xml/XMLEncryption.aspx
>>
>> ---
>>
>> IDEAS:
>>
>> If the organization as whole wants to encrypt some of the rules so
>> that we can get "pre-release patches" this would be my route:
>>
>> Create an CA and X.509 certificate chain for the root signing of
>> Asymmetric encryption of the rules.
>>
>> Have anyone/company who wants to have access to utilize encrypted
>> rules register for a certificate with the OSIF organization
>> Certificate Authority.
>>
>> Anyone without a certificate can utilize the /rule file/ without
>> error, but the system just skips the encrypted rules.
>>
>> Allow multiple X.509 chains for rule decryption, such that an
>> organization can create its own encryption cert and have "propriety
>> rules" mixed with "org subscription rules" and "org community rules".
>>
>> This is never going to be trivial....
>>
>> Having something like this.. rough sketch of an xml encrypted rule file...
>>
>> The below uses RSA to decode off a keychain named OSIF
>>
>> Reference: http://www.w3.org/TR/xmlenc-core/
>>
>> <?xml version="1.0" standalone="no"?>
>> <rules>
>>  <rule>
>>    <title>OSIF Pre-Release $soft GDI overflow </title>
>>    <vulnerability>
>>      <title>$soft GDI overflow </title>
>>      <para>This space can have a more detailed description.</para>
>>      <para>This space can have additional description.</para>
>>      <releasedate>01-01-2039</releasedate>
>>      <url>https://www.osif.org/rules/reference/[$sid]</url>
>>      <author>OSIF blah....</author>
>>          <status>Limited Public Release</status>
>>    </vulnerability>
>>    <tracking>
>>      <version>1.0</version>
>>      <classtype>bad-unknown</classtype>
>>      <priority>1</priority>
>>          <id>112273645111111</id>
>>    </tracking>
>>    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
>> xmlns="http://www.w3.org/2001/04/xmlenc#">
>>          <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
>>          <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
>>                <KeyName>osif</KeyName>
>>          </KeyInfo>
>>          <CipherData>
>>                <CipherValue>
>>                <![CDATA[
>>                1892379812730981273091287301298731290837019283709
>>                1238712098379182736897126398761293876192876912876
>>                ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
>>                98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
>>                ]]>
>>                </CipherValue>
>>          </CipherData>
>>    </EncryptedData>
>>  </rule>
>> </rules>
>>
>> ~~~~ shadowbq
>> _______________________________________________
>> Oisf-wg-ruleslanguage mailing list
>> Oisf-wg-ruleslanguage at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>>
> _______________________________________________
> Oisf-wg-ruleslanguage mailing list
> Oisf-wg-ruleslanguage at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>


From shadowbq at gmail.com  Mon Sep 21 15:35:30 2009
From: shadowbq at gmail.com (Scott MacGregor)
Date: Mon, 21 Sep 2009 11:35:30 -0400
Subject: [Oisf-wg-ruleslanguage] Supporting languages / Getting things done..
Message-ID: <d6c2aca00909210835l58cf570eq57c61d2365b277fb@mail.gmail.com>

What languages to support as external scripts that can feed
information back to a rule (i.e. a function for a rule to call). Perl,
Ruby, Python? All?

YAML..as markup. (so many bindings)

Scripting languages can be utilized to create the YAML files "on the
fly". (RUBY PERL PYTHON) This is the important thing about the way the
rules are loaded, they need to be able to be loaded via a listening
API services. The API starts, I run script, walla my ruby code loaded
a new rule configuration. This way we can write XML, YAML, ruby,
python, java whatever is in our hearts.. as long as the output of the
scripts creates YAML to be loaded by API.

ON Self Contained Rules:..

Many people feel rule configurations need to be self contained..

This can be solved with reverse distribution scripts.

ON: "seperate the matching and actions pieces"

> This allows users to reuse/inherit matching and lets users build their
> own logic for a sequence of things to do when matches occur.  What do
> you all think?

Good idea

ON : Relational Trees / Stackable Rule files.

Good Idea

Reverse Distribution Example / Fully Expanded Rules.. (For debugging
and redistribution.)

Take an array of signature ids, put it in a text file, and run it
through a (ruby) script that loads all the YAML rules files. It would
output individual distributional rules filling all the variables with
actual values and action modules combined.


Remember :


From jdickey at seven-sigma.com  Wed Sep 23 03:36:08 2009
From: jdickey at seven-sigma.com (Jeff Dickey)
Date: Wed, 23 Sep 2009 11:36:08 +0800
Subject: [Oisf-wg-ruleslanguage] On encrypting rule files (xml example)
In-Reply-To: <d6c2aca00909210723p41c8b6e9ncdbcba0d2b132075@mail.gmail.com>
Message-ID: <C6DFB8A8.1D9D5%jdickey@seven-sigma.com>

I'm late to the party, but I'll put my two rupiah in.... What I seem to be
hearing is what I've believed for some time, that the "traditional" X509 and
Friends toolchain is a heavier burden than we may want to deal with
(technically/financially/other areas?)... Has anybody given any serious
thought to making something like GPG work for signing/authentication?

Jeff

(who also thinks we *so* need Google Wave for these discussions....)


On 21/9/09 22:23 , "Scott MacGregor" <shadowbq at gmail.com> wrote:

> No matter how you do this.. the end user /engineer will be able to
> decrypt the rule off the box. Its not ok to run a "black box" on my
> network, and it will never be ok.. That being said, I would recommend
> digital signing as an authorship verification.. much later in
> development.
> 
> NDA will be required for the certificate issuance in any manner if we
> end up down this route.
> 
> Just an Idea(tm).. If we came up with our own way of distributing
> rules.. requiring certs (X509), a SSL server, common NDA template, a
> common web retrieval API with front end example app (rubyrails app/php
> app/py egg) etc.. open source and document the exchange process so
> that $soft, redhat, my own company could all be rule "sources"
> ..maybe... oinkmaster++ .. NDA with each rule source server etc...
> 
> If OSIF wanted to continue down the encryption in the rule file: = >
> It would take a significant effort to a accomplish rule distributions
> that are signed by each individuals public key for rule issuance. This
> could cause bursting high volume requests that require a lot of
> encryption processing on the OSIF servers on "new rule days/pre-patch
> Mondays".
> 
> Reminder: A significant problem not faced with other applications is
> that IDS are often place out-of-band and are not capable of
> communicating back to a online website for bi-directional
> communications during operation.
> 
> 
> 
> On Mon, Sep 21, 2009 at 2:28 AM, Brian Rectanus <brectanu at gmail.com> wrote:
>> I agree we should not do it.  What does it give us, really?  The end
>> user that gets these rules can still decrypt it and send it to
>> whomever they wish, so the rules are not "protected" other than from
>> someone that should not have had access to download them in the first
>> place and that should just be protected in another manner (ie do not
>> let unauthorized users download them).  The same could be handled (and
>> better, I think) through a legal document/NDA.
>> 
>> -B
>> 
>> On Sun, Sep 20, 2009 at 5:02 PM, Scott MacGregor <shadowbq at gmail.com> wrote:
>>> My personal opinion :=> "lets please not do this.." (sniffle)
>>> 
>>> ---
>>> FYI: There are some decent articles on encrypting xml data..
>>> 
>>> http://dotnetslackers.com/articles/xml/XMLEncryption.aspx
>>> 
>>> ---
>>> 
>>> IDEAS:
>>> 
>>> If the organization as whole wants to encrypt some of the rules so
>>> that we can get "pre-release patches" this would be my route:
>>> 
>>> Create an CA and X.509 certificate chain for the root signing of
>>> Asymmetric encryption of the rules.
>>> 
>>> Have anyone/company who wants to have access to utilize encrypted
>>> rules register for a certificate with the OSIF organization
>>> Certificate Authority.
>>> 
>>> Anyone without a certificate can utilize the /rule file/ without
>>> error, but the system just skips the encrypted rules.
>>> 
>>> Allow multiple X.509 chains for rule decryption, such that an
>>> organization can create its own encryption cert and have "propriety
>>> rules" mixed with "org subscription rules" and "org community rules".
>>> 
>>> This is never going to be trivial....
>>> 
>>> Having something like this.. rough sketch of an xml encrypted rule file...
>>> 
>>> The below uses RSA to decode off a keychain named OSIF
>>> 
>>> Reference: http://www.w3.org/TR/xmlenc-core/
>>> 
>>> <?xml version="1.0" standalone="no"?>
>>> <rules>
>>>  <rule>
>>>    <title>OSIF Pre-Release $soft GDI overflow </title>
>>>    <vulnerability>
>>>      <title>$soft GDI overflow </title>
>>>      <para>This space can have a more detailed description.</para>
>>>      <para>This space can have additional description.</para>
>>>      <releasedate>01-01-2039</releasedate>
>>>      <url>https://www.osif.org/rules/reference/[$sid]</url>
>>>      <author>OSIF blah....</author>
>>>          <status>Limited Public Release</status>
>>>    </vulnerability>
>>>    <tracking>
>>>      <version>1.0</version>
>>>      <classtype>bad-unknown</classtype>
>>>      <priority>1</priority>
>>>          <id>112273645111111</id>
>>>    </tracking>
>>>    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"
>>> xmlns="http://www.w3.org/2001/04/xmlenc#">
>>>          <EncryptionMethod
>>> Algorithm="http://www.w3.org/2001/04/xmlenc#rsa-1_5" />
>>>          <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
>>>                <KeyName>osif</KeyName>
>>>          </KeyInfo>
>>>          <CipherData>
>>>                <CipherValue>
>>>                <![CDATA[
>>>                1892379812730981273091287301298731290837019283709
>>>                1238712098379182736897126398761293876192876912876
>>>                ahsfiouy23978r7ry98ef79237rhi7r9287ry23478rh98472
>>>                98u2h9r7h946rt92783987ghf9478f82h978f2978fg78429f
>>>                ]]>
>>>                </CipherValue>
>>>          </CipherData>
>>>    </EncryptedData>
>>>  </rule>
>>> </rules>
>>> 
>>> ~~~~ shadowbq
>>> _______________________________________________
>>> Oisf-wg-ruleslanguage mailing list
>>> Oisf-wg-ruleslanguage at openinfosecfoundation.org
>>> 
http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguag>>>
e
>>> 
>> _______________________________________________
>> Oisf-wg-ruleslanguage mailing list
>> Oisf-wg-ruleslanguage at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage
>> 
> _______________________________________________
> Oisf-wg-ruleslanguage mailing list
> Oisf-wg-ruleslanguage at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-ruleslanguage

-- 
Jeff Dickey         http://archlever.blogspot.com/
Email:              jdickey at seven-sigma.com
Phone/SMS:          +65 8333 4403
Skype:              jeff_dickey
LinkedIn:           jdickey
Yahoo! IM:          jeff_dickey
MSN IM:             jeff_dickey at hotmail.com (for IM only, please)
ICQ:                8053918
QQ:                 30302349
GnuPG key:
    Fingerprint     D367 FB97 4E59 BEC0 8EBC  D8E3 3BD4 7D4C DFE0 6488
    Valid from      01 July 2009 to 31 December 2009
    Download from   http://tr.im/qqQa