[Discussion] not a db schema

[Andre Ludwig]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

What is being discussed on this list in regards to technology and 
requirements for data rentetion and access just dont mesh well with 
current "standard RDBMS".  I would suggest looking at distributed file 
systems and some of the databases that sit on top of them (hypertable, 
hbase, etc come to mind).

Andre Ludwig

Matt Jonkman wrote:
> Agreed. I don't want to get deep into the schema itself. Just want to
> explore the idea of not being tied to one schema.
>
> A question for the sql guru's though: Is there a large enough difference
> in how you'd structure a scheme for large scale vs small/speedy to
> justify thinking about this?
>
> Matt
>
>
> Martin Holste wrote:
>   
>> I think that schema discussions are extremely premature since we still
>> haven't decided what we're even recording.
>>
>> --Martin
>>
>> On Tue, Oct 28, 2008 at 11:54 AM, Matt Jonkman <jonkman at jonkmans.com
>> <mailto:jonkman at jonkmans.com>> wrote:
>>
>>     I wonder if we ought to build several db schema's for different goals.
>>
>>     I'm not a deep-down sql expert (we'll get one involved when we get
>>     here), but I suspect if your goal is different a different schema will
>>     do best.
>>
>>     For instance, if you intend to keep every alert ever generated for
>>     historical comparison you need one schema to keep it snappy. If you
>>     intend to have a smaller number of alerts but a massive number of
>>     sensors inserting frequently then you may need a different structure.
>>
>>     So perhaps we should build several schemas and let you choose at setup.
>>     Then the engine just reads the version tag and inserts in that form.
>>
>>     Any sql guru's that could speak more to this?
>>
>>     Matt
>>
>>     James McQuaid wrote:
>>     > I'm *still* listening... please continue.
>>     >
>>     > Message: 3
>>     > Frank Knobbe wrote:
>>     >> No, that's a bad idea (at least if you talk about Snort). If you
>>     create
>>     >> new/different message texts, Snort will create a new entry in the
>>     >> signature table (unique to msg+gid+sid+rev). Also, you would not
>>     get the
>>     >> same text with barnyard or in barnyard (and probably flop) based
>>     >> installs since BY only reports the sid (the msg is pulled from the
>>     >> sid-msg.map file).
>>     >
>>     > We are not talking snort. This is totally different.
>>     >
>>     > And we'll definitely not use a db schema with this issue.
>>     >
>>     > Matt
>>     >
>>     >> While you could of course fork barnyard, my concern would be the
>>     bloat
>>     >> of the signature table due to unique msg texts.
>>     >>
>>     >
>>     > No forking here, all new.
>>     >
>>     > Everything from the pattern matcher on up. :)
>>     >
>>     > Matt
>>     >
>>     >
>>     > --
>>     > --------------------------------------------
>>     > Matthew Jonkman
>>     > Emerging Threats
>>     > Phone 765-429-0398
>>     > Fax 312-264-0205
>>     > http://www.emergingthreats.net
>>     >
>>     >
>>     > --------------------------------------------
>>     >
>>     > PGP: http://www.jonkmans.com/mattjonkman.asc
>>     >
>>     >
>>     >
>>     >
>>     >
>>
>>     --
>>     --------------------------------------------
>>     Matthew Jonkman
>>     Emerging Threats
>>     Phone 765-429-0398
>>     Fax 312-264-0205
>>     http://www.emergingthreats.net
>>     --------------------------------------------
>>
>>     PGP: http://www.jonkmans.com/mattjonkman.asc
>>
>>
>>     _______________________________________________
>>     Discussion mailing list
>>     Discussion at openinfosecfoundation.org
>>     <mailto:Discussion at openinfosecfoundation.org>
>>     http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>>
>>
>>     
>
>   


-----BEGIN PGP SIGNATURE-----
Version: PGP Desktop 9.9.0 (Build 397)
Charset: ISO-8859-1

wsBVAwUBSQivBcjAfVnRK9hXAQjszAgAuo00m4GoBrroLxj6zcRjpEyiNP/S831u
TWPwiF5YjEoDMpGDBlVqJiOowJ2LvO2bAJCPvR9dXnQPK7pOVgRy1ODt0cdeAV5a
18ALVe6P9SNCQXpt3Qnay8sSHC7fNcUxvLpyVwlRXm+2OnC/UKEjtGtBlrbXn0eO
Aapm8i4IH53QysFuuWAEFq3tPH6T1XbrnAHEdXrUiC972gqFFy2G8i2UqxN92076
KX0OhdeWS7QPJOTXiBRBftK9ZLvLQ42zXUtW/I7/Cmh2+hMT6/ime+QblCwX3k9q
9+8YOLiVTG1/KRtXSJakZ/2f4h7dK55j7az4UoD3hdzbgzLUNUagZQ==
=6IAU
-----END PGP SIGNATURE-----