From jonkman at jonkmans.com Sun Mar 1 10:11:47 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:11:47 -0500 Subject: [Discussion] a few ideas In-Reply-To: References: Message-ID: <49AAA5B3.2040806@jonkmans.com> Great ideas Kevin!!! If you don't mind I'd like to split these up into separate threads. Matt Kevin Ross wrote: > Hey here are a few things I would like to see. Possibly already > discussed or not as useful as I think. > > 1) First Ossim (http://www.ossim.net/) has an interesting use of the > reliability of the sig, the priority of the host and some other things > to assign a risk to the attack. Using a similar system individual > signatures can be given a reliability which could mean sure fire attacks > are flagged immediately while unreliable signatures are not flagged > immediately until other factors are met. For instance under ossim you > can basically say (in an xml directive) if there are these snort sids it > is a reliability of 3 and if these snort sigs appear +2, if it > persistant (for a set time) +1 and if a web page error message appears > +1 and so on. Using such a system could mean false positives can > automatically lowered while making more reliable attacks against > priority resources and the events related to that attack available to > the analyst (being able to define the priority of an asset such as a > server farm in comparison so the secretary's desktop would be useful). > Also things like if the attack was blocked by IPS or even a firewall if > the logs are available that the attack was mitigated the risk level can > be reduced. > > 2) For active response the ability to specify agentless blocking > devices. i.e set up a pix/asa/iptables/pf firewall and say ssh into the > device from the master sensor, go into the correct configuration mode > and then enter in the appropriate command i.e ssh pix at firewall; enters > the password; en; enters the password; conf t; shun ip or acl. Then > after a set time remove the block. Also if it is an IPS the ability to > say act inline but also have active response such as dropping the attack > and blocking the ip or an attack is detected once drop it, if the same > host attacks again block it completely, > > 3) an installer on a cutdown linux/bsd system perhaps with a simple > installer, also perhaps configuration by a web interface. That way a > non-unix person can install the system selecting the relevant options, > then use the web interface to set up the distributed system. This would > attract more users by helping to simplify a basic setup. Possibly even > installers consisting of different tools, i.e an installer for > master/slave sensors for normal IDS/IPS and correlation and another say > for a honeypot with nepenthes or honeyd and in the install you can point > it to the master sensor. That way dedicated parts of the distributed > system can be installed easily by inexperienced users (which everyone > will be who comes to use this system at first till they learn it). Also > using this methods means different types of systems can be added to the > distributed IDS/IPS as need dictates such as some new type of detection > tool to some future type of attack. > > 4) Perhaps the ability to either autofind or being able to enter in the > network topology it can determine the source of the attack within the > network kind of like csmars does (demos here > http://www.demolabs.co.uk/ciscoportal.htm). Also gathering information > such as hostname/netbios name, mac-address etc using tools like nbtscan > on the detection of a local attack (to avoid scanning outside the > network which is a bit scetchy). So if an attack is detected from an > inside host (by specifying rfc 1918 addresses) then execute information > gathering tools to provide more information to the analyst about the > source or target of the attack. That was it becomes easier to determine > if it is an fp. i.e if there is a buffer overflow for a windows system > but the target determined by a tool such as xprobe, nmap or whatever is > some other OS and that information is available immediately upon opening > the even then the analyst has a better understanding of the attack risk > and likely result. Also such a system could be intergrated into some > sort of risk system, such as a netbios attack against a linux system > would lower the risk rating of the attack. > > 5) Perhaps the most ambitious of them all. If an attack is seen through > various methods, say a new worm. If it is unknown by the system and > confirmed by an analyst a signature can be created by the sensor, > perhaps with help from the analyst specifiying a few options like what > to match upon and distubuted to other systems around the world that > choose to accept such updates. Perhaps submitted and checked first by > some central body to avoid someone submitting fake sigs to the > distributed system, then it can be automatically downloaded by sensors > which allow such updates. During a new fast spreading worm this could > mean sensors can be updated with this information quickly with little > intervention from the "clients" in the distributed system such as homes > and businesses. > > I hope some of these ideas sound interesting. > > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:15:23 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:15:23 -0500 Subject: [Discussion] OSSIM and Sig Reliability In-Reply-To: References: Message-ID: <49AAA68B.5070807@jonkmans.com> First of Kevin's ideas: Ossim (http://www.ossim.net/) has an interesting use of the reliability of the sig, the priority of the host and some other things to assign a risk to the attack. Using a similar system individual signatures can be given a reliability which could mean sure fire attacks are flagged immediately while unreliable signatures are not flagged immediately until other factors are met. For instance under ossim you can basically say (in an xml directive) if there are these snort sids it is a reliability of 3 and if these snort sigs appear +2, if it persistent (for a set time) +1 and if a web page error message appears +1 and so on. Using such a system could mean false positives can automatically lowered while making more reliable attacks against priority resources and the events related to that attack available to the analyst (being able to define the priority of an asset such as a server farm in comparison so the secretary's desktop would be useful). Also things like if the attack was blocked by IPS or even a firewall if the logs are available that the attack was mitigated the risk level can be reduced. -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:15:50 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:15:50 -0500 Subject: [Discussion] Distributed Blocking In-Reply-To: References: Message-ID: <49AAA6A6.8010205@jonkmans.com> Kevin Ross wrote: For active response the ability to specify agentless blocking devices. i.e set up a pix/asa/iptables/pf firewall and say ssh into the device from the master sensor, go into the correct configuration mode and then enter in the appropriate command i.e ssh pix at firewall; enters the password; en; enters the password; conf t; shun ip or acl. Then after a set time remove the block. Also if it is an IPS the ability to say act inline but also have active response such as dropping the attack and blocking the ip or an attack is detected once drop it, if the same host attacks again block it completely, -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:16:20 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:16:20 -0500 Subject: [Discussion] Installers In-Reply-To: References: Message-ID: <49AAA6C4.2070901@jonkmans.com> Kevin Ross wrote: an installer on a cutdown linux/bsd system perhaps with a simple installer, also perhaps configuration by a web interface. That way a non-unix person can install the system selecting the relevant options, then use the web interface to set up the distributed system. This would attract more users by helping to simplify a basic setup. Possibly even installers consisting of different tools, i.e an installer for master/slave sensors for normal IDS/IPS and correlation and another say for a honeypot with nepenthes or honeyd and in the install you can point it to the master sensor. That way dedicated parts of the distributed system can be installed easily by inexperienced users (which everyone will be who comes to use this system at first till they learn it). Also using this methods means different types of systems can be added to the distributed IDS/IPS as need dictates such as some new type of detection tool to some future type of attack. -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:17:02 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:17:02 -0500 Subject: [Discussion] Automated Info Gathering In-Reply-To: References: Message-ID: <49AAA6EE.60705@jonkmans.com> Kevin Ross wrote: Perhaps the ability to either autofind or being able to enter in the network topology it can determine the source of the attack within the network kind of like csmars does (demos here http://www.demolabs.co.uk/ciscoportal.htm). Also gathering information such as hostname/netbios name, mac-address etc using tools like nbtscan on the detection of a local attack (to avoid scanning outside the network which is a bit scetchy). So if an attack is detected from an inside host (by specifying rfc 1918 addresses) then execute information gathering tools to provide more information to the analyst about the source or target of the attack. That was it becomes easier to determine if it is an fp. i.e if there is a buffer overflow for a windows system but the target determined by a tool such as xprobe, nmap or whatever is some other OS and that information is available immediately upon opening the even then the analyst has a better understanding of the attack risk and likely result. Also such a system could be intergrated into some sort of risk system, such as a netbios attack against a linux system would lower the risk rating of the attack.n/listinfo/discussion -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:17:44 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:17:44 -0500 Subject: [Discussion] Auto-Sig Creation In-Reply-To: References: Message-ID: <49AAA718.1030205@jonkmans.com> Kevin Ross wrote: Perhaps the most ambitious of them all. If an attack is seen through various methods, say a new worm. If it is unknown by the system and confirmed by an analyst a signature can be created by the sensor, perhaps with help from the analyst specifiying a few options like what to match upon and distubuted to other systems around the world that choose to accept such updates. Perhaps submitted and checked first by some central body to avoid someone submitting fake sigs to the distributed system, then it can be automatically downloaded by sensors which allow such updates. During a new fast spreading worm this could mean sensors can be updated with this information quickly with little intervention from the "clients" in the distributed system such as homes and businesses. -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:24:53 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:24:53 -0500 Subject: [Discussion] Auto-Sig Creation In-Reply-To: <49AAA718.1030205@jonkmans.com> References: <49AAA718.1030205@jonkmans.com> Message-ID: <49AAA8C5.8040303@jonkmans.com> > Kevin Ross wrote: > > Perhaps the most ambitious of them all. If an attack is seen through > various methods, say a new worm. If it is unknown by the system and > confirmed by an analyst a signature can be created by the sensor, > perhaps with help from the analyst specifiying a few options like what > to match upon and distubuted to other systems around the world that > choose to accept such updates. Perhaps submitted and checked first by > some central body to avoid someone submitting fake sigs to the > distributed system, then it can be automatically downloaded by sensors > which allow such updates. During a new fast spreading worm this could > mean sensors can be updated with this information quickly with little > intervention from the "clients" in the distributed system such as homes > and businesses. A few thoughts of mine: 1. We're way into post event analysis, not engine stuff. But still something to discuss. (Our core charter is to build a new engine, but we aren't LIMITED to that necessarily, other than by resources) 2. I've never been a huge fan of automated sig creation tools. They are useful in finding patterns that a human can then do something with, but the human still has to fully understand the protocol. I have tried using the auto creation tools on malware CnC channels though which may not be the fairest test. Most I had to try the tools on were encrypted without anything in the way of headers, so tough to sig regardless. They may fare better on a netbios overflow or something, I don't know. But I like the idea of "if we see something strange lets have a way to see where else that's happening" in a quick way the average ids admin could employ... Matt -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:28:54 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:28:54 -0500 Subject: [Discussion] Automated Info Gathering In-Reply-To: <49AAA6EE.60705@jonkmans.com> References: <49AAA6EE.60705@jonkmans.com> Message-ID: <49AAA9B6.3070101@jonkmans.com> We're definitely into post-engine event processing here. But it may make sense for the engine to have an agent or thread that could answer questions for the core event manager? For instance an event is generated by the engine, does to the DB, the event manager decides it does not know what it needs to know about the source internal ip. The event manager then puts in the db a request for a netbios scan and OS ID for that agent (assuming that agent will be the closest to the target...). The agent checks the db periodically via the engine's db connection and executes the request and inserts the data to the db. It's not a true engine function, but it makes the sensor smarter and more versatile and reduces the number of other tools required for deployment... Matt Matt Jonkman wrote: > Kevin Ross wrote: > > Perhaps the ability to either autofind or being able to enter in the > network topology it can determine the source of the attack within the > network kind of like csmars does (demos here > http://www.demolabs.co.uk/ciscoportal.htm). Also gathering information > such as hostname/netbios name, mac-address etc using tools like nbtscan > on the detection of a local attack (to avoid scanning outside the > network which is a bit scetchy). So if an attack is detected from an > inside host (by specifying rfc 1918 addresses) then execute information > gathering tools to provide more information to the analyst about the > source or target of the attack. That was it becomes easier to determine > if it is an fp. i.e if there is a buffer overflow for a windows system > but the target determined by a tool such as xprobe, nmap or whatever is > some other OS and that information is available immediately upon opening > the even then the analyst has a better understanding of the attack risk > and likely result. Also such a system could be intergrated into some > sort of risk system, such as a netbios attack against a linux system > would lower the risk rating of the attack.n/listinfo/discussion > -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:35:34 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:35:34 -0500 Subject: [Discussion] Installers In-Reply-To: <49AAA6C4.2070901@jonkmans.com> References: <49AAA6C4.2070901@jonkmans.com> Message-ID: <49AAAB46.8080709@jonkmans.com> Kevin Ross wrote: > > an installer on a cutdown linux/bsd system perhaps with a simple > installer, also perhaps configuration by a web interface. That way a > non-unix person can install the system selecting the relevant options, > then use the web interface to set up the distributed system. This would > attract more users by helping to simplify a basic setup. Possibly even > installers consisting of different tools, i.e an installer for > master/slave sensors for normal IDS/IPS and correlation and another say > for a honeypot with nepenthes or honeyd and in the install you can point > it to the master sensor. That way dedicated parts of the distributed > system can be installed easily by inexperienced users (which everyone > will be who comes to use this system at first till they learn it). Also > using this methods means different types of systems can be added to the > distributed IDS/IPS as need dictates such as some new type of detection > tool to some future type of attack. Great idea! I've thrown out I think something similar privately to have a web interface. I think the largest hurdle for new folks getting into snort is the initial build and config. It's a HUGE learning curve, and there isn't much to help you because EVERY net is very different. The only real good way to get into snort is a class or someone that already knows it guiding you through. There's not much of an avenue for a quick self-learning process. A web interface would be great, but we have to make sure it just writes the text config, or inserts into the db the config created. (BTW: I'm lobbying for a config that can be kept in the DB or text file, would make remote sensors FAR easier to manage) The web interface could ask a few basic questions (what's your internal ip space, is my sensing interface inside or outside, what rulesets do you want to load, do you care about P2P traffic, etc) and guide some basic decisions for the user. The extras like honeyd and all is very interesting. I'd like to see a nepenthes as well integrated somehow. Maybe an intermediary for nepenthes/honeyd to talk to the engine. If there's an exploit against a honeypot that info goes directly to the engine and immediately puts that attacker's IP into the hostile area in it's IP Reputation? Maybe the binary captures by the sandnet goes right to the analyst for analysis, maybe an auto-sig for the binary in transit other places gets inserted? matt > > > -------------------------------------------- > Matthew Jonkman > Emerging Threats > Phone 765-429-0398 > Fax 312-264-0205 > http://www.emergingthreats.net > -------------------------------------------- > > PGP: http://www.jonkmans.com/mattjonkman.asc > > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:40:32 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:40:32 -0500 Subject: [Discussion] Distributed Blocking In-Reply-To: <49AAA6A6.8010205@jonkmans.com> References: <49AAA6A6.8010205@jonkmans.com> Message-ID: <49AAAC70.8030404@jonkmans.com> You've described Snortsam here (www.snortsam.net). Long time favorite tool of mine written by Frank Knobbe et al. You use hubs that can talk to firewalls pretty much as you described. You define sids that block, src or dst, and for how long. Then snortsam handles the inserting and removing. Great tool, highly recommend using it. We are definitely going to have to have something similar in the new engine because we're looking at IP reputation. We also want to be able to compare and block based on massive lists of IPs and reputation. But we also need to be able to share block info with other perimeter devices. I cant begin to tell you how effective it is to block an attacker on your entire enterprise when they start to fool around at one point. I've seen that drop the number of events of serious threat in an enterprise more than 40%. That's a huge load off an analyst knowing that once an IP demonstrated overt hostility they were blocked everywhere. Makes things go very nice and quiet. So yes, this definitely is something we need to make happen. Seems it'd be a feature of a core central hub to manage the blocks, then maybe relay block commands to remote sensors who may be the acutal blocker, or closest to the external blocking device to communicate and make that block. I'd definitely like to see this communication happen through the existing sensor to core db connection though. Much easier for outside tools to also then to read and manipulate that block stream as well. Matt Matt Jonkman wrote: > Kevin Ross wrote: > > For active response the ability to specify agentless blocking devices. > i.e set up a pix/asa/iptables/pf firewall and say ssh into the device > from the master sensor, go into the correct configuration mode and then > enter in the appropriate command i.e ssh pix at firewall; enters the > password; en; enters the password; conf t; shun ip or acl. Then after a > set time remove the block. Also if it is an IPS the ability to say act > inline but also have active response such as dropping the attack and > blocking the ip or an attack is detected once drop it, if the same host > attacks again block it completely, > > > -------------------------------------------- > Matthew Jonkman > Emerging Threats > Phone 765-429-0398 > Fax 312-264-0205 > http://www.emergingthreats.net > -------------------------------------------- > > PGP: http://www.jonkmans.com/mattjonkman.asc > > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From jonkman at jonkmans.com Sun Mar 1 10:43:42 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Sun, 01 Mar 2009 10:43:42 -0500 Subject: [Discussion] OSSIM and Sig Reliability In-Reply-To: <49AAA68B.5070807@jonkmans.com> References: <49AAA68B.5070807@jonkmans.com> Message-ID: <49AAAD2E.9060006@jonkmans.com> I definitely like this too. But it's way into post processing. Out of our immediate scope. But what information could the engine provide to help the event manager make these decisions? Surely there's something it could help with? Maybe on the behavioral note the engine could start full captures or something when suspicious things happen so the analyst would immediately have more context? Matt > First of Kevin's ideas: > > Ossim (http://www.ossim.net/) has an interesting use of the reliability > of the sig, the priority of the host and some other things to assign a > risk to the attack. Using a similar system individual signatures can be > given a reliability which could mean sure fire attacks are flagged > immediately while unreliable signatures are not flagged immediately > until other factors are met. For instance under ossim you can basically > say (in an xml directive) if there are these snort sids it is a > reliability of 3 and if these snort sigs appear +2, if it persistent > (for a set time) +1 and if a web page error message appears +1 and so > on. Using such a system could mean false positives can automatically > lowered while making more reliable attacks against priority resources > and the events related to that attack available to the analyst (being > able to define the priority of an asset such as a server farm in > comparison so the secretary's desktop would be useful). Also things like > if the attack was blocked by IPS or even a firewall if the logs are > available that the attack was mitigated the risk level can be reduced. > > -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From mcholste at gmail.com Sun Mar 1 13:32:54 2009 From: mcholste at gmail.com (Martin Holste) Date: Sun, 1 Mar 2009 12:32:54 -0600 Subject: [Discussion] OSSIM and Sig Reliability In-Reply-To: <49AAAD2E.9060006@jonkmans.com> References: <49AAA68B.5070807@jonkmans.com> <49AAAD2E.9060006@jonkmans.com> Message-ID: I disagree that this is out-of-scope. I think that this falls well within the detection engine's purview, as evidenced by the fact that we've all been using flowbits in Snort for a long time now. It's the detection engine's job because we need to be able to write rules which have some correlative properties, and as far as I know, we're writing the rules for the detection engine. The extra properties that frameworks like OSSIM have allowed for are intellectual property to be shared just like the Emerging Threats detection rules. Sharing with the community that "pattern X is malicious" is helpful, but it's even more helpful to share with the community that "pattern X in context Y" is malicious. In order to accomplish this, I think we want our engine to provide contextual traffic immediately surrounding the incident. I find that the Sguil-style full capture is by far the most effective, but I recognize that it is impossible to do that in lots of environments for various technical and legal reasons. Something like an improved version of Snort's "session" and 'tag" keywords for rules would probably go a long way towards this goal. As an example, one of my favorite command line switches for the venerable grep program is "-C " which means return plus and minus n lines of the file from the match. This establishes the context of the match in the document you are searching so you have some idea as to what it meant in that particular part of the doc. This is exactly the same requirement we have when grepping network traffic, but most current tools provide no way of seeing the traffic immediately preceding the sought event. This is obviously because you have to record everything since you don't know ahead of time what you'll be looking for. However, if you only save a few seconds or minutes prior to the current time, this is not as daunting as it sounds. As I've previously mentioned, Bro and Timemachine do this already, and they do it very effectively. Timemachine writes traffic to RAM/hard disk and Bro can query it to get some context. When Bro queries Timemachine for traffic still in RAM, it gets the results in milleseconds. The queries are built right into the Bro signatures. Our engine could have rules that specify prior events right in the rule, which opens to door to a much more flexible system and rules that can specify a dynamic event hierarchy. For instance: Look for event X If found, look for event Y in the traffic up to five seconds before event X. If found, begin looking for event Z for the hosts referred to in event X for the next five minutes. A real-life example would be something like this: Look for the possible Trojan check-in URI pattern "x=digits&y=digits" If found, check if the source host downloaded any EXE or PDF files in the last five seconds. If found, check to see if the source host makes any POSTs or requests any EXE's for the next ten seconds. Bro already allows for this, and if we're going to be creating a "next-gen" detection engine, we need to at least match this capability. So from a signature structure perspective, we're looking at a basic content match sig which contains an array of signature ID's that should match prior traffic and an array of signature ID's that should match subsequent traffic. So, it's a lot like Snort's flowbits except you can specify boolean operators on arrays of flowbits, instead of just one flowbit. Depending on how we make the signature syntax, maybe you could even include the other sigs as partial sigs inlined into the main sig so that everything is contained in just one signature. Something like: content:"this"; content: "that"; prior: 5 minutes; content "the other thing"; subsequent: 30 seconds; Or as references to other sigs: content:"this"; sig_sid:1001; prior: 5 minutes; sig_sid:1002; subsequent: 30 seconds; If the match for "that" is extremely common, then you wouldn't want to search for it all the time, you'd only want to search for it when you know something interesting has occurred, retroactively. Additionally, instead of white and black matching, maybe these auto-adjust the signature's fidelity rating: content:this"; (sig_sid:1001; prior: 5 minutes; fidelity:+5) (sig_sid:1002; subsequent: 30 seconds; fidelity: +10) I think that it's important to put the fidelity modifiers right into the signature instead of forcing everyone to figure that out on their own. If you leave that information out of the signature, then it's much harder for the community to contribute improvements to those properties. Naturally, there will be org-specific modifications necessary like any rule, but if the signature creator has the chance to say up front that the signature is a better match in a given context, I think that's a major leap forward. --Martin On Sun, Mar 1, 2009 at 9:43 AM, Matt Jonkman wrote: > I definitely like this too. But it's way into post processing. Out of > our immediate scope. > > But what information could the engine provide to help the event manager > make these decisions? Surely there's something it could help with? > > Maybe on the behavioral note the engine could start full captures or > something when suspicious things happen so the analyst would immediately > have more context? > > Matt > > > > First of Kevin's ideas: > > > > Ossim (http://www.ossim.net/) has an interesting use of the reliability > > of the sig, the priority of the host and some other things to assign a > > risk to the attack. Using a similar system individual signatures can be > > given a reliability which could mean sure fire attacks are flagged > > immediately while unreliable signatures are not flagged immediately > > until other factors are met. For instance under ossim you can basically > > say (in an xml directive) if there are these snort sids it is a > > reliability of 3 and if these snort sigs appear +2, if it persistent > > (for a set time) +1 and if a web page error message appears +1 and so > > on. Using such a system could mean false positives can automatically > > lowered while making more reliable attacks against priority resources > > and the events related to that attack available to the analyst (being > > able to define the priority of an asset such as a server farm in > > comparison so the secretary's desktop would be useful). Also things like > > if the attack was blocked by IPS or even a firewall if the logs are > > available that the attack was mitigated the risk level can be reduced. > > > > > > -- > -------------------------------------------- > Matthew Jonkman > Emerging Threats > Phone 765-429-0398 > Fax 312-264-0205 > http://www.emergingthreats.net > -------------------------------------------- > > PGP: http://www.jonkmans.com/mattjonkman.asc > > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090301/b962c8a6/attachment.html From kevross33 at googlemail.com Tue Mar 3 17:24:55 2009 From: kevross33 at googlemail.com (Kevin Ross) Date: Tue, 3 Mar 2009 22:24:55 +0000 Subject: [Discussion] a few ideas Message-ID: Hey I have a few more ideas: 1) rather than have the engine with all the signatures required for the environment for the everything it is watching why not be able to have directed rulesets? I.e say only process these rules for these networks or machines but don't do it for these. For instance if there was a single sensor watching traffic to and from a network segment with a few Linux servers and windows clients, rather than enabling all the rules necessary I can say watch the necessary rules for windows while not applying the Linux rules and vice versa. i.e have netbios rules for 192.168.1.0/24 network but not for hosts 192.168.1.40-45, that way uncessary rules are not applied on traffic going hosts/networks which don't need them. It may not be a good example for netbios due to specific ports that are used but in an IIS/Apache web farm for example it could be useful. I think the best way to decide on this would be whitelist/blaclist approach so you can say apply these rules/rulesets to this entire network except for these hosts or say do not apply the ruleset to this network except for these hosts. That was a sensor watching a mixed network segment can apply the rules more accurately to the traffic. 2) suggestive rule tuning. i.e the sensor does not see any netbios traffic within a learning period, it can then say "no netbios traffic has been seen, do you wish to disable this ruleset" or something similiar like do you have windows machines?. Likewise if the sensor sees something that a ruleset/rule is not enabled for it can suggest it is enabled. This suggestive tuning could make it easier for new users to tune the system. This would lend itself well to a webgui/gui approach where an area could have suggested tuning options. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090303/87068074/attachment.html From edward.fjellskal at redpill-linpro.com Wed Mar 4 10:44:01 2009 From: edward.fjellskal at redpill-linpro.com (=?ISO-8859-1?Q?Edward_Bjarte_Fjellsk=E5l?=) Date: Wed, 04 Mar 2009 16:44:01 +0100 Subject: [Discussion] a few ideas In-Reply-To: References: Message-ID: <49AEA1C1.2040800@redpill-linpro.com> Kevin Ross wrote: > Hey I have a few more ideas: > > 1) rather than have the engine with all the signatures required for > the environment for the everything it is watching why not be able to > have directed rulesets? I.e say only process these rules for these > networks or machines but don't do it for these. > > For instance if there was a single sensor watching traffic to and from > a network segment with a few Linux servers and windows clients, rather > than enabling all the rules necessary I can say watch the necessary > rules for windows while not applying the Linux rules and vice versa. > i.e have netbios rules for 192.168.1.0/24 > network but not for hosts 192.168.1.40-45, that way uncessary rules > are not applied on traffic going hosts/networks which don't need them. > It may not be a good example for netbios due to specific ports that > are used but in an IIS/Apache web farm for example it could be useful. > > I think the best way to decide on this would be whitelist/blaclist > approach so you can say apply these rules/rulesets to this entire > network except for these hosts or say do not apply the ruleset to this > network except for these hosts. That was a sensor watching a mixed > network segment can apply the rules more accurately to the traffic. > > 2) suggestive rule tuning. i.e the sensor does not see any netbios > traffic within a learning period, it can then say "no netbios traffic > has been seen, do you wish to disable this ruleset" or something > similiar like do you have windows machines?. Likewise if the sensor > sees something that a ruleset/rule is not enabled for it can suggest > it is enabled. This suggestive tuning could make it easier for new > users to tune the system. This would lend itself well to a webgui/gui > approach where an area could have suggested tuning options. > I have been thinking of this to, but I always end up with the same conclusion... Have all rules enabled for all hosts :) Then doing the correlation afterwards... ie. I dont have control over our lab or our wireless. So I dont know when there is a Windows machine or a unix machine there. I like the way sguil (http://sguil.sourceforge.net/) handles this. You can set up "auto categorization" on events, and if you know the event is a false positive, you make a rule for it. I made a "wish" of having it done easy from the GUI, like right click and things would be auto-populated, and you can just click "OK" if the data fits your needs. (http://nsmwiki.org/Sguil_Feature_Wish_List) I want to take this one step further, and try to do this automatic... Im working on a little perl daemon, to sniff the traffic, and detect OS and Services running on my network. Hopefully, in the future, this could be used to automatically help in the "auto categorization" of events... in sguil or other IDS gui... ( http://www.gamelinux.org/?p=43 and http://gamelinux.github.com/prads/ ) Basically, I want to have all the events for "historic" reasons... and I want to filter out what the analysts sees. If there was an event, that did pass, it would be in the database, and other anomaly detection running through the database might pick it up etc. Im lacking sleep at the moment, and my head is fried... Hope my email makes sens. E From jlewis at packetnexus.com Wed Mar 4 12:18:41 2009 From: jlewis at packetnexus.com (Jason Lewis) Date: Wed, 04 Mar 2009 12:18:41 -0500 Subject: [Discussion] CATCH in DC Message-ID: <49AEB7F1.5080905@packetnexus.com> I should have asked this yesterday, but didn't dawn on me until I was reading email this morning. Did anyone make it to the DHS CATCH conference in DC? jas From dagon at cc.gatech.edu Wed Mar 4 13:38:11 2009 From: dagon at cc.gatech.edu (David Dagon) Date: Wed, 4 Mar 2009 13:38:11 -0500 Subject: [Discussion] CATCH in DC In-Reply-To: <49AEB7F1.5080905@packetnexus.com> References: <49AEB7F1.5080905@packetnexus.com> Message-ID: <20090304183811.GA63396@fritz.cc.gt.atl.ga.us> On Wed, Mar 04, 2009 at 12:18:41PM -0500, Jason Lewis wrote: > I should have asked this yesterday, but didn't dawn on me until I was > reading email this morning. Did anyone make it to the DHS CATCH > conference in DC? http://www.cyber.st.dhs.gov/docs/CATCHv1H.pdf Yes; I attended and have electronic copies of presentations, if you wanted to see anything. -- David Dagon /"\ "When cryptography dagon at cc.gatech.edu \ / ASCII RIBBON CAMPAIGN is outlawed, bayl Ph.D. Student X AGAINST HTML MAIL bhgynjf jvyy unir Georgia Inst. of Tech. / \ cevinpl." From frank at knobbe.us Wed Mar 4 19:47:31 2009 From: frank at knobbe.us (Frank Knobbe) Date: Wed, 04 Mar 2009 18:47:31 -0600 Subject: [Discussion] a few ideas In-Reply-To: References: Message-ID: <1236214051.8383.66.camel@localhost> On Tue, 2009-03-03 at 22:24 +0000, Kevin Ross wrote: > Hey I have a few more ideas: > > 1) rather than have the engine with all the signatures required for > the environment for the everything it is watching why not be able to > have directed rulesets? I.e say only process these rules for these > networks or machines but don't do it for these. Uhm... and the reason you can't do that with a current Snort config is what exactly? I have several sensors where HOME_NET is redefined and the appropriate rule sets loaded again, multiple times. Works pretty good for me. (I know, most folks define HOME_NET and EXTERNAL_NET *once* in their snort.conf, and are missing out on some decent snort.conf tuning) > 2) suggestive rule tuning. i.e the sensor does not see any netbios > traffic within a learning period, it can then say "no netbios traffic > has been seen, do you wish to disable this ruleset" or something > similiar like do you have windows machines?. And what if NetBIOS attacks are being performed *after* the tuning period is over and the IDS blinded itself? That's my biggest beef with "self-tuning" or "learning periods". I'd rather describe the network to Snort myself instead of letting software make that decisions and get it wrong. Cheers, Frank -- It is said that the Internet is a public utility. As such, it is best compared to a sewer. A big, fat pipe with a bunch of crap sloshing against your ports. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part Url : http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090304/9d089343/attachment.bin From lists at inliniac.net Thu Mar 5 02:26:41 2009 From: lists at inliniac.net (Victor Julien) Date: Thu, 05 Mar 2009 08:26:41 +0100 Subject: [Discussion] conditional signature loading Message-ID: <49AF7EB1.4040504@inliniac.net> I was wondering about the possible usefulness of an extension of the signature language. Right now in Snort one can limit the signatures loaded by the selection of which signature files are included in the config, by have oinkmaster disable specific signatures and of course by manually editting the sig files to comment out certain sigs. The idea I'm thinking about is having a way to enable or disable blocks of signatures using condition statements. For example (don't mind the syntax for now): #ifndef IGNORE_EDONKEY #endif The above would mean that unless the is a configuration directive defined called 'IGNORE_EDONKEY', the edonkey sigs are loaded. I can see use for this for e.g. bad performaning sigs, likely fp sigs, ignoring certain protocols, applications, http server brands, etc. It would be in addition to oinkmaster, not to replace it per se. The idea would that creators of rulesets would predefine most of these statements. Thoughts anyone? Cheers, Victor -- --------------------------------------------- Victor Julien http://www.inliniac.net/ PGP: http://www.inliniac.net/victorjulien.asc --------------------------------------------- From lists at inliniac.net Thu Mar 5 02:47:54 2009 From: lists at inliniac.net (Victor Julien) Date: Thu, 05 Mar 2009 08:47:54 +0100 Subject: [Discussion] a few ideas In-Reply-To: <49AEA1C1.2040800@redpill-linpro.com> References: <49AEA1C1.2040800@redpill-linpro.com> Message-ID: <49AF83AA.1070402@inliniac.net> Edward Bjarte Fjellsk?l wrote: > I want to take this one step further, and try to do this automatic... Im > working on a little perl daemon, to sniff the traffic, and detect OS and > Services running on my network. Hopefully, in the future, this could be > used to > automatically help in the "auto categorization" of events... in sguil or > other IDS gui... > ( http://www.gamelinux.org/?p=43 and http://gamelinux.github.com/prads/ ) I'm still a bit torn on whether we should have the engine itself do the detection of this information or if we should enable the engine to be fed this info by external programs like your prads. Thoughts anyone? Regards, Victor -- --------------------------------------------- Victor Julien http://www.inliniac.net/ PGP: http://www.inliniac.net/victorjulien.asc --------------------------------------------- From mcholste at gmail.com Thu Mar 5 09:58:45 2009 From: mcholste at gmail.com (Martin Holste) Date: Thu, 5 Mar 2009 08:58:45 -0600 Subject: [Discussion] a few ideas In-Reply-To: <49AF83AA.1070402@inliniac.net> References: <49AEA1C1.2040800@redpill-linpro.com> <49AF83AA.1070402@inliniac.net> Message-ID: All I want from the engine is protocol decoding, grepping, and a big fat two-way pipe to everything else. I think those other tasks are still in scope for the OISF group, I just want to keep them as far away from libpcap as possible. --Martin On Thu, Mar 5, 2009 at 1:47 AM, Victor Julien wrote: > Edward Bjarte Fjellsk?l wrote: > > I want to take this one step further, and try to do this automatic... Im > > working on a little perl daemon, to sniff the traffic, and detect OS and > > Services running on my network. Hopefully, in the future, this could be > > used to > > automatically help in the "auto categorization" of events... in sguil or > > other IDS gui... > > ( http://www.gamelinux.org/?p=43 and > http://gamelinux.github.com/prads/ ) > > I'm still a bit torn on whether we should have the engine itself do the > detection of this information or if we should enable the engine to be > fed this info by external programs like your prads. > > Thoughts anyone? > > Regards, > Victor > > -- > --------------------------------------------- > Victor Julien > http://www.inliniac.net/ > PGP: http://www.inliniac.net/victorjulien.asc > --------------------------------------------- > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090305/7e3afcf0/attachment.html From edward.fjellskal at redpill-linpro.com Fri Mar 6 14:19:07 2009 From: edward.fjellskal at redpill-linpro.com (=?ISO-8859-1?Q?Edward_Bjarte_Fjellsk=E5l?=) Date: Fri, 06 Mar 2009 20:19:07 +0100 Subject: [Discussion] a few ideas In-Reply-To: <49AF83AA.1070402@inliniac.net> References: <49AEA1C1.2040800@redpill-linpro.com> <49AF83AA.1070402@inliniac.net> Message-ID: <49B1772B.90006@redpill-linpro.com> Victor Julien wrote: > Edward Bjarte Fjellsk?l wrote: > >> I want to take this one step further, and try to do this automatic... Im >> working on a little perl daemon, to sniff the traffic, and detect OS and >> Services running on my network. Hopefully, in the future, this could be >> used to >> automatically help in the "auto categorization" of events... in sguil or >> other IDS gui... >> ( http://www.gamelinux.org/?p=43 and http://gamelinux.github.com/prads/ ) >> > > I'm still a bit torn on whether we should have the engine itself do the > detection of this information or if we should enable the engine to be > fed this info by external programs like your prads. > > Thoughts anyone? > > Regards, > Victor My thoughts are to keep them outside the engine, if it sucks up too much juice. Or the option to turn it off/on in the engine... and be able to have input from another sensor, or an external program. Im also in favor on the thought, that an external program would be better. The external program could be updated separately with fingerprints/signatures/rules without dependency on the Engine. The external program could also be used for other stuff.... Larger community :) But that said, exact values for ttl etc. the Engine should be using for a host, would best be predicted from the same data that the Engine sees. If the Engine depends on correct ttl (etc.) values........ So an external program might need to be placed correct, listening on the same TAP etc. e From jmenerick at netsuite.com Fri Mar 6 18:28:41 2009 From: jmenerick at netsuite.com (Menerick, John) Date: Fri, 6 Mar 2009 15:28:41 -0800 Subject: [Discussion] a few ideas In-Reply-To: <49B1772B.90006@redpill-linpro.com> References: <49AEA1C1.2040800@redpill-linpro.com> <49AF83AA.1070402@inliniac.net>,<49B1772B.90006@redpill-linpro.com> Message-ID: <10CD0A2672F6814A988052F37D8D67554E1F53E1@corpmail2007.corp.netledger.com> My preference would be to keep it outside of the engine. Or at least in a modular fashion built into the engine. Not only for performance reasons but for KISS pragmatic design. John Menerick ________________________________________ From: discussion-bounces at openinfosecfoundation.org [discussion-bounces at openinfosecfoundation.org] On Behalf Of Edward Bjarte Fjellsk?l [edward.fjellskal at redpill-linpro.com] Sent: Friday, March 06, 2009 11:19 AM To: Victor Julien Cc: Kevin Ross; discussion at openinfosecfoundation.org Subject: Re: [Discussion] a few ideas Victor Julien wrote: > Edward Bjarte Fjellsk?l wrote: > >> I want to take this one step further, and try to do this automatic... Im >> working on a little perl daemon, to sniff the traffic, and detect OS and >> Services running on my network. Hopefully, in the future, this could be >> used to >> automatically help in the "auto categorization" of events... in sguil or >> other IDS gui... >> ( http://www.gamelinux.org/?p=43 and http://gamelinux.github.com/prads/ ) >> > > I'm still a bit torn on whether we should have the engine itself do the > detection of this information or if we should enable the engine to be > fed this info by external programs like your prads. > > Thoughts anyone? > > Regards, > Victor My thoughts are to keep them outside the engine, if it sucks up too much juice. Or the option to turn it off/on in the engine... and be able to have input from another sensor, or an external program. Im also in favor on the thought, that an external program would be better. The external program could be updated separately with fingerprints/signatures/rules without dependency on the Engine. The external program could also be used for other stuff.... Larger community :) But that said, exact values for ttl etc. the Engine should be using for a host, would best be predicted from the same data that the Engine sees. If the Engine depends on correct ttl (etc.) values........ So an external program might need to be placed correct, listening on the same TAP etc. e _______________________________________________ Discussion mailing list Discussion at openinfosecfoundation.org http://lists.openinfosecfoundation.org/mailman/listinfo/discussion NOTICE: This email and any attachments may contain confidential and proprietary information of NetSuite Inc and is for the sole use of the intended recipient for the stated purpose. Any improper use or distribution is prohibited and subject to legal sanctions. If you are not the intended recipient, please notify the sender; do not review, copy or distribute; and promptly delete or destroy all transmitted information. From kevross33 at googlemail.com Mon Mar 9 07:00:36 2009 From: kevross33 at googlemail.com (Kevin Ross) Date: Mon, 9 Mar 2009 12:00:36 +0000 Subject: [Discussion] rule language Message-ID: Hey, I was thinking if it is possibly to build on the rule language (using snort syntax as an example and including reliability) to include other device information. For example: alert tcp $HOME_NET any -> any 137 (msg"possible worm"; content:"iamabigbadworm"; depth:40; reliability:9; nepenthes_event:reliability:+5; firewall:to_port_137,count:50; reliability:+2; track:by_source; timeout:600 seconds; sid:1600001; rev:1;) I am just making up the syntax but the idea is you have what you want to match upon within the entire distributed IDS. For instance here a worm triggers an event and in this crude example the simple content match is simple snort and this is enough to generate an alert. However say in this example nepenthes generated an alert from the same source ip it will include that information in an alert and increase the overall relaibility then if a firewall (possibly through syslog or agents) sees the source IP trying to reach that port a certain amount of times it increases the reliability. Then the whole rule times out after 10 minutes (with the reliability:match having confirmed hostility anyway). If then it generates an alert but then subsequent events get grouped under the same alert. Then what it is passed over to whatever the analysis output is it can say: this worm sig triggered then this was seen on nepenthes plus firewall activity providing more information as well as possibly creating signatures to match upon unknown threats by correlating in a rule unusual activity. Having such a rule system could help tie together parts of the distibuted IDS. Oh and while I am at it; syslog content matches perhaps could be useful because then syslog messages can be matched upon in part of the sig. i.e content:syslog,"whatthecontentis". That way syslog compatible device messages can be sent to a syslog server and then the message can be parsed for an event increasing the reach of the distributed IDS. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090309/76efccf5/attachment.html From lists at inliniac.net Thu Mar 12 11:48:44 2009 From: lists at inliniac.net (Victor Julien) Date: Thu, 12 Mar 2009 17:48:44 +0100 Subject: [Discussion] rfc: different processing stages for rules Message-ID: <49B93CEC.40101@inliniac.net> I've been thinking some about a problem in the ideas as we currently see them. Many of the ideas require quite expensive checks, such as Martin Holste's timemachine ideas or Kevin Ross' ideas about querying external data sources like nepenthes and firewall logs. Add to that the ideas of having more higher level scripting languages for alert (post)processing, and we're going to run into serious latency and performance issues, especially when we're running inline. One possible solution would be to have two processing stages. Stage 1 is (relatively) simple: it detects content, stuff that can be detected in the current packet, or in multiple packets with the help of flowbits and the like. Basically, pretty much what Snort/Snort_inline does now. If in this stage an event is detected, we can still drop on it (most of the time, fragmented packets and spliced sessions can still be a challenge). In the second stage, all the expensive checks would come in to play. Detection here would be after-the-fact by definition, so dropping isn't possible no more. In a IPS setup we could fall back to dropping the rest of the packets for that flow/stream, active responses, snortsam style blocking. This detection could be offloaded to another thread or process to interrupt the normal packet flow as little as possible. I look forward to ideas & comments on this. Regards, Victor -- --------------------------------------------- Victor Julien http://www.inliniac.net/ PGP: http://www.inliniac.net/victorjulien.asc --------------------------------------------- From martin.fong at sri.com Thu Mar 12 15:29:21 2009 From: martin.fong at sri.com (Martin Fong) Date: Thu, 12 Mar 2009 13:29:21 -0700 Subject: [Discussion] [In the Weeds] Defining By Negation (Well, Actually, By Default) Message-ID: <49B970A1.5010907@sri.com> The current Snort variable syntax permits the Bourne shell-like syntax $(var:-default) but unlike Bourne shell, Snort does not permit "default" to be _empty_. This makes fine for requisite variables bound to rule action parameters, but this constraint is unnecessarily restrictive for preprocessors that employ their own parsers (-- in my case, I want users to optionally specify whitelisted IP addresses, CIDRs, port, and port ranges; thus, as part of the BotHunter Snort patches, I patched parser.c to meet my needs). My frustration is that the Snort implementation diverged from the underlying Bourne shell variable model, and that its documentation only documented the syntax, but not the semantics, of its rule grammar. Or, at a higher level, (1) when we decide to adopt a design paradigm, let's not bait-and-switch, and (2) let's create much more comprehensive and useful documentation. Cheers! ...Martin -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5193 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090312/f44188a7/smime.bin From c.criscione at securenetwork.it Wed Mar 18 04:12:57 2009 From: c.criscione at securenetwork.it (Claudio Criscione) Date: Wed, 18 Mar 2009 10:12:57 +0100 Subject: [Discussion] Just one question Message-ID: <200903181012.57515.c.criscione@securenetwork.it> Hello all, I've been quietly lurking in the last months, but I've got a question. It's still unclear to me if the new IDS will be snort_NG or not. It's not an issue by itself, mind you, but most paradigms or even programming techniques I've seen discussed in this list are natural evolutions of what is already there in snort. This can be perfectly ok if you are evolving snort, otherwise maybe you should try to discusse more... core issues.* Let me explain that with an example: why should you want to have just one rule processor? Whatever semantic you pull in it, it can't possibly cover whatever evolution we will have in the next years - or will do it unsatisfactory. See the limitations of snort on webapps, for instance, or in virtualized datacenters or cloud services. What about a plugin based rule system? A nibble of data gets in a score gets out, then you combin all the scores. Want to develop a virtualization based detector? Just plug it in and choose your semantic. Now, this is just an example, but I've got the feeling this project is very snort-oriented (which, let me say it again, is not an issue!). Even if it was obvious and stated somewhere, please make it clear for me :-) * please note that many of the insights I have seen in the list, be them snort related or not, are very interesting and good ideas! -- Claudio Criscione Secure Network S.r.l. Via Venezia, 23 - 20099 Sesto San Giovanni (MI) - Italia Tel: +39 02.24126788 Mob: +39 392 3389178 email: c.criscione at securenetwork.it web: www.securenetwork.it From jonkman at jonkmans.com Wed Mar 18 08:44:21 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Wed, 18 Mar 2009 09:44:21 -0400 Subject: [Discussion] Just one question In-Reply-To: <200903181012.57515.c.criscione@securenetwork.it> References: <200903181012.57515.c.criscione@securenetwork.it> Message-ID: <49C0FAB5.8020707@jonkmans.com> Very good question Claudio. Our charter is to build something new. We are talking about a lot of things that are the next thing for Snort, or what we wish Snort could do. But our primary goal is to break out of whatever limitations we've had in the past to build what we need for the long term. Plugins as you mentioned are definitely the right idea. I like your scoring idea. We have something similar on the roadmap. I liken it to spamassassin-style decision making. A lot of little factors could add up to be enough to cause an alert, or cause a stream to be captured in anticipation of it likely being bad or needing human analysis. Now will the new thing be 100% different from Snort? Likely not. We're trying to keep signature compatibility in some form to the old rules (no one wants to rewrite the last 10 years of signature intelligence). But we'll also likely have a second language that fits what we're hoping to get to in the future and that will likely be plugin based as you suggest. That help? Matt Claudio Criscione wrote: > Hello all, > I've been quietly lurking in the last months, but I've got a question. > It's still unclear to me if the new IDS will be snort_NG or not. It's not an > issue by itself, mind you, but most paradigms or even programming techniques > I've seen discussed in this list are natural evolutions of what is already > there in snort. This can be perfectly ok if you are evolving snort, otherwise > maybe you should try to discusse more... core issues.* > > Let me explain that with an example: why should you want to have just one rule > processor? > Whatever semantic you pull in it, it can't possibly cover whatever evolution > we will have in the next years - or will do it unsatisfactory. See the > limitations of snort on webapps, for instance, or in virtualized datacenters > or cloud services. > What about a plugin based rule system? A nibble of data gets in a score gets > out, then you combin all the scores. Want to develop a virtualization based > detector? Just plug it in and choose your semantic. > > Now, this is just an example, but I've got the feeling this project is very > snort-oriented (which, let me say it again, is not an issue!). Even if it was > obvious and stated somewhere, please make it clear for me :-) > > > > * please note that many of the insights I have seen in the list, be them snort > related or not, are very interesting and good ideas! > -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From c.criscione at securenetwork.it Wed Mar 18 09:12:07 2009 From: c.criscione at securenetwork.it (Claudio Criscione) Date: Wed, 18 Mar 2009 15:12:07 +0100 Subject: [Discussion] Just one question In-Reply-To: <49C0FAB5.8020707@jonkmans.com> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> Message-ID: <200903181512.08211.c.criscione@securenetwork.it> On mercoled? 18 marzo 2009 14:44:21 Matt Jonkman wrote: > I like your scoring idea. ;-) Thanks, that's what I've implemented in Masibty btw. > Now will the new thing be 100% different from Snort? Likely not. We're > trying to keep signature compatibility in some form to the old rules (no > one wants to rewrite the last 10 years of signature intelligence). [...] > That help? Definitely Matt, thank you. So - We have a plugin which can emulate snort... or run snort as a plugin :) - We have a "translator" - We start up with "snort_ng" adding features to the core snort engine while we eventually will move toward something different. Which one is the roadmap? (i've been trying to follow up the project, but I feel I'm lacking some project management info: my fault). -- Claudio Criscione Secure Network S.r.l. Via Venezia, 23 - 20099 Sesto San Giovanni (MI) - Italia Tel: +39 02.24126788 Mob: +39 392 3389178 email: c.criscione at securenetwork.it web: www.securenetwork.it From jjohnson at jdmc.org Wed Mar 18 13:30:56 2009 From: jjohnson at jdmc.org (John Johnson) Date: Wed, 18 Mar 2009 13:30:56 -0500 Subject: [Discussion] Just one question In-Reply-To: <49C0FAB5.8020707@jonkmans.com> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> Message-ID: <49C13DE0.8050107@jdmc.org> Hey guys, I'd like to take a sec to go over an idea. It's bad, but what the heck. :) There is a post about a new exec going around on the sigs list. I downloaded a copy, yup, not flagged by clamav. Wouldn't it really be nice to have a list of md5sum objects that could be a trigger? I don't mind if it can't be blocked, but it sure would be nice to say - it came from this IP at this time. John From hall.692 at osu.edu Wed Mar 18 14:14:02 2009 From: hall.692 at osu.edu (Seth Hall) Date: Wed, 18 Mar 2009 15:14:02 -0400 Subject: [Discussion] Just one question In-Reply-To: <49C13DE0.8050107@jdmc.org> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> Message-ID: <238B7B0B-AD5D-4C2D-81C0-533DD6FEFB74@osu.edu> On Mar 18, 2009, at 2:30 PM, John Johnson wrote: > There is a post about a new exec going around on the sigs list. I > downloaded a > copy, yup, not flagged by clamav. Wouldn't it really be nice to have > a list of md5sum > objects that could be a trigger? I don't mind if it can't be > blocked, > but it sure would > be nice to say - it came from this IP at this time. We've been doing this for quite a while with Bro based on the Team Cymru Malware Hash Registry (at least for files transferred over HTTP) by doing a DNS query after each md5 sum that we collect when we identify windows executables. It's nice because we don't have to maintain a list of md5 sums since we're doing the lookups over DNS. .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From jonkman at jonkmans.com Thu Mar 19 10:22:47 2009 From: jonkman at jonkmans.com (Matt Jonkman) Date: Thu, 19 Mar 2009 11:22:47 -0400 Subject: [Discussion] Just one question In-Reply-To: <49C13DE0.8050107@jdmc.org> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> Message-ID: <49C26347.8060504@jonkmans.com> How do you mean? In order to hash and have an md5 to compare the engine would have to grab and reconstruct the binary. I'm scared of the impact that'd have. Matt John Johnson wrote: > Hey guys, > > I'd like to take a sec to go over an idea. It's bad, but what the > heck. :) > > There is a post about a new exec going around on the sigs list. I > downloaded a > copy, yup, not flagged by clamav. Wouldn't it really be nice to have > a list of md5sum > objects that could be a trigger? I don't mind if it can't be blocked, > but it sure would > be nice to say - it came from this IP at this time. > > John > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion -- -------------------------------------------- Matthew Jonkman Emerging Threats Phone 765-429-0398 Fax 312-264-0205 http://www.emergingthreats.net -------------------------------------------- PGP: http://www.jonkmans.com/mattjonkman.asc From thorsten.holz at informatik.uni-mannheim.de Thu Mar 19 10:34:14 2009 From: thorsten.holz at informatik.uni-mannheim.de (Thorsten Holz) Date: Thu, 19 Mar 2009 08:34:14 -0700 Subject: [Discussion] Just one question In-Reply-To: <49C26347.8060504@jonkmans.com> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> Message-ID: <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> On 19.03.2009, at 08:22, Matt Jonkman wrote: > In order to hash and have an md5 to compare the engine would have to > grab and reconstruct the binary. I'm scared of the impact that'd have. Bro can already do that for HTTP traffic: "- The new analysis script http-identified-files.bro identifies the type of items returned by Web servers using libMagic (if available) and generates notices for interesting types and mismatches between URLs and types (Seth Hall). You configure it using two variables. watched_mime_types is a pattern (default /application\/x-dosexec/ | / application\/x-executable/ ) for which any MIME type matching the pattern generates a HTTP_WatchedMIMEType notice. mime_types_extensions is a table mapping strings to patterns specifying how URLs for the given MIME type should appear. (Ideally, this would be a table mapping patterns to patterns, but Bro doesn't currently support that.) It defaults to: ["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL] [lL])/ i.e., do Windows executables end in .exe or .dll. You can also redef the pattern ignored_urls to specify URLs that should not generate complaints. It defaults to matching Windows Update. - The new script http-extract-items.bro extracts the items from HTTP traffic into individual files (Vern Paxson). Files are named: .._._. where is a redef'able prefix (default: "http-item"), is a number uniquely identifying the item, the next four are describe the connection tuple, and is "orig" if the item was transferred from the originator to the responder, "resp" otherwise." Taken from the changelog (http://bro-ids.org/wiki/index.php/Version_1.4) Seth Hall is using that in production, perhaps he can report in the performance impact. Cheers, Thorsten From hall.692 at osu.edu Thu Mar 19 11:25:41 2009 From: hall.692 at osu.edu (Seth Hall) Date: Thu, 19 Mar 2009 12:25:41 -0400 Subject: [Discussion] Just one question In-Reply-To: <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> Message-ID: On Mar 19, 2009, at 11:34 AM, Thorsten Holz wrote: > Seth Hall is using that in production, perhaps he can report in the > performance impact. When we started running it, I was surprised because the additional performance impact was fairly negligible. Also, we aren't writing the files to disk (I agree, that would be bad), as chunks of the file come through we add each chunk to the incremental md5sum so the total CPU time required to calculate the md5sum is amortized across the time it takes for the file to be downloaded. We aren't calculating md5sums for everything either. We only calculate the md5sum if the file being downloaded was identified as a windows executable by libmagic. As a final data point, it looks like we see about 3000 - 3500 unique URLs daily where the server returns a Windows executable. .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From frank at knobbe.us Thu Mar 19 14:12:06 2009 From: frank at knobbe.us (Frank Knobbe) Date: Thu, 19 Mar 2009 14:12:06 -0500 Subject: [Discussion] Just one question In-Reply-To: References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> Message-ID: <1237489926.61678.30.camel@localhost> On Thu, 2009-03-19 at 12:25 -0400, Seth Hall wrote: > When we started running it, I was surprised because the additional > performance impact was fairly negligible. Also, we aren't writing the > files to disk (I agree, that would be bad), as chunks of the file come > through we add each chunk to the incremental md5sum so the total CPU > time required to calculate the md5sum is amortized across the time it > takes for the file to be downloaded. Nice. Does it also support zipped/compressed files? If so, I assume it does unzip/uncompress on the fly as well, without having to capture the whole file? Regards, Frank -- It is said that the Internet is a public utility. As such, it is best compared to a sewer. A big, fat pipe with a bunch of crap sloshing against your ports. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 188 bytes Desc: This is a digitally signed message part Url : http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090319/c5d13d90/attachment.bin From william.metcalf at gmail.com Thu Mar 19 18:19:47 2009 From: william.metcalf at gmail.com (Will Metcalf) Date: Thu, 19 Mar 2009 18:19:47 -0500 Subject: [Discussion] Just one question In-Reply-To: References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> Message-ID: This is cool stuff. Can you guy's currently reconstruct downloads that are offset from the end of the HTTP headers via content-disposition? Just curious... Regards, Will On Thu, Mar 19, 2009 at 11:25 AM, Seth Hall wrote: > > On Mar 19, 2009, at 11:34 AM, Thorsten Holz wrote: > >> Seth Hall is using that in production, perhaps he can report in the >> performance impact. > > > When we started running it, I was surprised because the additional > performance impact was fairly negligible. ?Also, we aren't writing the > files to disk (I agree, that would be bad), as chunks of the file come > through we add each chunk to the incremental md5sum so the total CPU > time required to calculate the md5sum is amortized across the time it > takes for the file to be downloaded. ?We aren't calculating md5sums > for everything either. ?We only calculate the md5sum if the file being > downloaded was identified as a windows executable by libmagic. ?As a > final data point, it looks like we see about 3000 - 3500 unique URLs > daily where the server returns a Windows executable. > > ? .Seth > > --- > Seth Hall > Network Security - Office of the CIO > The Ohio State University > Phone: 614-292-9721 > > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion > From hall.692 at osu.edu Thu Mar 19 21:11:13 2009 From: hall.692 at osu.edu (Seth Hall) Date: Thu, 19 Mar 2009 22:11:13 -0400 Subject: [Discussion] Just one question In-Reply-To: References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> Message-ID: <7B13E6DA-6A93-4BEE-8653-2DBEA59053ED@osu.edu> On Mar 19, 2009, at 7:19 PM, Will Metcalf wrote: > This is cool stuff. Can you guy's currently reconstruct downloads > that are offset from the end of the HTTP headers via > content-disposition? Just curious... I'm not sure I understand what you mean about the download being offset by the content-disposition header. As long as the body of the HTTP response is the content of the file then it should work. Maybe there's some functionality behind content-disposition I don't fully understand? .Seth From hall.692 at osu.edu Thu Mar 19 21:17:16 2009 From: hall.692 at osu.edu (Seth Hall) Date: Thu, 19 Mar 2009 22:17:16 -0400 Subject: [Discussion] Just one question In-Reply-To: <1237489926.61678.30.camel@localhost> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> <1237489926.61678.30.camel@localhost> Message-ID: On Mar 19, 2009, at 3:12 PM, Frank Knobbe wrote: > Nice. Does it also support zipped/compressed files? If so, I assume it > does unzip/uncompress on the fly as well, without having to capture > the > whole file? That's an interesting thought, and I'm sure it could be added relatively easily but no, there isn't support for that at the moment. .Seth From william.metcalf at gmail.com Thu Mar 19 23:55:02 2009 From: william.metcalf at gmail.com (Will Metcalf) Date: Thu, 19 Mar 2009 23:55:02 -0500 Subject: [Discussion] Just one question In-Reply-To: <7B13E6DA-6A93-4BEE-8653-2DBEA59053ED@osu.edu> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> <7B13E6DA-6A93-4BEE-8653-2DBEA59053ED@osu.edu> Message-ID: oops, sorry had content-disposition on the brain today. I meant range requests, so for example malware x gets installed outside of your environment and a user brings it back into your environment. Twice a day malware x checks for a new copy of itself but to avoid detection by inline AV's and something like the md5hash checks you speak of it pulls pieces of itself using range requests so almost like a download manager. How/can you deal with content reconstruction across multiple tcp sessions. I know inline AV scanners for the most part can't properly deal with this, I was just wondering if bro could. Hopefully that makes sense, I'm pretty sleepy at this point.... ;-) Regards, Will On Thu, Mar 19, 2009 at 9:11 PM, Seth Hall wrote: > > On Mar 19, 2009, at 7:19 PM, Will Metcalf wrote: > >> This is cool stuff. ?Can you guy's currently reconstruct downloads >> that are offset from the end of the HTTP headers via >> content-disposition? ?Just curious... > > > I'm not sure I understand what you mean about the download being offset by > the content-disposition header. ?As long as the body of the HTTP response is > the content of the file then it should work. ?Maybe there's some > functionality behind content-disposition I don't fully understand? > > ?.Seth > From hall.692 at osu.edu Fri Mar 20 05:56:21 2009 From: hall.692 at osu.edu (Seth Hall) Date: Fri, 20 Mar 2009 06:56:21 -0400 Subject: [Discussion] Just one question In-Reply-To: References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> <7B13E6DA-6A93-4BEE-8653-2DBEA59053ED@osu.edu> Message-ID: On Mar 20, 2009, at 12:55 AM, Will Metcalf wrote: > oops, sorry had content-disposition on the brain today. I meant range > requests, so for example malware x gets installed outside of your > environment and a user brings it back into your environment. Twice a > day malware x checks for a new copy of itself but to avoid detection > by inline AV's and something like the md5hash checks you speak of it > pulls pieces of itself using range requests so almost like a download > manager. How/can you deal with content reconstruction across multiple > tcp sessions. I know inline AV scanners for the most part can't > properly deal with this, I was just wondering if bro could. Hopefully > that makes sense, I'm pretty sleepy at this point.... ;-) Hah, that would be pretty sneaky. Is there any malware that does this? What's nice about Bro is that you can always modify your script you account for strange situation like this. I think I might write a script soon to see how common range requests are. What this scenario does make difficult is that it makes it harder to even identify that it's a windows executable, but I suppose they'd have to download the beginning of the file eventually anyway so you'd see one chunk of it matching as a windows executable. The easy hack we could do is kick of a download of the file once we notice a range request for an executable file. Bro will then have an opportunity to see the full file. Thanks for the question, that had never even crossed my mind. :) .Seth --- Seth Hall Network Security - Office of the CIO The Ohio State University Phone: 614-292-9721 From jjohnson at jdmc.org Fri Mar 20 10:41:40 2009 From: jjohnson at jdmc.org (jjohnson@jdmc.org) Date: Fri, 20 Mar 2009 10:41:40 -0500 (CDT) Subject: [Discussion] Just one question In-Reply-To: <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> Message-ID: <40432.72.198.121.84.1237563700.squirrel@www.jdmc.org> > Thorsten wrote: > > Bro can already do that for HTTP traffic: ... > Taken from the changelog (http://bro-ids.org/wiki/index.php/Version_1.4) > Seth Hall is using that in production, perhaps he can report in the > performance impact. Thank you for the pointer to a tool I had never heard of. I've got it compiled and am playing with it. For those of you looking for an example of a reassembly tool, my favorite has been tcpflow . http://www.circlemud.org/~jelson/software/tcpflow/ However, it caused some grief for me, apparently some of the ssh exploits looked like one big flow. Came in to the office and it looked like a 2 gig transfer to a foreign ip instead of a bunch of little attempts. John This message is confidential, intended only for the named recipient(s) and may contain information that is privileged or exempt from disclosure under applicable law. If you are not the intended recipient(s), you are notified that the dissemination, distribution or copying of this message is strictly prohibited. If you receive this message in error, or are not the named recipient(s), please notify the sender at the e-mail address above and delete this e-mail from your computer. Thank you. From william.metcalf at gmail.com Fri Mar 20 11:50:04 2009 From: william.metcalf at gmail.com (Will Metcalf) Date: Fri, 20 Mar 2009 11:50:04 -0500 Subject: [Discussion] Just one question In-Reply-To: References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> <7B13E6DA-6A93-4BEE-8653-2DBEA59053ED@osu.edu> Message-ID: Yeah I think there was malware in the wild last year that used bits... This guy published a PoC I believe.... that uses BITS to pull down an exe from his site... http://reconstructer.org/code/bitscode.zip Regards, Will On Fri, Mar 20, 2009 at 5:56 AM, Seth Hall wrote: > > On Mar 20, 2009, at 12:55 AM, Will Metcalf wrote: > >> oops, sorry had content-disposition on the brain today. ?I meant range >> requests, so for example malware x gets installed outside of your >> environment and a user brings it back into your environment. ?Twice a >> day malware x checks for a new copy of itself but to avoid detection >> by inline AV's ?and something like the md5hash checks you speak of it >> pulls pieces of itself using range requests so almost like a download >> manager. How/can you deal with content reconstruction across multiple >> tcp sessions. ?I know inline AV scanners for the most part can't >> properly deal with this, I was just wondering if bro could. ?Hopefully >> that makes sense, I'm pretty sleepy at this point.... ;-) > > > Hah, that would be pretty sneaky. ?Is there any malware that does this? > > What's nice about Bro is that you can always modify your script you account > for strange situation like this. ?I think I might write a script soon to see > how common range requests are. ?What this scenario does make difficult is > that it makes it harder to even identify that it's a windows executable, but > I suppose they'd have to download the beginning of the file eventually > anyway so you'd see one chunk of it matching as a windows executable. ?The > easy hack we could do is kick of a download of the file once we notice a > range request for an executable file. ?Bro will then have an opportunity to > see the full file. > > Thanks for the question, that had never even crossed my mind. :) > > ?.Seth > > --- > Seth Hall > Network Security - Office of the CIO > The Ohio State University > Phone: 614-292-9721 > > From nate+oisf at richmond-family.org Fri Mar 20 12:43:44 2009 From: nate+oisf at richmond-family.org (Nathaniel Richmond) Date: Fri, 20 Mar 2009 13:43:44 -0400 (EDT) Subject: [Discussion] Just one question In-Reply-To: <20090320165158.209F8A4053@medusa.richmond-family.org> References: <200903181012.57515.c.criscione@securenetwork.it> <49C0FAB5.8020707@jonkmans.com> <49C13DE0.8050107@jdmc.org> <49C26347.8060504@jonkmans.com> <694BBA98-836A-4CB4-A69D-A95865761F16@informatik.uni-mannheim.de> <7B13E6DA-6A93-4BEE-8653-2DBEA59053ED@osu.edu> <20090320165158.209F8A4053@medusa.richmond-family.org> Message-ID: <20090320174344.829F5A4053@medusa.richmond-family.org> Another interesting evasion technique was part of David Kennedy's Shmoocon presentation. Fast-Track can use Windows debug as an evasion method when downloading tools to a compromised host. A brief mention here: http://eatingsecurity.blogspot.com/2009/02/shmoocon-2009-notes.html His slides are here (3+MB PDF) and the use of Windows debug is mentioned around slide 10: http://www.thepentest.com/presentations/FastTrack_ShmooCon2009.pdf >From the slides: *  There is a payload delivery method using windows debug, this method takes specially formatted hexadecimal files and uses windows debug to convert our hex back to a binary. Slight problem with this technique is it has a limit of 64kb. If our payload is larger than that, we have an issue (examples meterpreter, vnc, etc.) *  Most attacks using this method drop a stager (like netcat for example) and netcat will initiate an outbound connection to download an additional payload > (often called a stager). Instead we created a small 5kb executable that takes in raw hex and spits out binary. *  So we use our ?stager? using the windows debug method for our 5kb file, then use our custom application to then convert raw hex to binary completely bypassing the 64kb restriction. Will Metcalf wrote: > Yeah I think there was malware in the wild last year that used > bits... > This guy published a PoC I believe.... that uses BITS to pull down > an > exe from his site... > > http://reconstructer.org/code/bitscode.zip > > Regards, > > Will > > On Fri, Mar 20, 2009 at 5:56 AM, Seth Hall wrote: >> >> On Mar 20, 2009, at 12:55 AM, Will Metcalf wrote: >> >>> oops, sorry had content-disposition on the brain today. ?I meant >>> range >>> requests, so for example malware x gets installed outside of your >>> environment and a user brings it back into your environment. >>> ?Twice a >>> day malware x checks for a new copy of itself but to avoid >>> detection >>> by inline AV's ?and something like the md5hash checks you speak >>> of it >>> pulls pieces of itself using range requests so almost like a >>> download >>> manager. How/can you deal with content reconstruction across >>> multiple >>> tcp sessions. ?I know inline AV scanners for the most part can't >>> properly deal with this, I was just wondering if bro could. >>> ?Hopefully >>> that makes sense, I'm pretty sleepy at this point.... ;-) >> >> >> Hah, that would be pretty sneaky. ?Is there any malware that does >> this? >> >> What's nice about Bro is that you can always modify your script >> you account >> for strange situation like this. ?I think I might write a script >> soon to see >> how common range requests are. ?What this scenario does make >> difficult is >> that it makes it harder to even identify that it's a windows >> executable, but >> I suppose they'd have to download the beginning of the file >> eventually >> anyway so you'd see one chunk of it matching as a windows >> executable. ?The >> easy hack we could do is kick of a download of the file once we >> notice a >> range request for an executable file. ?Bro will then have an >> opportunity to >> see the full file. >> >> Thanks for the question, that had never even crossed my mind. :) >> >> ?.Seth >> >> --- >> Seth Hall >> Network Security - Office of the CIO >> The Ohio State University >> Phone: 614-292-9721 >> >> > _______________________________________________ > Discussion mailing list > Discussion at openinfosecfoundation.org > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion > >