[Oisf-devel] DNS/SMTP content modifier fields.

[Puranik, Aditya]

Hi Victor,
Hope you are doing good. 
This is in continuation to the new keyword and content modifier buffers we are trying to implement for SMTP/DNS/SMP.
I am currently working on enhancing SMTP keywords.

I followed the approach suggested by you for adding these a sticky buffers. I have added the code for "smtp_mail_from" and "smtp_rcpt_to". The two new keywords are parsed by the sig engine and alerts are raised for content match on these buffer.

However I have few questions with respect to implementing keywords for other MIME types like subject, user_agent, x_originating_ip, content_md5 and a bunch of other header/document fields.
1. I am able to find various fields using decode util functions (MimeDecFindField). This function looks for each of the field and puts the value in MimeDecField variable. For each of these keyword where each keyword is looked up repeatedly for getting the buffer and then passing to InspectionBufferSetup may be little costly.

I was thinking if we can parse each of this entity in transaction and keep a lookup of "entity":"buffer address" which can then be passed to InspectionBufferSetup when a particular keyword is seen in signature.
Please let me know your thoughts on this approach and any concerns you see if we implement like this.

Regards,
Aditya 

-----Original Message-----
From: Puranik, Aditya 
Sent: Monday, April 1, 2019 5:37 PM
To: Victor Julien <lists at inliniac.net>; oisf-devel at lists.openinfosecfoundation.org
Cc: Hossain, Mahmud <mahhossa at visa.com>; Samaroo, Joel <jsamaroo at visa.com>; Arabally, Gopal <garaball at visa.com>
Subject: RE: [Oisf-devel] DNS/SMTP content modifier fields.

Hi Victor,
Thanks for the response on the query. I will go through the files you have mentioned and try to add these functionality.
Will reach out to you in case I need further details.

Regards,
Aditya 

-----Original Message-----
From: Victor Julien <lists at inliniac.net>
Sent: Monday, April 1, 2019 2:43 PM
To: Puranik, Aditya <apuranik at visa.com>; oisf-devel at lists.openinfosecfoundation.org
Cc: Hossain, Mahmud <mahhossa at visa.com>; Samaroo, Joel <jsamaroo at visa.com>; Arabally, Gopal <garaball at visa.com>
Subject: Re: [Oisf-devel] DNS/SMTP content modifier fields.

Hi Aditya,

On 26-03-19 09:36, Puranik, Aditya wrote:
> We are trying to explore and add new content modifier fields for 
> SMTP/DNS/ SMB protocols. Also we would like to expand this to other 
> protocols based on our requirements.
> 
> At present, Suricata has a very good coverage for HTTP protocol.
> 
> https://suricata.readthedocs.io/en/suricata-4.1.2/rules/http-keywords.
> html
> 
> HTTP content modifiers gives us better control to write a new rule or 
> to modify existing signatures.
> 
>  
> 
> We are planning to have similar modifiers for protocols like SMTP, 
> DNS, SMB.
> 
> For example: SMTP packet exchange/flow contains a bunch of fields like 
> mailfrom, rcptto, date, from, to, cc, reply_to, subject, 
> x_originating_ip, user_agent et al.
> 
> Something like : alert smtp $EXTERNAL_NET any -> $HOME_NET 
> (content*/:"username at example.com"; mail_from/*; sid:1;)

We are doing all new keywords as 'sticky buffers', so:

'mail_from; content:abc;'

This is simpler to implement in Suricata and simpler from the rule writer perspective as well.


> The similar requirements holds good for other set of protocols mentioned.
> 
>  
> 
> We are interested in knowing
> 
> 1.       What is the right approach to go about adding or modifying 
> the rules with new content modifiers?

Rule modifications are usually done using suricata-update's modify.conf, or a similar facility in other tools like pulledpork.


> 2.       What is the placeholder in the code if we start adding these 
> new modifiers in the rules?

I would suggest having a look a the detect-template-buffer.[ch] files.
In scripts/ we have setup scripts to instantiate them into your own keyword.

See:
https://github.com/OISF/suricata/blob/master/src/detect-template-buffer.c
https://github.com/OISF/suricata/blob/master/src/detect-template-rust-buffer.c
https://github.com/OISF/suricata/blob/master/scripts/setup-app-layer.py


> 3.       If code changes are needed to implement this functionality 
> what is the correct approach to accommodate these changes in existing 
> Suricata design?

Code changes will certainly be needed. See the template I mentioned above.

Regards,
Victor


--
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------