[Oisf-users] SuriCon brainstorm summary posted

[Cooper F. Nelson]

Hi Victor,

Thanks for posting this, unfortunately I could make it this year due to
budget and timing issues.

A few notes based on what I've learned/observed in the field since last
years Suricon...

Failing better:

It would be great if there were something likes a 'drop' log that could
record what flows are causing suricata to drop packets.  I'm currently
doing this manually.

File extraction:

Last I checked the 'bypass bypass' feature that allowed for extracting
files past the stream depth was broken in the 4 series. 

QA/Misc:

RE: recording pcaps; I'm looking for a way to archive individual TCP and
UDP flows for a machine learning project.  One file per flow, with the
IP/port/protocol info in the filename.  I believe suricata supports this
for TCP already, if there is a way to do it via Lua in a generic way for
all protocols I would appreciate being pointed in the right direction. 

RE: traffic ID ruleset:  I've already done work scraping and collecting
content delivery networks from various sources.  I could probably work
on automating this, but I think this would be more of an EmergingThreats
thing to be released as an IP ruleset (like TOR).  I'll also suggest
there is a need for a 'bypass' rule action that functions like the
current 'pass' and also creates an ebpf bypass rule for both TCP and UDP
flows.  Google's UDP QUIC protocol is becoming more and more popular and
already causing performance issues on our sensors.  The flow metadata
should still be logged, if available. 

While I initially considered that adding the flow bypass as a default
behavior to the existing 'pass' rule action was the way to go, I
realized this could create performance problems if you are bypassing a
high volume of tiny flows.  So better to make it a separate action.

Addendum:

I was hoping to do a small 'B side' on improving suricata performance
(dramatically) via leveraging the offloading features in modern
NICs/drivers.  I know there has been some concern in the past that these
features (particularly LRO) breaks state tracking and the dsize
keyword.  I've done an extensive amount of testing using the tpacket-v3
capture mode with a max packet size of 65536 and so far it appears that
these are *not* issues when using GRO (which supersedes and disables
LRO).  I found this reference that explains the difference:

> https://lwn.net/Articles/358910/

The tl;dr is that GRO will only reassemble packets with identical TCP
headers (sans sequence number) and timestamps.  So TCP SYN/FIN packets
and small packets not part of a big bulk 'push' are not affected.

Thanks for all the hard work, I'll take a more in-depth look at the
ticket in January and see if I can contribute anything.  If anybody has
any ideas about the TCP/UDP flow recording for machine learning in
particular I would really appreciate it. 

-Coop

On 12/1/2017 8:27 AM, Victor Julien wrote:
> Hi all,
>
> We've written up a summary of the brainstorm in Prague last month.
>
> https://suricata-ids.org/2017/12/01/suricon-2017-brainstorm-summary/
>
> If you're looking for something to do, the following 'super ticket'
> contains links to all tickets that were discussed at SuriCon, or were
> created based on suggestions at SuriCon:
>
> https://redmine.openinfosecfoundation.org/issues/2309
>
> If you're interested in helping out with one or more of them, please let
> us know!
>

-- 
Cooper Nelson
Network Security Analyst
UCSD ITS Security Team
cnelson at ucsd.edu x41042


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20171216/ea2bec14/attachment-0002.sig>