<div dir="ltr"><div>Hello:</div><div><br></div><div>After having no luck with PF_RING (see email on File Extraction Woes) I decided to recompile a new kernel (3.14.5) on my CentOS machine and give AF_PACKET a shot. I am happy to report that the total loss is at a much more manageable level. </div>
<div><br></div><div>Unfortunately, I seemed to have encountered a different issue concerning filtering and alerting at the application layer. Suri seems to have a difficult time working with HTTP rules I create as well as logging when run on the wire. </div>
<div><br></div><div>For example, take the following rule...</div><div><br></div><div>alert tcp any any -> 216.34.181.45 any (msg:"This is a test TCP Rule"; flow:established,to_server; content:"<a href="http://slashdot.org">slashdot.org</a>"; http_header; sid:9998; rev:1;)</div>
<div><br></div><div>When running Suri on the wire. This alert will simply not fire at all. Moreover, I notice on several occasions the requests are not logged properly to http.log. This of course, has an amplifying effect on file extraction where samples are truncated at best, and at worst not ever logged.</div>
<div><br></div><div>I decided to test things by taking the same data as a pcap, and running it against Suri...</div><div><br></div><div>/usr/bin/suricata -c /etc/suricata/suricata.yaml --runmode single -r slashdot.pcap</div>
<div><br></div><div>At this point, Suri does detect on the traffic in the PCAP. </div><div><br></div><div>So it would seem the issue is isolated to traffic on the wire. I decided to check the following parameters to see if there were any issues...</div>
<div><br></div><div>Example thread (1 of 16).</div><div>capture.kernel_packets | AFPacketp4p21 | 60645314<br>capture.kernel_drops | AFPacketp4p21 | 71483<br>tcp.segment_memcap_drop | AFPacketp4p21 | 0<br>
tcp.reassembly_memuse | AFPacketp4p21 | 1306325000<br></div><div>decoder.invalid | AFPacketp4p21 | 0</div><div><br></div><div>Seems fine to me, and the rest of the threads are more/less the same. </div>
<div><br></div><div>Now if I take a non application layer rule, like a simple ICMP rule below...</div><div><br></div><div>alert icmp $MY_IP_ADDRESS any -> any any (msg:"This is a test ICMP Rule"; sid:9999; rev:1;)</div>
<div><br></div><div>I hit reliably both on the wire and on a pcap. This is why I feel there has to be some issue with application layer processing. I am somewhat perplexed by this and seek any guidance those with more experience may be able to give. Thanks in advance!</div>
<div><br></div><div>Here are some additional config attributes that might be helpful.</div><div><br></div><div>== Suri Config==</div><div><br></div><div><p>runmode: workers</p><p>host-mode: sniffer-only</p><p>default-packet-size: 9000<br>
</p></div><div>## Stream Settings ##</div><div>stream:<br> memcap: 1gb<br> checksum-validation: no <br> prealloc-sessions: 10000000</div><div> inline: no</div><div> reassembly:<br> memcap: 2gb<br> depth: 6mb # reassemble 1mb into a stream<br>
toserver-chunk-size: 2560<br> toclient-chunk-size: 2560<br> randomize-chunk-size: yes<br> #randomize-chunk-range: 10<br> #raw: yes<br> chunk-prealloc: 100000<br> segments:<br> - size: 4<br> prealloc: 100000<br>
- size: 16<br> prealloc: 100000<br> - size: 112<br> prealloc: 500000<br> - size: 248<br> prealloc: 250000<br> - size: 512<br> prealloc: 100000<br> - size: 768<br> prealloc: 50000<br>
- size: 1448<br> prealloc: 300000<br> - size: 65535<br> prealloc: 3000</div><div><br></div><div>### Flow Timeouts Snip ###</div><div> tcp:<br> new: 60<br> established: 120<br> closed: 30<br>
emergency-new: 10<br> emergency-established: 10<br> emergency-closed: 10<br></div><div><br></div><div>### Flow ###</div><div>flow:<br> memcap: 1gb<br> hash-size: 1048576<br> prealloc: 1048576<br> emergency-recovery: 30<br>
</div><div><br></div><div>### Detect Engine ###</div><div>detect-engine:<br> - profile: custom<br> - custom-values:<br> toclient-src-groups: 200<br> toclient-dst-groups: 200<br> toclient-sp-groups: 200<br>
toclient-dp-groups: 300<br> toserver-src-groups: 200<br> toserver-dst-groups: 400<br> toserver-sp-groups: 200<br> toserver-dp-groups: 250<br> - sgh-mpm-context: auto<br> - inspection-recursion-limit: 3000</div>
<div><br></div><div>Using the following hardware:</div><div><br></div><div>== Hardware Specs ==<br>CPU: Intel Xeon CPU @ 2.40Ghz x 32<br>RAM: 48G</div><div>NIC:</div><div> *-network:1<br> description: Ethernet interface<br>
product: Ethernet 10G 2P X520 Adapter<br> vendor: Intel Corporation<br> physical id: 0.1<br> bus info: pci@0000:42:00.1<br> logical name: p4p2<br> version: 01<br> serial: a0:36:9f:07:ec:02<br>
capacity: 1GB/s<br> width: 64 bits<br> clock: 33MHz<br> capabilities: pm msi msix pciexpress vpd bus_master cap_list rom ethernet physical fibre 1000bt-fd autonegotiation<br> configuration: autonegotiation=on broadcast=yes driver=ixgbe driverversion=3.19.1-k duplex=full firmware=0x8000030d latency=0 link=yes multicast=yes port=fibre<br>
resources: irq:76 memory:d0f00000-d0f7ffff ioport:7ce0(size=32) memory:d0ffc000-d0ffffff memory:d1100000-d117ffff memory:d1380000-d147ffff memory:d1480000-d157ffff<br></div></div>