[Oisf-users] (no subject)

[Cooper F. Nelson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Doh, did you follow this guide?

> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/File_Extraction

Specifically these parts?

> stream.checksum_validation controls whether or not the stream engine
> rejects packets with invalid checksums. A good idea normally, but the
> network interface performs checksum offloading a lot of packets may
> seem to be broken. This setting is enabled by default, and can be
> disabled by setting to "no". Note that the checksum handling can be
> controlled per interface, see "checksum_checks" in example
> configuration.
> 
> stream.reassembly.depth controls how far into a stream reassembly is
> done. Beyond this value no reassembly will be done. This means that
> after this value the HTTP session will no longer be tracked. By
> default a settings of 1 Megabyte is used. 0 sets it to unlimited.
> 
> libhtp.default-config.request-body-limit /
> libhtp.server-config.<config>.request-body-limit controls how much of
> the HTTP request body is tracked for inspection by the
> http_client_body keyword, but also used to limit file inspection. A
> value of 0 means unlimited.
> 
> libhtp.default-config.response-body-limit /
> libhtp.server-config.<config>.response-body-limit is like the request
> body limit, only it applies to the HTTP response body.

I would set these all to '0' for testing, especially if you are seeing
truncated files.

- -Coop

On 3/28/2014 7:22 AM, Travel Factory S.r.l. wrote:
> 
> 
>> Judging from the output - your drops are minimal, less than ~%0.02
> 
> Yes, I noticed that... ifconfig has a drop ratio of about 0.04%. If
> you notice the capture.kernel_drops doesn't increase.
> 
> But my tests show that I can't filestore reliably.
> 
> I uploaded the startup messages to http://pastebin.com/CPye8Wjf 
> Please have a look at two points: <Info> - AutoFP mode using default
> "Active Packets" flow load balancer and <Warning> - [ERRCODE:
> SC_ERR_AFP_READ(191)] - poll failed with retval 0 Of the Warnings I
> sometimes get 0, 1, or more... (http://pastebin.com/HQUjSc8x for the
> message when stopping suricata)
> 
> Can you please have a look at other logged values, if there is a
> hint that somehow a filestore is truncated... I don't know the
> meaning of the logged values, like these: tcp.stream_depth_reached 
> detect.alert
> 
> Repeating in this moment my tests (40 wget of the same file) I get
> 26 files stored ok, and the rest are partial...
> 
> # ll *.meta | wc -l 930 # grep -h STATE *.meta | sort | uniq -c 365
> STATE:             CLOSED 46 STATE:             TRUNCATED
> 
> On 930 downloads that suricata started to filestore, only 365 were 
> "correctly" closed (but I can't say that they were stored
> succesfully) 46 were flagged as truncated. What happened to the other
> 520 files?
> 
> Are these numbers/percentages expected ?
> 
>> I do not think there is a Suricata problem.
> 
> Don't know what to think.... at this point I think the only way is
> to take a specialized capture card and see if all packets are
> correctly forwarded to the server. Last week I used tcpdump to save a
> pcap file and used suricata to read it: it could not create complete
> files.
> 
> What puzzle me more is that last week the server captured flawlessy
> for about 2 hours after changing a setting with ethtool. Stopping
> and restarting suricata changed something: I realized that irq
> affinity was gone.
> 
> Thanks Francesco
> 
> 
>> Some of the drops might very well be "legal" , tcp gaps and such.
>> 
>> 
>> 
>> 
>> -- Regards, Peter Manev
> 
> _______________________________________________ Suricata IDS Users
> mailing list: oisf-users at openinfosecfoundation.org Site:
> http://suricata-ids.org | Support: http://suricata-ids.org/support/ 
> List:
> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users 
> OISF: http://www.openinfosecfoundation.org/


- -- 
Cooper Nelson
Network Security Analyst
UCSD ACT Security Team
cnelson at ucsd.edu x41042
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTNbyKAAoJEKIFRYQsa8FWSkAIAId5kiywh8Qx3cWKjIHTLsBT
coOr+bQzXHthk2ay/iQanOeu+nMjRfBqet+G7B2yhCSOyEdD0auvZIsGnZF2ARtL
MA8blwJfBYlNVFkXIyquNNA5L6iOy+PPuZkS8EnBrpaJJSHMJPH3IphDQTEw+BxY
QnlTHGToH/KauZV/5d9Vk1kCiPRotGKvoG0RrIuoNnfpTp/jNrmXfrmWmuy4WFSc
o+vG49MwC/JDda5sx5C3m6oTydlrnTBItTT1CQTy9jwNL9RO1CRzs+grDi0Q4hLN
XqqVgJ2E5Zk3JZdR9iEXsZsSYCXujLsTVyGi7sAr8AecTJkSC4w12SCppj4OrOY=
=46XB
-----END PGP SIGNATURE-----