[Oisf-users] [Question] suricata test with pcap-file(After upgrading the suricata version(2.0.11 --> 3.2))
박경호
pgh5247 at naver.com
Tue Jan 31 09:20:51 UTC 2017
Thank you for your efforts.
i was also able to have consistent number of logs/alerts through all the pcap runs (with --runmode=single) with the provided pcap and other pcap files.
When i ran the suricata the multiple pcap files with 'autofp runmode', the resulsts were different through all the pcap runs(reassemble memcap was set '2gb')
-----Original Message-----
From: "Peter Manev"<petermanev at gmail.com>
To: "Andreas Herz"<andi at geekosphere.org>;
Cc: "oisf-users at lists.openinfosecfoundation.org"<oisf-users at lists.openinfosecfoundation.org>;
Sent: 2017-01-28 (토) 02:07:58
Subject: Re: [Oisf-users] [Question] suricata test with pcap-file(After upgrading the suricata version(2.0.11 --> 3.2))
On Tue, Jan 24, 2017 at 11:49 PM, Andreas Herz <andi at geekosphere.org> wrote:
> On 24/01/17 at 15:01, 박경호 wrote:
>> Sorry to send an email which the huge file to be attached.
>> Instead of an email, i will share the google drive.
>> the link is https://drive.google.com/file/d/0B4Mdb8bpuRlnU282SzRmRGQ3VXc/view?usp=sharing.
>>
>
> Thanks for the pcap, we can reproduce the issue and will look into it!
After doing some investigation with a joint meerkat effort :)
- Andreas started digging ....
- Jason pointed out the timestamps are out of order (due many pcaps
merged into 1). Fixed by -
reordercap testpcap.pcap testpcap-order.pcap
- I noticed that no matter how many read runs (-r) run we were always
hitting segment memcaps (default suricata.yaml settings) - once the
reassembly memcap was raised to 256mb we did not have the tcp segment
memcap hits any more and we were able to have consistent number of
logs/alerts through all the pcap runs (with --runmode=single) with the
provided pcap.
When using autofp runmode though i needed to up the reassembly memcap
again to 512mb this time to get to the same consistency. This left me
a bit puzzled and ...
- Victor came to the rescue with figuring it all out:
Many of the segments in the pcap have a size of 1460, but our code has pools for
1448, then 65535 (a catch all). So very many of the segments use those
64k memory blocks.
( https://redmine.openinfosecfoundation.org/projects/suricata/repository/revisions/master/entry/suricata.yaml.in#L1223
)
Add 1460 like so
- size: 1448
prealloc: 1024
- size: 1460
prealloc: 1024
- size: 65535
prealloc: 128
(when Suri is compiled with --enable-debug, you get relevant info on
shutdown if more was used than the prealloc ones )
That solution (better segments size allocation adjustment for this
particular case) made my tcp segment memcaps hits
(stats.log/eve.json) go away in the test runs.
I think should also fix the issue you were having.
Thanks
>
>> -----Original Message-----
>> From: "Andreas Herz"<andi at geekosphere.org>
>> To: <oisf-users at lists.openinfosecfoundation.org>;
>> Cc:
>> Sent: 2017-01-24 (화) 05:11:57
>> Subject: Re: [Oisf-users] [Question] suricata test with pcap-file(After upgrading the suricata version(2.0.11 --> 3.2))
>>
>> You can't attach such huge files on mails to the mailinglist. Please try
>> to upload them somehwere you trust or try to reduce them to like 10 or
>> 20mb so you can send those to us directly via mail (not the
>> mailinglist!)
>>
>> On 20/01/17 at 13:11, 박경호 wrote:
>> > 대용량 첨부파일 1개(106MB)대용량 첨부 파일은 30일간 보관 / 100회까지 다운로드 가능 testpcap.pcap 106MB 다운로드 기간: 2017/01/20 ~ 2017/02/19I attached the pcap file to use for testing.
>> > file size is 111MBytes.
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: "Andreas Herz"<andi at geekosphere.org>
>> > To: <oisf-users at lists.openinfosecfoundation.org>;
>> > Cc:
>> > Sent: 2017-01-20 (금) 06:13:22
>> > Subject: Re: [Oisf-users] [Question] suricata test with pcap-file(After upgrading the suricata version(2.0.11 --> 3.2))
>> >
>> > On 16/01/17 at 17:15, 박경호 wrote:
>> > > I did the test to use two smaller pcap files. one is 111MB and another is 66MB.
>> > > when i run the suricata twice with 111MB pcap file, the alert messages are different.
>> > > But when i run the suricata twice with 66MB pcap file, the alert messagte is same.
>> > > I merged the two pcap files(45MB, 66MB) to one pcap file(111MB) using wire-shark.
>> >
>> > Can you share thoe 11MB pcap here or with us from the OISF team?
>> >
>> > > Is the this issue computing resources?(specially ram memory issue?)
>> >
>> > I wouldn't say for sure it's an memory issue.
>> >
>> > > Can you recommend me how much memory i need in the following situation?
>> > > when i check some pcap files which the size is more than 1GB with suricata, how much memory do i need?
>> >
>> > 8GB are not that low IMHO.
>> >
>> > > And,
>> > > If I add the memory in my computer, which parts are changed in configuration file(suricata.yaml)?
>> >
>> > Without you changing it, nothing.
>> >
>> > >
>> > > -----Original Message-----
>> > > From: "박경호"<pgh5247 at naver.com>
>> > > To: "Andreas Herz"<andi at geekosphere.org>; <oisf-users at lists.openinfosecfoundation.org>;
>> > > Cc:
>> > > Sent: 2017-01-16 (월) 16:17:33
>> > > Subject: Re: [Oisf-users] [Question] suricata test with pcap-file(After upgrading the suricata version(2.0.11 --> 3.2))
>> > >
>> > >
>> > > -----Original Message-----
>> > > From: "Andreas Herz"<andi at geekosphere.org>
>> > > To: <oisf-users at lists.openinfosecfoundation.org>;
>> > > Cc:
>> > > Sent: 2017-01-14 (토) 06:19:16
>> > > Subject: Re: [Oisf-users] [Question] suricata test with pcap-file(After upgrading the suricata version(2.0.11 --> 3.2))
>> > >
>> > > On 12/01/17 at 10:48, 박경호 wrote:
>> > > > After upgrading the version from 2.0.11 to 3.2, I did the test again.
>> > > > Unfortunately, alert messages were different whenever the suricata was
>> > > > run with same a pcap-file.
>> > >
>> > > Can you be more verbose about that?
>> > > ==> i run the suricata like the following command : suricata -c suricata.yaml -r testpcap.pcap
>> > > ( i never changed the configure file(.yaml)).
>> > >
>> > > > I didn't change the configure file(suricata.yaml) and pcap-file's size
>> > > > is 693MB. (pc memory is 8GB, cpu is intel i5-4460, os is Ubuntu
>> > > > 16.06)
>> > >
>> > > Can you try to reproduce the issue with a smaller pcap file that you can
>> > > share with us?
>> > > ==> After i try to reproduce with a smaller pcap file, i will share the result and pcap file.
>> > >
>> > > > please explain to me about this situation.
>> > >
>> > > I still need more details about your suricata configuration, how do you
>> > > run suricata, what did you configure?
>> > >
>> > > An easy way to reproduce that for us will help to find a solution (after
>> > > we found what's the real issue you have).
>> > >
>> > > --
>> > > Andreas Herz
>> > > _______________________________________________
>> > > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> > > Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> > > List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> > >
>> > >
>> >
>> > --
>> > Andreas Herz
>> > _______________________________________________
>> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> > Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> > List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>
>> --
>> Andreas Herz
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> --
> Andreas Herz
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
--
Regards,
Peter Manev
_______________________________________________
Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20170131/ca4538f6/attachment-0002.html>
More information about the Oisf-users
mailing list