[Oisf-users] Suricata 4.0.3 with Napatech problems
Steve Castellarin
steve.castellarin at gmail.com
Wed Jan 10 18:29:15 UTC 2018
Hey Peter,
Here are some snippets from my suricata.yaml. Let me know if you want more:
max-pending-packets: 10000
runmode: workers
autofp-scheduler: active-packets
default-packet-size: 9018 # Same as defined in Napatech's NTSYSTEM.INI file
defrag:
hash-size: 65536
trackers: 65535
max-frags: 65535
prealloc: yes
timeout: 10
flow:
memcap: 1gb
hash-size: 1048576
prealloc: 1048576
prune-flows: 50000
emergency-recovery: 30
managers: 10
stream:
memcap: 12gb
checksum-validation: no
prealloc-session: 200000
inline: no
bypass: yes
reassembly:
memcap: 24gb
depth: 1mb
detect:
- profile: custom
- custom-values:
toclient-src-groups: 200
toclient-dst-groups: 200
- sgh-mpm-context: auto
- inspection-recursion-limit: 3000
mpm-algo: hs
spm-algo: hs
threading:
set-cpu-affinity: yes
cpu-affinity:
- management-cpu-set:
cpu: [ 1,21 ] # include only these cpus in affinity settings
mode: "balanced"
prio:
default: "low"
- worker-cpu-set:
cpu: [ 5,7,9,11,13,15,17,19,23,25,27,29,31,33,35,37,39 ]
mode: "exclusive"
# Use explicitely 3 threads and don't compute number by using
# detect-thread-ratio variable:
# threads: 3
prio:
default: "high"
detect-thread-ratio: 1.5
napatech:
hba: -1
use-all-streams: yes
>From my "ntservice.ini" file:
HostBufferPollInterval = 100
HostBufferSegmentSizeRx = default
HostBufferSegmentTimeOut = 1000
HostBuffersRx = [0,0,0],[17,2048,1]
My NTPL file is:
delete = all
hashmode=hash5tuplesorted
Setup[NUMANode=1] = StreamId==(0..16)
assign[streamid=(0..16)]=all
Deduplication [DynOffset = Layer2And3HeaderSize; Offset = 16] = Port == 0
Here's a snippet from the stats.log - at a point where I had 5 CPUs and 5
host buffers dropping packets.
nt0.pkts | Total |
33234531
nt0.bytes | Total |
27909846228
nt0.drop | Total |
14615598
nt1.pkts | Total |
24602782
nt1.bytes | Total |
21241406511
nt2.pkts | Total |
27257184
nt2.bytes | Total |
24469757866
nt2.drop | Total |
6348
nt3.pkts | Total |
24319656
nt3.bytes | Total |
21574261237
nt4.pkts | Total |
27663068
nt4.bytes | Total |
25185504694
nt5.pkts | Total |
30080524
nt5.bytes | Total |
26568388252
nt6.pkts | Total |
22950925
nt6.bytes | Total |
19749641965
nt7.pkts | Total |
28862367
nt7.bytes | Total |
25376339092
nt8.pkts | Total |
29496528
nt8.bytes | Total |
24543380786
nt9.pkts | Total |
26201481
nt9.bytes | Total |
22933047311
nt10.pkts | Total |
25542345
nt10.bytes | Total |
22641947527
nt11.pkts | Total |
25294114
nt11.bytes | Total |
22047976270
nt12.pkts | Total |
27476779
nt12.bytes | Total |
23909136217
nt12.drop | Total |
30820
nt13.pkts | Total |
26040161
nt13.bytes | Total |
22638575761
nt14.pkts | Total |
26332258
nt14.bytes | Total |
20614848384
nt15.pkts | Total |
20398822
nt15.bytes | Total |
17797433848
nt15.drop | Total |
5895511
nt16.pkts | Total |
24904544
nt16.bytes | Total |
22191856727
nt16.drop | Total | 405
decoder.pkts | Total |
450175098
decoder.bytes | Total |
390994516538
decoder.invalid | Total |
44193
decoder.ipv4 | Total |
450515575
decoder.ipv6 | Total | 701
decoder.ethernet | Total |
450175098
decoder.tcp | Total |
396577426
decoder.udp | Total |
41434420
decoder.icmpv4 | Total |
28341
decoder.gre | Total | 32
decoder.vlan | Total |
450175098
decoder.vlan_qinq | Total |
450175098
decoder.teredo | Total | 701
decoder.avg_pkt_size | Total | 868
decoder.max_pkt_size | Total |
1526
flow.tcp | Total |
6112630
flow.udp | Total |
756331
defrag.ipv4.fragments | Total |
1252944
defrag.ipv4.reassembled | Total |
384591
decoder.icmpv4.ipv4_unknown_ver | Total | 26
decoder.tcp.hlen_too_small | Total | 4
decoder.tcp.opt_invalid_len | Total | 17
decoder.vlan.unknown_type | Total |
44146
tcp.sessions | Total |
4824148
tcp.pseudo | Total | 806
tcp.syn | Total |
4899087
tcp.synack | Total |
2350078
tcp.rst | Total |
1273503
tcp.stream_depth_reached | Total |
10285
tcp.reassembly_gap | Total |
1091339
tcp.overlap | Total |
209277
detect.alert | Total | 16
app_layer.flow.http | Total |
399740
app_layer.tx.http | Total |
822952
app_layer.flow.ftp | Total |
1644
app_layer.flow.smtp | Total |
4291
app_layer.tx.smtp | Total |
4791
app_layer.flow.tls | Total |
1376046
app_layer.flow.ssh | Total | 783
app_layer.flow.dns_tcp | Total |
1758
app_layer.tx.dns_tcp | Total |
2032
app_layer.flow.enip | Total |
1005
app_layer.flow.failed_tcp | Total |
114223
app_layer.flow.dcerpc_udp | Total | 20
app_layer.flow.dns_udp | Total |
550775
app_layer.tx.dns_udp | Total |
551434
app_layer.tx.enip | Total |
1005
app_layer.flow.failed_udp | Total |
204531
flow_mgr.closed_pruned | Total |
1663125
flow_mgr.new_pruned | Total |
4209517
flow_mgr.est_pruned | Total |
949111
flow_mgr.bypassed_pruned | Total | 557
flow.spare | Total |
10482693
flow.tcp_reuse | Total | 379
flow_mgr.flows_checked | Total |
9327
flow_mgr.flows_notimeout | Total |
3491
flow_mgr.flows_timeout | Total |
5836
flow_mgr.flows_timeout_inuse | Total |
4874
flow_mgr.flows_removed | Total | 962
flow_mgr.rows_checked | Total |
1048576
flow_mgr.rows_skipped | Total |
1038992
flow_mgr.rows_empty | Total | 599
flow_mgr.rows_maxlen | Total | 25
tcp.reassembly_memuse | Total |
243974776
dns.memuse | Total |
1588284
http.memuse | Total |
78622620
flow.memuse | Total |
382530784
On Wed, Jan 10, 2018 at 1:17 PM, Peter Manev <petermanev at gmail.com> wrote:
> On Wed, Jan 10, 2018 at 11:08 AM, Steve Castellarin
> <steve.castellarin at gmail.com> wrote:
> > All,
> >
> > I've been running Suricata 3.1.1 (with Hyperscan) on an Ubuntu 14.04.5
> 64bit
> > system with an older Napatech driver set for quite a while with no
> issues.
> > The system is running dual E5-2660 v3 @2.60Ghz processors with 128Gb of
> > memory. I've gone ahead and upgraded the Napatech drivers to 10.0.4 and
> > downloaded/compiled Suricata 4.0.3. I've done the best I can to copy
> > configuration settings from the 3.1.1 suricata.yaml to the 4.0.3
> > suricata.yaml. I run Suricata by issuing:
> > /usr/bin/suricata -c /etc/suricata/suricata.yaml --napatech --runmode
> > workers -D
> >
> > I continue to see issues where Suricata will run for a time when I notice
> > one of the CPUs hitting 100%, and stay there. Then when running
> Napatech's
> > "profiling" command I'll see one of the host buffers dropping 100% of the
> > packets. As time goes along another CPU/host buffer will have the same
> > issue, etc, etc.
> >
> > I've been banging my head over this for a couple weeks with no success,
> > other than killing the Suricata process then restarting - to only have
> this
> > issue crop up again.
> >
> > One thing I notice, when I issue the "kill `pidof suricata`" Suricata
> will
> > take a while to end gracefully. But, it leaves the PID file behind in
> > /var/run.
> >
> > Any ideas on how to attack this, before I have to roll back my upgrade?
> >
>
> Can you share some more info on your suricata config and any info in
> suricata.log/stats.log?
>
> > Thanks!!
> >
> > _______________________________________________
> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> > Site: http://suricata-ids.org | Support: http://suricata-ids.org/
> support/
> > List: https://lists.openinfosecfoundation.org/
> mailman/listinfo/oisf-users
> >
> > Conference: https://suricon.net
> > Trainings: https://suricata-ids.org/training/
>
>
>
> --
> Regards,
> Peter Manev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20180110/a82cba49/attachment-0002.html>
More information about the Oisf-users
mailing list