[Oisf-devel] 77b708=WIN?
Chris Wakelin
c.d.wakelin at reading.ac.uk
Tue Sep 27 14:33:56 UTC 2011
I've just upgraded my campus network monitor to latest SVN
(6bad2dbd7964a2e465ff4829022acf1e6c34062d) and PF_RING 5.1.0 (non-DNA).
I've applied two patches, my extended HTTP log one and Will's PF_RING
"single" runmode (included in part of Bug 315) but with the same changes
Anoop included in a844eecb0e115758eb87d515d37b9fcd877d2fec
I'm running with 6 threads and --runmode=single.
I'd not seen any more of the htp_tx_destroy crashes in the last week or
so, but with this version I've had two new ones (running fine for hours
in between):
> Core was generated by `/opt/RDGsuricata/bin/suricata --pfring-int eth1 -c /etc/suricata/suricata.yaml'.
> Program terminated with signal 11, Segmentation fault.
> #0 0x00000000004401ac in StreamPatternSearch (det_ctx=0x3605aa50, p=0x3605aaf8, smsg=0x7f335f5dadc0, flags=0 '\000') at detect-engine-mpm.c:427
> 427 if (smsg->data.data_len < det_ctx->sgh->mpm_streamcontent_maxlen)
> #0 0x00000000004401ac in StreamPatternSearch (det_ctx=0x3605aa50, p=0x3605aaf8, smsg=0x7f335f5dadc0, flags=0 '\000') at detect-engine-mpm.c:427
> r = <value optimised out>
> ret = 6
> cnt = 1 '\001'
> #1 0x0000000000427e30 in DetectMpmPrefilter (th_v=<value optimised out>, de_ctx=<value optimised out>, det_ctx=0x3605aa50, p=0x43b53990) at detect.c:1219
> No locals.
> #2 SigMatchSignatures (th_v=<value optimised out>, de_ctx=<value optimised out>, det_ctx=0x3605aa50, p=0x43b53990) at detect.c:1435
> sms_runflags = 1 '\001'
> alert_flags = <value optimised out>
> alproto = 1
> match = <value optimised out>
> fmatch = 0
> idx = <value optimised out>
> flags = 10 '\n'
> alstate = 0x7f3390217290
> smsg = 0x7f3362509930
> s = <value optimised out>
> sm = <value optimised out>
> alversion = 10
> mask = <value optimised out>
> #3 0x000000000042831c in Detect (tv=0x3605ab18, p=<value optimised out>, data=0x36061dad, pq=<value optimised out>, postpq=0x312) at detect.c:1857
> de_ctx = 0x0
> #4 0x000000000041e397 in FlowForceReassemblyForQ (q=<value optimised out>) at flow-timeout.c:433
> s = 0x22d37930
> p = 0x43b53990
> reassemble_p = {src = {family = 0 '\000', address = {address_un_data32 = {0, 0, 0, 0}, address_un_data16 = {0, 0, 0, 0, 0, 0, 0, 0},
> address_un_data8 = '\000' <repeats 15 times>}}, dst = {family = 0 '\000', address = {address_un_data32 = {0, 0, 0, 0}, address_un_data16 = {
> 0, 0, 0, 0, 0, 0, 0, 0}, address_un_data8 = '\000' <repeats 15 times>}}, {sp = 0, type = 0 '\000'}, {dp = 0, code = 0 '\000'},
> proto = 0 '\000', recursion_level = 0 '\000', flags = 256, flowflags = 1 '\001', flow = 0x7f33b3c19dd0, ts = {tv_sec = 0, tv_usec = 0}, {
> pcap_v = {<No data fields>}}, datalink = 0, action = 0 '\000', pktvar = 0x0, ethh = 0x0, ip4h = 0x0, ip4vars = {comp_csum = 0, ip_src_u32 = 0,
> ip_dst_u32 = 0, ip_opts = {{type = 0 '\000', len = 0 '\000', data = 0x0} <repeats 40 times>}, ip_opt_cnt = 0 '\000', o_rr = 0x0, o_qs = 0x0,
> o_ts = 0x0, o_sec = 0x0, o_lsrr = 0x0, o_cipso = 0x0, o_sid = 0x0, o_ssrr = 0x0, o_rtralt = 0x0}, ip6h = 0x0, ip6vars = {
> ip_opts_len = 0 '\000', l4proto = 0 '\000'}, ip6eh = {ip6fh = 0x0, fh_offset = 0, ip6rh = 0x0, ip6ah = 0x0, ip6eh = 0x0, ip6dh1 = 0x0,
> ip6dh2 = 0x0, ip6hh = 0x0, ip6hh_opt_hao = {ip6hao_type = 0 '\000', ip6hao_len = 0 '\000', ip6hao_hoa = {__in6_u = {
> __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}}, ip6hh_opt_ra = {
> ip6ra_type = 0 '\000', ip6ra_len = 0 '\000', ip6ra_value = 0}, ip6hh_opt_jumbo = {ip6j_type = 0 '\000', ip6j_len = 0 '\000',
> ip6j_payload_len = 0}, ip6dh1_opt_hao = {ip6hao_type = 0 '\000', ip6hao_len = 0 '\000', ip6hao_hoa = {__in6_u = {
> __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}}, ip6dh1_opt_ra = {
> ip6ra_type = 0 '\000', ip6ra_len = 0 '\000', ip6ra_value = 0}, ip6dh1_opt_jumbo = {ip6j_type = 0 '\000', ip6j_len = 0 '\000',
> ip6j_payload_len = 0}, ip6dh2_opt_hao = {ip6hao_type = 0 '\000', ip6hao_len = 0 '\000', ip6hao_hoa = {__in6_u = {
> __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}}, ip6dh2_opt_ra = {
> ip6ra_type = 0 '\000', ip6ra_len = 0 '\000', ip6ra_value = 0}, ip6dh2_opt_jumbo = {ip6j_type = 0 '\000', ip6j_len = 0 '\000',
> ip6j_payload_len = 0}, ip6_exthdrs = {{type = 0 '\000', next = 0 '\000', len = 0 '\000', data = 0x0} <repeats 40 times>},
> ip6_exthdrs_cnt = 0 '\000'}, tcph = 0x0, tcpvars = {comp_csum = 0, tcp_opt_cnt = 0 '\000', tcp_opts = {{type = 0 '\000', len = 0 '\000',
> data = 0x0} <repeats 20 times>}, ts = 0x0, sack = 0x0, sackok = 0x0, ws = 0x0, mss = 0x0}, udph = 0x0, udpvars = {comp_csum = 0},
> sctph = 0x0, icmpv4h = 0x0, icmpv4vars = {comp_csum = 0, id = 0, seq = 0, mtu = 0, error_ptr = 0, emb_ipv4h = 0x0, emb_tcph = 0x0,
> emb_udph = 0x0, emb_icmpv4h = 0x0, emb_ip4_src = {s_addr = 0}, emb_ip4_dst = {s_addr = 0}, emb_ip4_hlen = 0 '\000', emb_ip4_proto = 0 '\000',
> emb_sport = 0, emb_dport = 0}, icmpv6h = 0x0, icmpv6vars = {comp_csum = 0, id = 0, seq = 0, mtu = 0, error_ptr = 0, emb_ipv6h = 0x0,
> emb_tcph = 0x0, emb_udph = 0x0, emb_icmpv6h = 0x0, emb_ip6_src = {0, 0, 0, 0}, emb_ip6_dst = {0, 0, 0, 0}, emb_ip6_proto_next = 0 '\000',
> emb_sport = 0, emb_dport = 0}, ppph = 0x0, pppoesh = 0x0, pppoedh = 0x0, greh = 0x0, vlanh = 0x0, payload = 0x0, payload_len = 0, pkt = 0x0,
> ext_pkt = 0x0, pktlen = 0, alerts = {cnt = 0, alerts = {{num = 0, order_id = 0, action = 0 '\000', flags = 0 '\000', alert_msg = 0x0,
> s = 0x0} <repeats 15 times>}, alert_msgs = 0x0}, pcap_cnt = 0, tunnel_mutex = {__data = {__lock = 0, __count = 0, __owner = 0,
> __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0},
> tunnel_rtv_cnt = 0, tunnel_tpr_cnt = 0, events = {cnt = 0 '\000', events = '\000' <repeats 14 times>}, next = 0x0, prev = 0x0, root = 0x0}
> f = 0x7f33b3c19dd0
> ssn = <value optimised out>
> client_ok = 1135950224
> server_ok = 1
The odd thing here is that src and dst aren't set; is that expected?
It's using about 6GB of memory at the moment, and we're handling 50-60K
packets per second, and logging about 250 HTTP requests per second.
Best Wishes,
Chris
On 26/09/11 20:22, Martin Holste wrote:
>> How much ram does this take?
> About 18 GB. I have 144, so this is no problem for me :)
>
>> You're running the git master which was last updated the 21st, so we
>> won't have later code that can be of influence.
>
> Ok, maybe it's running with the ac full that's done it, but I thought
> I had tried that before.
>
> Meanwhile, no joy on the segfaults:
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffd8ff9710 (LWP 9655)]
> 0x00007ffff67e4f19 in free () from /lib64/libc.so.6
> (gdb) bt
> #0 0x00007ffff67e4f19 in free () from /lib64/libc.so.6
> #1 0x00007ffff7bd884e in htp_tx_destroy (tx=0x7ffe200aef80) at
> htp_transaction.c:115
> #2 0x00007ffff7bd5e12 in htp_conn_destroy (conn=0x7fff5a0bfb90) at
> htp_connection.c:65
> #3 0x00007ffff7bd1112 in htp_connp_destroy_all (connp=0x7fffe746e4c0)
> at htp_connection_parser.c:197
> #4 0x000000000061f97a in HTPStateFree (state=<value optimized out>)
> at app-layer-htp.c:210
> #5 0x000000000061399b in AppLayerParserCleanupState
> (f=0x7ffea0459c60) at app-layer-parser.c:1240
> #6 0x0000000000437665 in FlowL7DataPtrFree (f=0x4007ffe200afef0) at flow.c:127
> #7 0x00000000004376cf in FlowClearMemory (f=0x7ffea0459c60,
> proto_map=<value optimized out>) at flow.c:1323
> #8 0x00000000004379a3 in FlowPrune (q=0x9441d0, ts=0x7fffd8ff8e60,
> try_cnt=0) at flow.c:372
> #9 0x000000000043a682 in FlowManagerThread (td=<value optimized out>)
> at flow-manager.c:170
> #10 0x00007ffff6f265f0 in start_thread () from /lib64/libpthread.so.0
> #11 0x00007ffff683f87d in clone () from /lib64/libc.so.6
> #12 0x0000000000000000 in ?? ()
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
--
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin, c.d.wakelin at reading.ac.uk
IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
More information about the Oisf-devel
mailing list