[Oisf-devel] [COMMIT] OISF branch, master, updated. suricata-3.0.1RC1-18-gf836256

OISF Git noreply at openinfosecfoundation.org
Wed Mar 30 13:00:23 UTC 2016

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".

The branch, master has been updated
       via  f836256e20de1ede8d8fbb85e331278b8eb78eb1 (commit)
       via  31ed7042b55ddd85c3790346de8ab73055b0c47d (commit)
       via  13b87f5affa886d008caae4c54bce910f1c3c03d (commit)
       via  c37195c95fa57d8a9ce1b3bcf9c5e185b37fd5a3 (commit)
       via  c8e01a3d62c116ee5cad4d4a86c300d55d3781e9 (commit)
       via  30410e6900449a7f6d5798d5ac7bb86b7bd2b58d (commit)
       via  cae3ce9e1c141f47a31221c40a7f552af9fd6613 (commit)
       via  11099cfa42db2d122cee47e0131ffa558048ec14 (commit)
       via  eafd212661afbb57e07d237e7207ced2a44b5eee (commit)
       via  dd98bc353ea81c1626c4ab827a962140c42b7061 (commit)
       via  f78e990915803893c83dadb75b83d89cc12701e4 (commit)
       via  dc7d0c736b9f34dc008490578e63e125f0e8ece8 (commit)
       via  9c5ee76455361feaac3fc7207ef40175c485f7d7 (commit)
       via  f005310ddfcaf742c2ba8bb294c06014ac857abf (commit)
       via  a866d5d915bf472df535ed592218178ef5db0bc2 (commit)
       via  400fa5b9e5fab1bad4e78ab72e9ba8cda6ccd7e8 (commit)
       via  1e0b5eb529bda9a2d978a1242016e222f5e6f5c3 (commit)
       via  2e9279dd42840f5e2ba4e6ca969871f9f7b18ede (commit)
      from  0ac27e28abc735faded0ac71c0c48f9407cc3a5d (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit f836256e20de1ede8d8fbb85e331278b8eb78eb1
Author: Victor Julien <victor at inliniac.net>
Date:   Wed Mar 30 09:39:46 2016 +0200

    detect: fix small mem leak on duplicate sigs
    Direct leak of 80 byte(s) in 5 object(s) allocated from:
        #0 0x4c673b in __interceptor_malloc (/home/victor/dev/suricata/src/suricata+0x4c673b)
        #1 0xb7a425 in DetectEngineSignatureIsDuplicate /home/victor/dev/suricata/src/detect-parse.c:1715:10
        #2 0xb79390 in DetectEngineAppendSig /home/victor/dev/suricata/src/detect-parse.c:1836:19
        #3 0x86fe56 in DetectLoadSigFile /home/victor/dev/suricata/src/detect.c:357:15
        #4 0x815fee in ProcessSigFiles /home/victor/dev/suricata/src/detect.c:419:13
        #5 0x8139a8 in SigLoadSignatures /home/victor/dev/suricata/src/detect.c:499:15
        #6 0xfe435d in LoadSignatures /home/victor/dev/suricata/src/suricata.c:1979:9
        #7 0xfcd87e in main /home/victor/dev/suricata/src/suricata.c:2345:17
        #8 0x7fb66bf7cec4 in __libc_start_main /build/eglibc-3GlaMS/eglibc-2.19/csu/libc-start.c:287

commit 31ed7042b55ddd85c3790346de8ab73055b0c47d
Author: Victor Julien <victor at inliniac.net>
Date:   Tue Mar 29 11:44:00 2016 +0200

    hyperscan: add DrMemory suppressions

commit 13b87f5affa886d008caae4c54bce910f1c3c03d
Author: Justin Viiret <justin.viiret at intel.com>
Date:   Tue Mar 15 12:40:24 2016 +1100

    mpm: add Hyperscan integration
    This adds an MPM implementation that uses the Hyperscan regex engine
    library from Intel, accessible as the "hs" mpm-algo.

commit c37195c95fa57d8a9ce1b3bcf9c5e185b37fd5a3
Author: Justin Viiret <justin.viiret at intel.com>
Date:   Tue Mar 15 12:38:23 2016 +1100

    mpm: pass offset, depth args to add functions
    MpmAddPatternCI and MpmAddPatternCS had arguments for offset and depth,
    but these were not being passed in by the caller.

commit c8e01a3d62c116ee5cad4d4a86c300d55d3781e9
Author: Justin Viiret <justin.viiret at intel.com>
Date:   Tue Mar 29 09:32:26 2016 +1100

    util-hash-lookup3: Add hashlittle_safe() variant
    By default, hashlittle() will read off the end of the key, up to the
    next four-byte boundary, although the data beyond the end of the key
    doesn't affect the hash. This read causes uninitialized read warnings
    from Valgrind and Address Sanitizer.
    Here we add hashlittle_safe(), which avoids reading off the end of the
    buffer (using the code inside the VALGRIND-guarded block in the original
    hashlittle() implementation).

commit 30410e6900449a7f6d5798d5ac7bb86b7bd2b58d
Author: Victor Julien <victor at inliniac.net>
Date:   Wed Jun 25 16:16:30 2014 +0200

    capture: warn -i user if faster options are available
    If af-packet, netmap or pfring are available, users should use those
    for best performance.

commit cae3ce9e1c141f47a31221c40a7f552af9fd6613
Author: Victor Julien <victor at inliniac.net>
Date:   Sat Mar 26 19:56:00 2016 +0100

    netmap: implement capture inject packet flag

commit 11099cfa42db2d122cee47e0131ffa558048ec14
Author: Victor Julien <victor at inliniac.net>
Date:   Wed Mar 23 17:05:14 2016 +0100

    detect reload: generic packet injection for capture
    Capture methods that are non blocking will still not generate packets
    that go through the system if there is no traffic. Some maintenance
    tasks, like rule reloads rely on packets to complete.
    This patch introduces a new thread flag, THV_CAPTURE_INJECT_PKT, that
    instructs the capture thread to create a fake packet.
    The capture implementations can call the TmThreadsCaptureInjectPacket
    utility function either with the packet they already got from the pool
    or without a packet. In this case the util func will get it's own
    Implementations for pcap, AF_PACKET and PF_RING.

commit eafd212661afbb57e07d237e7207ced2a44b5eee
Author: Victor Julien <victor at inliniac.net>
Date:   Wed Mar 23 16:16:41 2016 +0100

    detect reload: call 'breakloop' on capture method
    Split wait loop into three steps:
    - first insert pseudo packets
    - 2nd nudge all capture threads to break out of their loop
    - third, wait for the detection thread contexts to be used
    Interupt capture more than once if needed
    Move packet injection into util func

commit dd98bc353ea81c1626c4ab827a962140c42b7061
Author: Victor Julien <victor at inliniac.net>
Date:   Thu Mar 24 11:51:49 2016 +0100

    signals: cleanup signal handling
    Simplify handling of USR2 signal. The SCLogInfo usage could lead to
    dead locks as the SCLog API can do many complicated things including
    memory allocations, syslog calls, libjansson message construction.
    If an existing malloc call was interupted, it could lead to the
    following dead lock:
     0  __lll_lock_wait_private () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:97
     1  0x0000003140c7d2df in _L_lock_10176 () from /lib64/libc.so.6
     2  0x0000003140c7ab83 in __libc_malloc (bytes=211543457408) at malloc.c:3655
     3  0x0000003140c80ec2 in __strdup (s=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ") at strdup.c:43
     4  0x000000000059dd4a in SCLogMessageGetBuffer (tval=0x7fff52b47360, color=1, type=SC_LOG_OP_TYPE_REGULAR, buffer=0x7fff52b47370 "", buffer_size=2048,
        log_format=0x259ca40 "[%i] %t - (%f:%l) <%d> (%n) -- ", log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
        message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:307
     5  0x000000000059e940 in SCLogMessage (log_level=SC_LOG_INFO, file=0x63dd00 "suricata.c", line=287, function=0x640f50 "SignalHandlerSigusr2StartingUp", error_code=SC_OK,
        message=0x7fff52b47bb0 "Live rule reload only possible after engine completely started.") at util-debug.c:549
     6  0x000000000057e374 in SignalHandlerSigusr2StartingUp (sig=12) at suricata.c:287
     7  <signal handler called>
     8  _int_malloc (av=0x3140f8fe80, bytes=<value optimized out>) at malloc.c:4751
     9  0x0000003140c7ab1c in __libc_malloc (bytes=296) at malloc.c:3657
     10 0x0000000000504d55 in FlowAlloc () at flow-util.c:60
     11 0x00000000004fd909 in FlowInitConfig (quiet=0 '\000') at flow.c:454
     12 0x0000000000584c8e in main (argc=6, argv=0x7fff52b4a3b8) at suricata.c:2300
    This patch simply sets a variable and lets the main loop act on that.

commit f78e990915803893c83dadb75b83d89cc12701e4
Author: Victor Julien <victor at inliniac.net>
Date:   Thu Mar 24 10:21:11 2016 +0100

    signals: handle INT/TERM signals in the main loop

commit dc7d0c736b9f34dc008490578e63e125f0e8ece8
Author: Victor Julien <victor at inliniac.net>
Date:   Thu Mar 24 09:31:19 2016 +0100

    pfring: use likely for fast path

commit 9c5ee76455361feaac3fc7207ef40175c485f7d7
Author: Victor Julien <victor at inliniac.net>
Date:   Sat Mar 26 12:05:50 2016 +0100

    tcp: fix unlikely NULL-ptr dereference
    If a TCP packet could not get a flow (flow engine out of flows/memory)
    and there were *only* TCP inspecting rules with the direction
    explicitly set to 'to_server', a NULL pointer deref could happen.
    PacketPatternSearchWithStreamCtx would fall through to the 'to_client'
    case which was not initialized.

commit f005310ddfcaf742c2ba8bb294c06014ac857abf
Author: Victor Julien <victor at inliniac.net>
Date:   Mon Sep 28 12:12:37 2015 +0200

    detect: add corner case mpm test

commit a866d5d915bf472df535ed592218178ef5db0bc2
Author: Victor Julien <victor at inliniac.net>
Date:   Thu Dec 17 10:34:17 2015 +0100

    tx logging: fix potential missed logging issue
    Wrong scope of proto_logged variable could potentially lead to
    incrementing logged tx id w/o actually being logged.
    Reported-By: Jason Ish

commit 400fa5b9e5fab1bad4e78ab72e9ba8cda6ccd7e8
Author: Victor Julien <victor at inliniac.net>
Date:   Sat Mar 26 11:28:30 2016 +0100

    stats-log: fix layout issue due to decoder stats

commit 1e0b5eb529bda9a2d978a1242016e222f5e6f5c3
Author: Victor Julien <victor at inliniac.net>
Date:   Sat Mar 26 10:59:52 2016 +0100

    autofp: print packet scheduler info only on autofp
    To avoid confusion about what runmode is active, only print autofp
    related scheduler information if autofp is the actual runmode.

commit 2e9279dd42840f5e2ba4e6ca969871f9f7b18ede
Author: cdwakelin <cwakelin at emergingthreats.net>
Date:   Wed Mar 23 17:13:55 2016 +0000

    autofp: add "ippair" scheduler
    Add "ippair" autofp scheduler to split traffic based on source and
    destination IP only (not ports).
    - This is useful when using the "xbits" feature to track events
      that occur between the same hosts but not necessarily the same
      flow (such as exploit kit landings/expoits/payloads)
    - The disadvantage is that traffic may be balanced very unevenly
      between threads if some host pairs are much more frequently seen
      than others, so it may be only practicable for sandbox or pcap
    - not tested for IPv6
    See https://redmine.openinfosecfoundation.org/issues/1661


Summary of changes:
 configure.ac            |   39 +
 qa/drmemory.suppress    |   31 +
 src/Makefile.am         |    1 +
 src/detect-content.c    |   13 +
 src/detect-engine-mpm.c |   37 +-
 src/detect-engine.c     |   85 +-
 src/detect-parse.c      |    2 +
 src/log-stats.c         |   16 +-
 src/output-tx.c         |    4 +-
 src/runmode-unittests.c |    4 +
 src/runmodes.c          |    6 +
 src/source-af-packet.c  |    4 +
 src/source-netmap.c     |    3 +
 src/source-pcap.c       |    2 +
 src/source-pfring.c     |   10 +-
 src/suricata.c          |  103 ++-
 src/suricata.h          |    5 -
 src/threadvars.h        |    5 +
 src/tm-threads.h        |   19 +
 src/tmqh-flow.c         |   71 +-
 src/tmqh-flow.h         |    2 +
 src/util-error.c        |    1 +
 src/util-error.h        |    1 +
 src/util-hash-lookup3.c |  174 ++++
 src/util-hash-lookup3.h |    5 +
 src/util-mpm-hs.c       | 2198 +++++++++++++++++++++++++++++++++++++++++++++++
 src/util-mpm-hs.h       |   76 ++
 src/util-mpm.c          |    4 +
 src/util-mpm.h          |    1 +
 29 files changed, 2843 insertions(+), 79 deletions(-)
 create mode 100644 src/util-mpm-hs.c
 create mode 100644 src/util-mpm-hs.h


More information about the Oisf-devel mailing list