[Oisf-devel] [COMMIT] OISF annotated tag, suricata-1.3beta1, created. suricata-1.3beta1

noreply at openinfosecfoundation.org noreply at openinfosecfoundation.org
Wed Apr 4 15:52:23 UTC 2012

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".

The annotated tag, suricata-1.3beta1 has been created
        at  56c630a7aac5342113b1f2e0f892166584374aa5 (tag)
   tagging  fbe0206c36b6bf4fd83d0c812aebb4f2a785aaa2 (commit)
  replaces  suricata-1.2.1
 tagged by  Victor Julien
        on  Wed Apr 4 17:52:02 2012 +0200

- Log -----------------------------------------------------------------
Tag 1.3beta1 release
Version: GnuPG v1.4.11 (GNU/Linux)


Anoop Saldanha (85):
      bug #403 - fix setting ip proto for packets
      bug #403 - fix setting ip proto for ipv6 packets
      Fix csum validation functions to not carry out csum calculation if respective headers are not present
      Set the packet protocol only if it can parsed without error
      bug #403 - add unittests
      Use SigInitReal() instead of SigInit() in raw uri tests. This should show that we have unittests failing, thus highlighting bug 411. The next commit is the fix for this bug
      bug #405 - fix bug where raw uri inspection sigs were not treated as stateful sigs
      Add function declaration for SigInitReal
      bug #412 - Unify SigInit() and SigInitReal(). Remove any use of SigInitReal()
      bug #412 - Remove the commented out SigInitReal()
      bug #412 - rebase commit. Remove the previous references to SigInitReal() with SigInit()
      bug #411 - don't modify within/distance at setup time
      bug #411 - fix failing unittest
      Support for new MPM ac-bs added
      treate ac-bs auto as single context
      Fix bug in ac-bs search function
      support splitting mpm ctxs based on direction v2
      if a signature is non-tcp, it's always a packet sig
      code cleanup over last 2 commits
      fix debug messages that have references to the old mpm contexts
      fix compilation error for the new http response header mpm feature
      Support http stat msg detection engine, fast pattern(mpm engine included). Fix http stat msg setup function. Fix pcre option for stat msg keyword
      Support http stat code detection engine, fast pattern(mpm engine included). Fix http stat code setup function. Fix pcre option for stat msg keyword. With this the pcre options for server_body is Q, for stat_msg is Y and for stat_code is S
      fast pattern unittests added for http server body
      fix seg fault due to wrong sm list access in hscd
      rebase commit for hscd and hsmd patches
      raw urilen inspection moves to raw uri list. Won't make any difference wrt inspection
      unify payload detection engines + fix other bugs in pcre init
      remove all old content inspection engines and references to them. We have cleaned the entire content inspection phase and improved alert accuracy
      remove det_ctx->payload_offset and use det_ctx->buffer_offset. Update hscd and hsmd to use the new generic content inspection engine
      feature #414 - support listing supported keywords. Remove support for dummy keywords __address__, __proto__, __port__. Remove support for recursive keyword and all references to it
      delete detect-recursive.[ch]
      DetectPatternGetId() cleanup. Remove separate search element creation for uricontent. We don't need this now since we have unified content structures for content and uricontent
      Use sm_list to differentiate between different content types while retrieving pattern ids instead of sm_type
      code cleanup - remove DetectUricontentGetLastPattern
      code cleanup - remove SigMatchGetLastPattern
      code cleanup - remove DetectContentHasPrevSMPattern
      code cleanup - remove DetectContentFindNextApplicableSM
      code cleanup - remove DetectContentGetLastPattern. Replace it with SigMatchGetLastSMFromLists
      code cleanup - replace SigMatchGetLastSM with SigMatchGetLastSMFromLists
      code cleanup - replace SigMatchAppendAppLayer with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendUricontent with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendPayload with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendDcePayload with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendTag with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendPacket with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendPostMatch with SigMatchAppendSMToList
      code cleanup - replace SigMatchAppendThreshold with SigMatchAppendSMToList
      code cleanup. Remove unused functions
      All uricontent modified patterns now are DETECT_CONTENT and not DETECT_URICONTENT. Step towards unifying all content based patterns. Makes way for easier management of patterns
      All http_client_body modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_CLIENT_BODY
      All http_server_body modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_SERVER_BODY
      All http_http_header modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_HEADER
      All http_http_raw_header modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_RAW_HEADER
      All http_http_method modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_METHOD
      All http_http_cookie modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_COOKIE
      All http_http_raw_uri modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_RAW_URI
      All http_http_stat_msg modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_STAT_MSG
      All http_http_stat_code modified patterns now are DETECT_CONTENT and not DETECT_AL_HTTP_STAT_CODE. Also remove dummy match/free functions for stat code and stat msg
      code cleanup
      Add BUG_ON to avoid overruning AppLayerDetectDirection map array
      Add new command line option --list-app-layer-protocols to list supported app layer protocols in sigs
      support for custom flow qhandlers - round robin support added
      support flow q handler schedulers active_flows and active_packets. Support new yaml option autofp_scheduler. Support for printing q handler stats as well
      Enable unittests for flow q handler
      neaten flow q handler code
      Support freeing flow q handler out ctx. Adapt unittests to use the same
      Adapt flow tmqh counters to be atomic vars. Remove support for active flows q handler. Introduce SC_ATOMIC_SET
      Introduce the address hash based flow q handler
      update all spm algos to use 16 bit pattern lengths. Should compress a lot of tables
      init every new pf instance in log pcap
      don't return TM failure on failing to remove log file
      restructure log pcap to use a different setup, which is resilient to thread failure/restarts
      clean log pcap
      b2g cuda up, compiling and running
      cuda pb tm should be in a thread of its own + pkt_acq should be as free as possible
      fix compiler warnings
      cleanup junk code in flow qh
      cleanup killing threads. As a consequence fixes invalid read/writes in tmqh flow
      restructure disabling receive threads. Introduce new flag to indicate that threads have finised running
      fix cppcheck analyzer warnings - bug 439
      restructure http logging to use fine grained locking
      update util-print.c to use snprintf
      provide generic macro to buffer data using snprintf
      update util-print.c to use new print macro

Eileen Donlon (14):
      Cleaned up some error messages for detect distance and offset.
      fixed relative handling for pcre cookie and method
      reject invalid combinations of pcre modifiers
      reject mixed relative and non-relative keywords
      fix invalid unittests with mixed content modifiers
      fix more invalid content unittests
      add null checks to fix bugs in StreamTcpTest23
      reject http_client_body with inconsistent flow dir
      added null checks for init_hash to all ac mpms
      reject rules with duplicate content modifiers
      reject rules with an invalid ttl range
      reject rules with invalid hex digits in content
      fix misleading comment

Eric Leblond (23):
      Add install-conf command to build system.
      del rules file deleted
      Improve output
      Remove autogen.sh generated files.
      Fix PCRE-JIT message
      Add sexy information messages to configure output.
      build: enable af-packet by default
      af-packet: mmap support
      decode: add PacketSetData funtion
      af-packet: Implement zero copy
      decode ASN.1: Factorize value reading
      TLS app layer: Add tls.issuerdn keyword.
      tls app layer: handle negation on subject and issuerdn.
      tls app layer: add missing free
      TLS parser: modify OCTETSTRING
      TLS parser: add sanity check
      TLS parser: add sanity checks on loop
      tls-handshake: DecodeAsn1BuildValue should return -1 for error
      tls-handshake: Add some missing free in error handling.
      tls-handshake: add sanity checks.
      Improve check of min requirement for AF_PACKET.
      af_packet: misc improvements.
      pcap: fix "work by luck" code.

Jason Ish (3):
      Implement single, autofp and workers run modes for DAG interfaces. Includes multiple interface support.
      Apply changes recommended by Stephen Donnely of Endace: - Skip pad records. - Don't log error on EGAIN, just try again. - Skip over extension headers. - Check we have the full packet (skip partial packets) - Remove obsolete rlen check. Also remove max_pending_packets to process more packets per iteration.
      Update the ERF file runmodes to support autofp and single.

Martin Holste (4):
      Added contrib folder with file_processor utility which is a plugin framework for reading the files-json.log and processing and taking action based on the files observed.
      Added some installation instructions to file_processor REAMDE.
      Added license.
      Added Shadowserver plugin.

Nikolay Denev (7):
      Convert config entries using underscores to dashes and emit deprecation warnings.
      Remove the underscored "sguil_base_dir" compatibility option.
      Do not use underscored config vars internally.
      Fix some warning message still using underscored config vars.
      Convert underscores to dashes in thread affinity type names.
      Consistently use dashes instead of underscores in the sample config file.
      Consistently use dashes instead of underscores in the sample config file.

Pierre Chifflier (11):
      Add ASN.1 parser for X509 certificates (in DER format)
      TLS handshake: decode the SERVER_CERTIFICATE message
      TLS handshake: get TLS ciphersuite and compression
      TLS app layer: fix number of bytes processed on SERVER_CERTIFICATE message.
      TLS app layer: rewrite decoder to handle multiple messages in records
      TLS keywords: fix match regex (remove extra space)
      TLS parser: add handing of UTF8STRING
      TLS: replace SigMatchAppendAppLayer with SigMatchAppendSMToList
      Add TLS decode events
      TLS app layer: misc fixes, reorder some fields to same memory
      TLS: add variable to store the error code in the decoder

Victor Julien (113):
      Move threshold to it's own sig match list.
      Make code default for pcre match limit match the suricata.yaml default.
      Add http-events.rules and smtp-events.rules to default suricata.yaml.
      Convert missing coredump config to debug.
      1.3 branch has opened
      Add files.rules to the dist.
      Make 'make check' happy in a ipproto unittest.
      Fix unittest missing a flow direction in the rule.
      Remove unused definitions in pcre code.
      Disable unittest that fails without libnet support.
      Initial Napatech support by Randy Caldejon / nPulse.
      Fix compilation without napatech tech support enabled.
      Napatech code formatting fixes.
      Minor layout fixes.
      Add atomics to ticks unittests.
      Fix a FP with negated filemagic inspection.
      Allow other yaml files to be included in the main yaml.
      Fix path handling for including rule files on win32.
      Initial on the fly MD5 calculation for extracted files using libnss.
      In PrintRawUriFp, consider " unprintable.
      Add line based log file to log-file module that logs each stored file's meta data in json records.
      Make sure that if not built against libnss, we still compile. Only no md5 for you then\!
      Add referer header to .meta and json file logs.
      Add a print function specially for json output that escapes all characters json requires to be escaped.
      Fix UtilMiscParseSizeStringTest01 unittest on 32 bit.
      file-inspection: use filename= value from Content-Disposition where available to determine the filename in GET requests.
      file-inspection: support POST requests that do not use multipart.
      Improve config details overview at the end of configure.
      Config should be set up in sysconfdir/suricata. Add reference to oinkmaster guide.
      Fix NULL dereference in PacketPatternSearchWithStreamCtx code.
      Misc afpacket changes.
      Fix compiler warning and silence complaining unittests.
      Fix locking error in filestore handling. Add debug validate check for asserting a flow is locked.
      Add more flow lock assertions to the debug validation code.
      file-inspection: split 'file' output module into file-store and file-log. Store stores files. Log logs json records.
      Add line number to warning about mangled yaml parameters. Limit number of warnings to 10.
      Various fixes and improvements based on feedback by Coverity analyzer.
      Various improvements to error handling found by Coverity.
      Another batch of minor fixed for issues found by Coverity.
      flow: Refactor how FlowPrune deals with forced timeouts, improving locking logic.
      file extract: improve multipart parsing and set events on some error conditions.
      Move PACKET_RECYCLE outside of flow lock in FlowForceReassemblyForQ as it confuses static code checkers.
      Fix minor fgetc issue.
      Fix compilation with profiling enabled. Minor unittest fixes.
      Improve http filename parsing.
      Fix UTHBuildFlow setup using wrong address.
      Fix minor memleak in case af-packet init fails.
      Fix issue discovered by Anoop. Passing u32 ptr to a size_t can caused badness.
      Fix json output typo.
      Do not assume the include dir for nspr to be nspr. On F16 it's nspr4.
      Do not assume the include dir for nss to be nss. On F16 it's nss3.
      Add libnss/libnspr support output to configure. Clean up configure.in.
      Fix broken unittest.
      flow engine: improve scalability
      Remove trailing zero's from some counters output.
      Misc fixes.
      Implement stream memcap enforcements using atomics instead of spinlocked counters.
      Add way to profile mutex/spin locks per thread module.
      Fix invalid declaration of enable_nss and enable_nspr in configure.in.
      Add atomic stack implementation. Convert flow spare queue to use this stack. Remove now unused flow-queue code.
      Undo changes from 88b8f15663076560b2237e6d8b8cae7e23d92bb6. Atomic stack implementation had a-b-a problem.
      Introduce host table, make tag use it
      Move over src and dst thresholding to use host table. Fix a bug in threshold 'both' handling.
      Fix 2 compilation issues.
      Small http.log improvement: bail out early if there is nothing to log. Make output locking more fine grained.
      Make sure stream debug code is only used in debug mode.
      Profile pcap file callback.
      profiling: add per lock location profiling
      Clean up error message.
      Make list-app-layer-protos option name match the help explanation. Make sure it works w/o passing a config.
      Minor flowq updates.
      Fix bug in app layer event handling causing http event rules to fail loading.
      flow: fix atomic var not being initialized and destroyed.
      profiling: fix lock profiling int print issue.
      Minor error message cleanups
      Silence ac-gfbs debug message.
      Bail out early if we're in http tunnel mode.
      libhtp: update to sync with upstream 0.2.x
      http: 'HTTP Host header ambiguous' after libhtp update. It now fires if hostname is present both in URL and Host header and the 2 are not equal.
      Various small flow and host table fixes.
      Make 'autofp' the default runmode. Increase default max-pending-packets to 1024. Move some advanced and uncommonly changed settings down in the stock suricata.yaml. Closes #433.
      Fix typo in spm prototype declaration.
      Enforce memcap limit before allocating hash table in host and flow engines.
      Add host section to stock yaml.
      file magic: don't disable inspecting magic for both directions if files in only one direction don't need magic.
      flow: create a flow lock macro API, implement it for mutex and rwlocks. Mutex remains the default.
      hash: add lookup3.c by Bob Jenkins
      flow: make flow use lookup3.c hashing algorithm. Improves hash table distribution.
      host: convert host hash to use lookup3.c
      Fix compilation of atomic api spinlocked fallback code.
      Fix __WORDSIZE redeclaration warning on Windows builds.
      Fix misc issues picked up by coccinelle.
      Fix CUDA build from a release tarball.
      nfq: switch locking code to macro's to lock profiling can track the exact lock locations.
      Fix some minor clang scan-build warnings.
      Fix parsing of tcp-pkt and tcp-stream sigs, add unittest.
      ipv6: fix detection engine using the originals IPv6 header's nxt hdr value instead of the upper layer one.
      ipv6: properly deal with packets containing a FH header that has offset 0 and no more frags flag set.
      ipv6: fix routing header parsing leading to rejection of valid packets.
      host: convert use_cnt to a atomic var (like in flow).
      flow: add missing unlocks for rare error condition at flow shut down.
      Minor unittest fixes to make Coverity happy.
      Small compile fix.
      Minor optimizations to unified2 and fast.log.
      Make fast.log use finer grained locking, move protocol lookup outside of the lock.
      Minor stream optimization.
      Fix error in per packet detection engine profiling.
      Apply http.log formatting fix by Chris Wakelin.
      Fix error in proto handling for ipv6 in fast.log.
      ipv6: make sure we pass the defragged packet from the ipv6 layer to the decoder.
      defrag: don't increment recursion level for reassembled packets. Fixes defragged packets not seeing the same flow.
      Fix minor compiler warning.
      Update Changelog for 1.3beta1

Xavier Lange (4):
      Added conf_test flag and behavior
      Do not spawn threads for conf test
      Make conf_test local. Simplify if/else to if.
      Include conf_test in special cases for unset RUNMODE



More information about the Oisf-devel mailing list