[OISF/outreachy] [outreachy] Claiming Bug #2851

Vidushi Agrawal vidushi229 at gmail.com
Sat Mar 16 05:30:04 UTC 2019
Previous message (by thread): [OISF/outreachy] [outreachy] Claiming Bug #2851
Next message (by thread): [outreachy] Working on Task #2879
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
On Fri, Mar 15, 2019, 9:27 AM Shivani Bhardwaj <
sbhardwaj at openinfosecfoundation.org wrote:

> On Thu, Mar 14, 2019 at 3:35 PM Vidushi Agrawal <vidushi229 at gmail.com>
> wrote:
> >
> >
> >
> > On Thu, Mar 14, 2019 at 11:04 AM Shivani Bhardwaj <
> sbhardwaj at openinfosecfoundation.org> wrote:
> >>
> >> On Thu, Mar 14, 2019 at 12:42 AM Vidushi Agrawal <vidushi229 at gmail.com>
> wrote:
> >> >
> >> >
> >> >
> >> > On Wed, Mar 13, 2019 at 11:46 PM Shivani Bhardwaj <
> sbhardwaj at openinfosecfoundation.org> wrote:
> >> >>
> >> >> On Wed, Mar 13, 2019 at 4:18 PM Vidushi Agrawal <
> vidushi229 at gmail.com> wrote:
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Wed, Mar 13, 2019 at 2:05 PM Shivani Bhardwaj <
> sbhardwaj at openinfosecfoundation.org> wrote:
> >> >> >>
> >> >> >> Hi, Vidushi!
> >> >> >>
> >> >> >> On Wed, Mar 13, 2019 at 1:28 AM Vidushi Agrawal via Outreachy
> >> >> >> <outreachy at lists.openinfosecfoundation.org> wrote:
> >> >> >> >
> >> >> >> > Hi,
> >> >> >> > I'd like to start working on Bug #2851. If none's assigned to
> it yet, can it be assigned to me?
> >> >> >> Yes, please. Go ahead and start working on it. Do you mind making
> an
> >> >> >> account on redmine? I shall assign the task on tracker to you.
> >> >> >
> >> >> > I've made an account on redmine.
> >> >> >
> >> >> >> > Also, could you please guide me how to reproduce the issue
> after changing update.yaml file?
> >> >> >> All the explanation is assuming that you have a working
> installation
> >> >> >> of suricata-update.
> >> >> >
> >> >> > Yes, now I  do have a working installation of suricata-update.
> >> >> >
> >> >> >> See below.
> >> >> >> One good thing to do would be to use "-v" flag while running
> >> >> >> suricata-update. You'll get to know about a lot of settings and
> >> >> >> environment like where the different cofiguration and rule files
> >> >> >> reside which suricata-update is trying to read/write from/to.
> >> >> >>
> >> >> >> For this particular issue, in the log after running
> "./bin/suricata-update -v",
> >> >> >> Look for "Loading /path/to/update.yaml"
> >> >> >
> >> >> >
> >> >> > On running  "./bin/suricata-update -v", I get the loading message
> >> >> > 13/3/2019 -- 15:54:13 - <Info> -- Loading
> /usr/local/etc/suricata/suricata.yaml
> >> >> >
> >> >> > There's "Loading /path/to/suricata.yaml" not update.yaml
> >> >> >
> >> >> Could you please create one in /etc/suricata? See sample update.yaml
> >> >> here:
> https://suricata-update.readthedocs.io/en/latest/update.html#example-configuration-file-etc-suricata-update-yaml
> >> >
> >> >
> >> > I created one update.yaml in /etc/suricata and I checked the path in
> config.py. It still loads suricata.yaml instead of update.yaml. What could
> the possible fix be?
> >>
> >> Could you please give the entire log?
> >
> >
> > On running ./bin/suricata-update -v I get the following output:
> >
> > 14/3/2019 -- 15:27:01 - <Debug> -- This is suricata-update version
> 1.1.0dev0 (rev: None); Python: 2.7.15rc1 (default, Nov 12 2018, 14:31:15) -
> [GCC 7.3.0]
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value force ->
> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value verbose
> -> True
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value enable ->
> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value no-merge
> -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value version
> -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value
> dump-sample-configs -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value no-test
> -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value
> subcommand -> update
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value modify ->
> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value no-reload
> -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value no-ignore
> -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value disable
> -> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value etopen ->
> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value now ->
> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value url -> []
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value drop ->
> False
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting configuration value ignore ->
> []
> > 14/3/2019 -- 15:27:01 - <Debug> -- Looking for suricata in
> /home/vidushi/bin
> > 14/3/2019 -- 15:27:01 - <Debug> -- Looking for suricata in
> /home/vidushi/.local/bin
> > 14/3/2019 -- 15:27:01 - <Debug> -- Looking for suricata in
> /usr/local/sbin
> > 14/3/2019 -- 15:27:01 - <Debug> -- Looking for suricata in /usr/local/bin
> > 14/3/2019 -- 15:27:01 - <Debug> -- Found /usr/local/bin/suricata.
> > 14/3/2019 -- 15:27:01 - <Info> -- Using data-directory
> /usr/local/var/lib/suricata.
> > 14/3/2019 -- 15:27:01 - <Info> -- Using Suricata configuration
> /usr/local/etc/suricata/suricata.yaml
> > 14/3/2019 -- 15:27:01 - <Info> -- Found Suricata version 5.0.0-dev at
> /usr/local/bin/suricata.
> > 14/3/2019 -- 15:27:01 - <Info> -- Loading
> /usr/local/etc/suricata/suricata.yaml
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto dhcp
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto tftp
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto krb5
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto ntp
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto modbus
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto enip
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto dnp3
> > 14/3/2019 -- 15:27:01 - <Info> -- Disabling rules with proto nfs
> > 14/3/2019 -- 15:27:01 - <Info> -- No sources configured, will use
> Emerging Threats Open
> > 14/3/2019 -- 15:27:01 - <Info> -- Checking
> https://rules.emergingthreats.net/open/suricata-5.0.0/emerging.rules.tar.gz.md5
> .
> > 14/3/2019 -- 15:27:01 - <Debug> -- Setting HTTP User-Agent to
> Suricata-Update/1.1.0dev0 (OS: Linux; CPU: x86_64; Python: 2.7.15rc1; Dist:
> Ubuntu/18.04; Suricata: 5.0.0-dev)
> > 14/3/2019 -- 15:27:03 - <Debug> -- Local
> checksum=|71780cede70d4e28397745292843be1b|; remote
> checksum=|af5c3120d83827ba36a05d1c50a4fc9c|
> > 14/3/2019 -- 15:27:03 - <Info> -- Fetching
> https://rules.emergingthreats.net/open/suricata-5.0.0/emerging.rules.tar.gz
> .
> > 14/3/2019 -- 15:27:03 - <Debug> -- Setting HTTP User-Agent to
> Suricata-Update/1.1.0dev0 (OS: Linux; CPU: x86_64; Python: 2.7.15rc1; Dist:
> Ubuntu/18.04; Suricata: 5.0.0-dev)
> >  100% - 2333940/2333940
> > 14/3/2019 -- 15:27:25 - <Info> -- Done.
> > 14/3/2019 -- 15:27:25 - <Warning> -- No distribution rule directory
> found.
> > 14/3/2019 -- 15:27:25 - <Debug> -- Parsing
> rules/emerging-mobile_malware.rules.
> > 14/3/2019 -- 15:27:25 - <Debug> -- Parsing rules/emerging-icmp.rules.
> > 14/3/2019 -- 15:27:25 - <Debug> -- Parsing rules/tor.rules.
> > 14/3/2019 -- 15:27:25 - <Debug> -- Parsing rules/emerging-activex.rules.
> > 14/3/2019 -- 15:27:25 - <Debug> -- Parsing
> rules/emerging-icmp_info.rules.
> > 14/3/2019 -- 15:27:25 - <Debug> -- Parsing rules/emerging-policy.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-pop3.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing
> rules/emerging-shellcode.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing
> rules/emerging-attack_response.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-trojan.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-dns.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-telnet.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-scada.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-misc.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/dshield.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing rules/emerging-sql.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing
> rules/emerging-inappropriate.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing
> rules/emerging-web_server.rules.
> > 14/3/2019 -- 15:27:26 - <Debug> -- Parsing
> rules/emerging-web_specific_apps.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing
> rules/emerging-user_agents.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-exploit.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-malware.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-info.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/botcc.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-rpc.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/compromised.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-tftp.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-ftp.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing rules/emerging-netbios.rules.
> > 14/3/2019 -- 15:27:27 - <Debug> -- Parsing
> rules/emerging-current_events.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-p2p.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/drop.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-scan.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-games.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-imap.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-deleted.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-chat.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing
> rules/emerging-web_client.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/botcc.portgrouped.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-smtp.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-dos.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-snmp.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/ciarmy.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-worm.rules.
> > 14/3/2019 -- 15:27:28 - <Debug> -- Parsing rules/emerging-voip.rules.
> > 14/3/2019 -- 15:27:28 - <Info> -- Loaded 26887 rules.
> > 14/3/2019 -- 15:27:29 - <Info> -- Disabled 0 rules.
> > 14/3/2019 -- 15:27:29 - <Info> -- Enabled 0 rules.
> > 14/3/2019 -- 15:27:29 - <Info> -- Modified 0 rules.
> > 14/3/2019 -- 15:27:29 - <Info> -- Dropped 0 rules.
> > 14/3/2019 -- 15:27:29 - <Debug> -- Found 195 required flowbits.
> > 14/3/2019 -- 15:27:29 - <Debug> -- Found 187 rules to enable to for
> flowbit requirements
> > 14/3/2019 -- 15:27:29 - <Debug> -- Found 197 required flowbits.
> > 14/3/2019 -- 15:27:29 - <Debug> -- Found 1 rules to enable to for
> flowbit requirements
> > 14/3/2019 -- 15:27:29 - <Debug> -- Found 197 required flowbits.
> > 14/3/2019 -- 15:27:29 - <Debug> -- Found 0 rules to enable to for
> flowbit requirements
> > 14/3/2019 -- 15:27:29 - <Debug> -- All required rules enabled.
> > 14/3/2019 -- 15:27:29 - <Info> -- Enabled 188 rules for flowbit
> dependencies.
> > 14/3/2019 -- 15:27:29 - <Info> -- Backing up current rules.
> > 14/3/2019 -- 15:27:29 - <Debug> -- Recording existing file
> /usr/local/var/lib/suricata/rules/suricata.rules with hash
> 'e5319f78798e445dce8219fc470f9c5c'.
> > 14/3/2019 -- 15:27:32 - <Info> -- Writing rules to
> /usr/local/var/lib/suricata/rules/suricata.rules: total: 26887; enabled:
> 19533; added: 7; removed 3; modified: 1204
> > 14/3/2019 -- 15:27:33 - <Info> -- Testing with suricata -T.
> > 14/3/2019 -- 15:27:33 - <Debug> -- Running /usr/local/bin/suricata -T -l
> /tmp -c /usr/local/etc/suricata/suricata.yaml -S
> /usr/local/var/lib/suricata/rules/suricata.rules; env={'SC_LOG_FORMAT': '%t
> - <%d> -- ', 'SC_LOG_LEVEL': 'Warning', 'ASAN_OPTIONS': 'detect_leaks=0'}
> > 14/3/2019 -- 15:27:37 - <Info> -- Done.
> >
> hmm this is weird. I actually just tried the exact installation
> instructions and what I mentioned to you on a fresh Docker container,
> works out. Check the permissions of /etc/suricata.
> Anyway, for now, you could pass the path to update.yaml file with -c
> option.
> Read about it here:
> https://suricata-update.readthedocs.io/en/latest/update.html#cmdoption-c


I'd already checked the permissions for /etc/suricata. But for now I used
-c option to pass the path. I found the fix to the problem and I've made a
PR. Here's the link to it.

https://github.com/OISF/suricata-update/pull/116

Could you please review it and let me know if further changes are required.
Thanks!

>
> >>
> >>
> >> >>
> >> >>
> >> >>
> >> >> >> Then make the changes as described by the person (in the issue on
> >> >> >> redmine) in update.yaml file on the path you just discovered from
> the
> >> >> >> above mentioned log line.
> >> >> >> On looking closely at the log, you will see a line "Parsing
> >> >> >> rules/emerging-deleted.rules."
> >> >> >
> >> >> > Yes, I do see this line.
> >> >> >
> >> >> >> This is the problem that the person is defining. Despite defining
> in
> >> >> >> the configuration for update to ignore any rule files with
> >> >> >> "deleted.rules" in their name, a file with name *deleted.rules is
> >> >> >> still being processed.
> >> >> >
> >> >> > I did understand the problem. But on running suricata-update with
> -v flag, it is loading suricata.yaml not update.yaml. Am I doing it wrong
> or is it a problem with the installation?
> >> >> >
> >> >> >> Now, apply the changes you think are ideal for this case. Run
> >> >> >> suricata-update again with -v flag. Observe the output. The
> "Parsing
> >> >> >> rules/emerging-deleted.rules." should no longer be in the log.
> >> >> >>
> >> >> >> You could store the logs in both the cases and then run a diff on
> them
> >> >> >> to see if something changed as per your expectations.
> >> >> >> Does this help?
> >> >> >
> >> >> > Thank you
> >> >> >>
> >> >> >> > Thanks,
> >> >> >> > Vidushi
> >> >> >> > _______________________________________________
> >> >> >> > Outreachy mailing list
> >> >> >> > Outreachy at lists.openinfosecfoundation.org
> >> >> >> > https://lists.openinfosecfoundation.org/listinfo/outreachy
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Shivani
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Shivani
> >>
> >>
> >>
> >> --
> >> Shivani
>
>
>
> --
> Shivani
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/outreachy/attachments/20190316/5bbd5989/attachment-0001.html>
Previous message (by thread): [OISF/outreachy] [outreachy] Claiming Bug #2851
Next message (by thread): [outreachy] Working on Task #2879
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the Outreachy mailing list