[Oisf-devel] how to setup the suricata for extracting the files from ftp protocol and save into disk

[Victor Julien]

On 21-03-18 12:03, zhangqs wrote:
> Thanks Victor, but i still confuse about how the data write into the
> disk after FTP parse, where the app-layer-ftp call the
> logFilestoreLogger? I only find the below relations:
> 
> LogFilestoreLogger--->LogFilestoreRegister--->OutputRegisterLoggers--->TmModuleLoggerRegister-->RegisterAllModules-->PostConfLoadedSetup-->Main()

The path is indirect:

If you look at flow-worker.c:FlowWorker you can see that each packet
goes through the same steps:

1. flow handle
2. tcp tracking/reassembly and app-layer (this includes FTP)
3. detect
4. outputs by a call to OutputLoggerLog.

The OutputLoggerLog then runs all output modules that are enabled by the
config.

Cheers,
Victor


> 
> 
> Best regards,
> 
> Kris
> 
> 
> 在 2018年03月21日 05:12, Victor Julien 写道:
>> On 19-03-18 10:34, zhangqs wrote:
>>> Hi guys,
>>>
>>> I have been struggling a few days to the function file extraction,  the
>>> reference doc is:
>>> http://suricata.readthedocs.io/en/latest/file-extraction/file-extraction.html?highlight=ftp.
>>>
>>> The protocol that I want to use is FTP.
>>> 1) Suricata version is latest that cloned from github.
>>> 2) I setup the suricata.yaml: file-store.enabled: yes
>>> 3) I create a rule file hello.rules, its content is:
>>>
>>> alert http any any -> any any (msg:"FILE store all"; filestore;
>>> sid:1; rev:1;)
>>>
>>> 4) ./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/
>>> 5) make && make install
>>>
>>> My testing pcap is in the attachment. but I cannot find the
>>> file(Music.mp3) was extracted and saved into the disk
>>> (/var/log/suricata/files/).
>>> Has anybody ever been successful about extraction FTP file into disk?
>>>
>>> And then I read the code, and cannot find which code is responsible for
>>> saving file into the disk?
>>> I guess the process is:
>>>
>>> FTPDataParseRequest-->FTPDataParse-->FileOpenFile|FileAppendData-->StreamingBuffer
>>>
>>>
>>> but the data is still in memory, where is save the StreamingBuffer into
>>> the disk?
>> It's stored by the filestore output module. This is defined in
>> src/log-filestore.c where the main logging function is LogFilestoreLogger
>>
>> The API this runs on top of is in output-filedata.c: OutputFiledataLog
>>
> 


-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------