[Oisf-devel] aligning data structures ...

[Victor Julien]

On 06/26/2011 07:07 PM, chetan loke wrote:
> Hello,
> 
> Some low hanging fruits if not done already -
> 
> I think it would be better if we align the _Packet struct on a
> cache-line boundary.
> This will avoid cache pollution amongst multiple threads.
> 
> So how do we do it in a platform(32/64 bit , different CPU models etc)
> agnostic way?
> 
> 1) Existing definition
> 
> typedef struct _Packet {
> ...
> } Packet;
> 
> 
> 2) New proposed definition:
> 
> #define suricata_pkt_alignment __attribute__((__aligned__(_CLS)))
> 
> 
> typedef struct _Packet {
> 
> } Packet suricata_pkt_alignment;
> 
> Also good to align counter etc structs.
> 
> Now the million dollar question - how do we get _CLS?
> 
> In the makefile add the following:
> 
> SURICATA_DEFINE_FLAGS=
> 
> ...
> 
> PLATFORM_CLS=$(shell getconf LEVEL1_DCACHE_LINESIZE)
> 
> SURICATA_DEFINE_FLAGS := -D_CLS=$(PLATFORM_CLS) $(SURICATA_DEFINE_FLAGS)
> 
> example - CFLAGS=-Wall -Werror -g -ggdb -c $(SURICATA_DEFINE_FLAGS)
> 
> 
> 3) I would also recommend using posix_memalign as opposed to a 'malloc'
>     while creating the packetpool during init-time in suricata.c:
> 
> posix_memalign((void
> **)(&queue_ptr),sysconf(_SC_PAGESIZE),(SIZE_OF_PACKET)*(max_pending_packets))
> 
> malloc all the packets once and avoid the for loop.

I like these suggestions.

> 4) Checking a global flag(even if page/cache aligned etc) to see if we
> need to stop the suricata engine kill's performance.
> One trick to get around this global flag is for the main thread to
> shutdown the interface[s] which is[are] receiving packets.

You mean just calling pcap_close (for pcap mode) from the main thread?
That would mean we get into "undefined behavior" territory for many
libraries I fear as this violates thread safety. Adding a lock to each
pcap operation just to support the shutdown trick is likely to be
counter productive.

> 
> If someone can please dump a short/quick(no fancy diagrams needed)
> call-flow on the website(or in the doc section) then it will be
> helpful for new comers to start contributing in less than a day, test
> all these low hanging fruits and submit patches.

In addition to Will's suggestion, have a look at this page. It's far
from complete, but it's a start :)

https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricata_Developers_Guide

Cheers,
Victor

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------