[tcpdump-workers] Query about running many, many, rules
How does PCAP handle running many rules? The current approach is to open one PCAP interface per rule. The end goal is to be able to run an live application with 1000's of simultaneous rules. The issue is that the current pcap_open_* interfaces are pretty limited. I could open one pcap structure per rule. But for my eventual application, that would be bad. It would mean copying each packet from kernel space to user space, once for every rule. At gigabit speeds, we quickly become overloaded. I have some hardware which does a quick pass over the packet, binning it into one of N buckets. It uses a bloom filter approach to reject many packets which won't match any rule. The output of the HW is a tag which says "looks like it matches rule X". I'd like to be able to take that tag, and use it to feed the packet into a PCAP filter which does more in-depth checks. However... I can't do this right now. There's pcap_open_live() for interfaces. There's pcap_open_offline() for files. There's no interface which says "here's a packet, run the rule against it". I think that this may be possible, without too much work. I propose the following: 1) Create a function which will allow packet injection into pcap: typedef int (*pcap_read_callback)(pcap_t *p, void *ctx, u_char *buffer, int bufsize) This function would be called through pcap_read(). It would use a user-supplied context to write data into the buffer, and return the number of bytes written. 2) create a function which would open a "pcap" structure, using the above callback: pcap_t *pcap_open_callback(pcap_read_callback callback, void *ctx, char *ebuf, size_t size); The application could then open 1000's of these structures, as it's just memory. It would then get packets from the kernal (once), along with the hardware tag. The tag would tell it which pcap_t to use for more detailed checks. I think that API is fairly simple. It seems to fit with the current philosophy of the pcap library and API. Comments? Alan DeKok. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Query about running many, many, rules
Guy Harris wrote: > Older versions of libpcap don't have that, Ah, that's why I couldn't find it. > Fill in a "struct pcap_pkthdr" (the filter doesn't look at the time stamp; > all it cares about is "caplen", which tells it how much packet data there is, > and "len", which tells it what the length is for the "len" value and the > "less" and "greater" tests), and pass that and a pointer to the raw packet > data to pcap_offline_filter(). > > To compile a filter, you could create a pcap_t with pcap_open_dead() (unless > you have a *really* old version of libpcap), passing it the appropriate DLT_ > value for the particular set of link-layer headers and possible metadata > headers your packets have (if they have more than one, you'll need multiple > filters and run the appropriate one for each packet) and a snapshot length > (all you're doing with the filter is getting a "yes or no" answer, so just > pass in a non-zero value). Thanks. Alan DeKok. ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] Query about running many, many, rules
Michael Richardson wrote:
> I'm unclear if you want to run many rules (filter1 OR filter2 OR filter3) on
> a single interface, or you want to run many pcap filters on different
> interfaces.
One interface. 1000's of filter rules.
> I think that Guy's answer suggesting that your pcap library was old should
> satify, but you mention hardware, and the current interface is really about
> either using the kernel interface ("live") or from a file ("dead"), while
> I think you want an in-memory interface.
Largely, yes. The system we're using doesn't support 1000's of filter
rules in the kernel for one pcap_t. So we're stuck doing the work in
user space.
Alan DeKok.
___
tcpdump-workers mailing list
tcpdump-workers@lists.tcpdump.org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
Re: [tcpdump-workers] OpenSSL deprecation warnings on MacOS
Denis Ovsienko wrote: > Below are some warnings specific to MacOS 10.8.5, i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00), would anybody be willing to troubleshoot that? > > ./print-esp.c: In function ‘esp_print_decrypt_buffer_by_ikev2’: > ./print-esp.c:133: warning: ‘EVP_CipherInit’ is deprecated (declared at /usr/include/openssl/evp.h:598) Apple wants applications to use the Apple crypto library instead of OpenSSL. I'm not sure that this is a good idea for pcap. The way to get rid of these warnings is the following code. See the FreeRADIUS src/include/build.h include file for more examples. /* * Macros for controlling warnings in GCC >= 4.2 and clang >= 2.8 */ #define DIAG_JOINSTR(x,y) XSTRINGIFY(x ## y) #define DIAG_DO_PRAGMA(x) _Pragma (#x) #if defined(__GNUC__) && ((__GNUC__ * 100) + __GNUC_MINOR__) >= 402 # define DIAG_PRAGMA(x) DIAG_DO_PRAGMA(GCC diagnostic x) # if ((__GNUC__ * 100) + __GNUC_MINOR__) >= 406 #define DIAG_OFF(x) DIAG_PRAGMA(push) DIAG_PRAGMA(ignored DIAG_JOINSTR(-W,x)) #define DIAG_ON(x) DIAG_PRAGMA(pop) # else #define DIAG_OFF(x) DIAG_PRAGMA(ignored DIAG_JOINSTR(-W,x)) #define DIAG_ON(x) DIAG_PRAGMA(warning DIAG_JOINSTR(-W,x)) # endif #elif defined(__clang__) && ((__clang_major__ * 100) + __clang_minor__ >= 208) # define DIAG_PRAGMA(x) DIAG_DO_PRAGMA(clang diagnostic x) # define DIAG_OFF(x) DIAG_PRAGMA(push) DIAG_PRAGMA(ignored DIAG_JOINSTR(-W,x)) # define DIAG_ON(x) DIAG_PRAGMA(pop) #else # define DIAG_OFF(x) # define DIAG_ON(x) #endif /* * For dealing with APIs which are only deprecated in OSX (like the * OpenSSL API) */ #ifdef __APPLE__ # define USES_APPLE_DEPRECATED_API DIAG_OFF(deprecated-declarations) # define USES_APPLE_RST DIAG_ON(deprecated-declarations) #else # define USES_APPLE_DEPRECATED_API # define USES_APPLE_RST #endif ___ tcpdump-workers mailing list tcpdump-workers@lists.tcpdump.org https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers
