[Libevent-users] event base: multi-thread producers, single thread consumer?
I've got a server with multiple threads, each with their own event base. Works wonderfully. I have a new situation that would be solved quite easily if I could have all threads (but one) push events onto the event base for the final thread. That final thread will process events as they come up. Reading through some of the additional libevent documentation, I'm wondering if I'm going to need to engage the various locking bits (evthread_set_id_callback, evthread_set_lock_callbacks, etc) to preserve the sanctity of that one event base? So far, I've been able to avoid any kind of locking in this server and would be quite reluctant to add any at this point. Thoughts? Thanks! - scott
Re: [Libevent-users] event base: multi-thread producers, single thread consumer?
On Fri, Apr 22, 2011 at 11:43 AM, Nick Mathewson wrote: > On Thu, Apr 21, 2011 at 6:29 PM, Scott Dorr > wrote: > > I've got a server with multiple threads, each with their own event base. > > Works wonderfully. I have a new situation that would be solved quite > > easily if I could have all threads (but one) push events onto the event > base > > for the final thread. That final thread will process events as they come > > up. > > Reading through some of the additional libevent documentation, I'm > wondering > > if I'm going to need to engage the various locking bits > > (evthread_set_id_callback, evthread_set_lock_callbacks, etc) to preserve > the > > sanctity of that one event base? > > Probably. If you want to do anything that affects an event_base from > multiple threads (and this includes touching events that are > associated with an event_base), you need to enable locking before you > do anything else with libevent, or construct any event_bases. > > Most people don't want to call the "evthread_set_*_callback()" > functions on their own: evthread_use_windows_threads() and > evthread_use_pthreads() are what you'll probably want, unless you need > to interface with an existing locking library that isn't one of those. > Yeah, after posting my question, I continued to do a bit of digging and came to the conclusion that probably all I'd need was evthread_use_pthreads() -- the other, more manual, evthread*callback() functions were in place in case you didn't use one of the already supported thread libraries (eg. pthreads). > > > So far, I've been able to avoid any kind of locking in this server and > would > > be quite reluctant to add any at this point. > > Well, if you want to communicate between threads without any locking, > you can't use changes to Libevent data structures to do it: Libevent's > data structures are only safe to access from multiple concurrent > threads if locking is enabled. > > You might be able to achieve what you want by some other means, like > using pipe() or socketpair() to create a communications channel > between threads. Have the thread that receives all the information > listen for EV_READ on the pipe/socketpair, and have other threads use > write to send it information. You'd either need to give each sending > thread its own pipe to write on, or you'd need to figure out a way to > prevent messages from getting mixed up -- maybe by using one-byte > messages, or datagram-based socketpairs, or some such. > For now, I'll stick with the locking. :( It's really only to handle an edge-condition that will rarely ever happen, so the chance of serializing behind a lock is greatly mitigated. I'll probably convert to some kind of pipe()/socketpair() solution when I've got more time to work on efficiency updates for edge cases. ;) Thanks for the response! - scott
[Libevent-users] Unitialized Memory Read when pushing >64 events at once
Purify complains about Unitialized Memory Reads when I try to push more than 64 events onto the queue at the same time. This is in a 64bit application running on RHEL linux. I'm currently using libevent-2.0.10-stable. epoll_dispatch triggers a set of of 4 UMRs for each event past the 64th that I push onto the event base. The first three are 1 byte reads on the stack (which it says is 20 bytes below the frame pointer in the epoll_dispatch function), and the fourth is a 4 byte read which is 388 bytes into a malloc'd block of 768 bytes. That malloc'd block was a realloc() that was called by epoll_dispatch->event_mm_realloc_ I don't have libevent instrumented, otherwise I'd provide more detail. Is this a known issue?
Re: [Libevent-users] Unitialized Memory Read when pushing >64 events at once
Sorry I didn't get a chance to fiddle with this until just now. The short
program below does generate uninitialized memory reads when executing the
printf(). If I add a memset() right after the events[200] declaration
(before the epoll_wait() call), the uninitialized memory issue goes away,
though it's clear the epoll underpinnings are actually filling in that
memory before we get back to our program space (gdb examination looks
clean).
Looks like a false positive to me at this point as well.
- scott
On Sun, May 29, 2011 at 9:34 PM, Nick Mathewson wrote:
> On Sun, May 29, 2011 at 9:25 PM, Nick Mathewson
> wrote:
> [...]
> > Below is a short program I tried to use to reproduce this, but
> > valgrind didn't tell me about any reads of uninitialized memory. Does
> > purify complain about the program below?
> >
>
> And here's a simpler program to check whether purify has the false
> positive that I suspect it might.
>
> If purify complains about this code using uninitialized RAM, I believe
> purify is wrong, or my understanding of epoll is somehow deficient.
> If purify doesn't complain about this code, then we are likely to have
> a genuine libevent bug on our hands.
>
> =
> #include
> #include
> #include
> #include
>
> int main(int c, char **v)
> {
>int epfd;
> int fd[128];
> int i;
>
> epfd = epoll_create(1000);
>
> for (i=0;i<128;++i) {
> struct epoll_event ctl;
> fd[i] = socket(AF_INET, SOCK_DGRAM, 0);
> if (fd[i]<0) {
> perror("socket");
> return 1;
> }
> memset(&ctl, 0, sizeof(ctl));
> ctl.data.fd = fd[i];
> ctl.events = EPOLLOUT;
> if (epoll_ctl(epfd, EPOLL_CTL_ADD, fd[i], &ctl) < 0) {
> perror("epoll_ctl");
> return 1;
> }
> }
>
> for (i = 0; i < 10; ++i) {
> int j, res;
> struct epoll_event events[200];
> res = epoll_wait(epfd, events, 200, 0);
> printf("%d\n", res);
> if (res < 0) break;
> for (j=0;jprintf(" - %d\n", events[j].data.fd);
> }
> }
>
> return 0;
> }
> =
>
> hth,
> --
> Nick
>
[Libevent-users] max number of supported events?
Is there an upper limit to the amount of outstanding events you can queue up on a base? (and if so, is there a difference between using an explicit base vs. the default base for this limit?) I'm trying to track down an issue where it feels like I'm losing events -- this is on a very heavily trafficked server. I just want to make sure I'm not butting up against an upper bound somewhere. :) Thanks! - jsd*** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] max number of supported events?
Is there an upper limit to the amount of outstanding events you can queue up on a base? (and if so, is there a difference between using an explicit base vs. the default base for this limit?) I'm trying to track down an issue where it feels like I'm losing events -- this is on a very heavily trafficked server. I just want to make sure I'm not butting up against an upper bound somewhere. :) Thanks! - jsd*** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] event_del and already 'in-flight' event triggers
When an event_del() is called on an evq, what happens to triggerings that were already 'in-flight' but haven't had their callbacks called yet. For example: read event R is active on fd 42, uses 'callback_func()' as its callback new packet P1 comes in on fd 42 new packet P2 comes in on fd 42 libevent calls callback_func() for P1 callback_func() calls event_del() on event R What happens with P2? Is callback_func() still going to be called for it? Thanks! - scott
Re: [Libevent-users] event_del and already 'in-flight' event triggers
On Jul 5, 2012, at 4:20 PM, Nick Mathewson wrote: > On Thu, Jul 5, 2012 at 3:34 PM, Scott Dorr wrote: >> When an event_del() is called on an evq, what happens to triggerings that >> were already 'in-flight' but haven't had their callbacks called yet. > > event_del() will make an active event inactive; if its callback was > scheduled but has not been run, it won't be run. This is the answer I was looking for. Thanks. :) >> For example: >> >> read event R is active on fd 42, uses 'callback_func()' as its callback >> new packet P1 comes in on fd 42 >> new packet P2 comes in on fd 42 >> libevent calls callback_func() for P1 >> callback_func() calls event_del() on event R > > In this case, callback_func would not even be scheduled until the next > time around the event loop. > >> What happens with P2? Is callback_func() still going to be called for it? > > Nope, not unless you call event_add() on R again. > > The easiest pattern here is to try to drain your socket completely on > the callback. Libevent doesn't promise you one callback per packet; > it promises you what you will get a callback so long as the socket is > active and readable. Yup, makes sense. I just needed to get my head around the specifics of what happens with that event_del(). Thanks for the response! - scott*** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] UDP Support
UDP works fine in libevent. I have several UDP based applications that run solely over UDP. You might have to run edge triggered to support UDP. I know I had to move to edge triggered, though I can't recall if that was for UDP, or for the OS I was using, or a combination. - scott Sent from my iPhone 5 On Dec 12, 2012, at 11:08 AM, John Dunn wrote: > Has there been any progress on UDP support in libevent? The last mention I > can find in the email list is from Dec 2010. It sounded like supporting > unconnected sockets was going to be difficult in 2.0.x - will any of the > changes in 2.1.x make this easier? > > I'm not familiar enough with libevent to be of any use developing such a > feature but would be more than happy to test. > > Thanks- > > John > > *** > To unsubscribe, send an e-mail to majord...@freehaven.net with > unsubscribe libevent-usersin the body. *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] Mixing ssl bufferevents with normal events
I have a libevent based server that used to use a custom ssl wrapper to handle ssl traffic. This server takes ssl traffic on one end and speaks non-crypto on the other end. For a variety of reasons, the switch was made to use the built in ssl bufferevents to handle the ssl portion. The non-crypto traffic continues to use 'regular' events. They use the same event base. However, the server is behaving as if the bufferevents are acting at a higher priority than the normal events. Under sustained load, the normal events are just not being pulled off the queue, or being pulled off so slowly that the inbound network buffer to the server gets filled (errno 11/resource temporarily unavailable seen from clients connecting from the localhost via UDS). Has anyone seen anything like this before? Is it considered bad form to mix bufferevents and non-bufferevents on the same event base? Thanks! - scott*** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
