Re: [Libevent-users] Win64 compile issues
On Mon, Nov 09, 2009 at 05:19:26PM -0500, Patrick Galbraith wrote: > Hi all, > > I'm wondering if there's anyone out there who has had luck with mingw > compiling libevent on Windows 64-bit? I have had no problem on win32, > but I have this error on a windows 64-bit (windows 2008) in the compile > right now: > In case anybody is tracking this, or wants to help getting Libevent to build with 64-bit windows, it's on the bugtracker as https://sourceforge.net/tracker/?func=detail&atid=461322&aid=2895423&group_id=50884 yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] Migrating from Subversion to Git
Libevent has migrated its version control system from Subversion to Git. This will affect developers, as well as anybody who has been tracking development on the Subversion repository. My sense is that many developers are using Git already; at least half of the patches I get seem to be generated by the git-svn wrapper, which I've been using myself for a while too. We're making the shift so we can do a better job of testing out and sharing big patch series in separate branches before we merge them into the main repository. (Even with just a few developers, it's a pain passing revised patches back and forth without committing them, but committing unfinished patches is risky business for stability.) Anyway, the read-only URL for our official public git repository is: git://levent.git.sourceforge.net/gitroot/levent/libevent So to fetch a copy, you'd run git clone git://levent.git.sourceforge.net/gitroot/levent/libevent If you want email notification when a patch is merged, you can still subscribe to the levent-commits mailing list: https://lists.sourceforge.net/lists/listinfo/levent-commits If you're new to Git, there is a pretty good tutorial and a pretty good user manual on the main git site (http://git-scm.org/): http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html http://www.kernel.org/pub/software/scm/git/docs/user-manual.html Some people like the "Everyday Git" quick-reference: http://www.kernel.org/pub/software/scm/git/docs/everyday.html Git for SVN users: http://git.or.cz/course/svn.html This may also be a good time to consider the parable of the bikeshed: http://www.bikeshed.com/;) -- Nick Mathewson *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] [PATCH] evdns lock violation while thread is used
On Wed, Nov 25, 2009 at 03:30:08PM +0800, Zhuang Yuyao wrote: > evdns contains a bug related to thread lock. > > enable thread lock by evthread_use_pthreads() will cause successive > evdns_base_resolve_ipv4() (and other resolve functions i think) to > hang on EVDNS_LOCK(base) after one or several successful call to > evdns_base_resolve_ipv4(). > > > the patch attached with this mail solved this problem on my machine. I've merged this to the master branch; thanks! I've also applied code for a 'lock debugging mode' so we can try to catch problems like this more reliably in the future. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Valgrind patch
On Fri, Nov 27, 2009 at 05:56:25PM -0800, William Ahern wrote: > Valgrind complains on startup because kq_init passes to kevent only a > partially initialized structure. The code doesn't expect kevent to look at > .fflags, .udata, or .data, I suppose, because it merely tickles the > kernel looking for an error response. But perhaps that's unwarranted > chuminess (notwithstanding that it's checking for an OS X bug), and needless > noise nonetheless. > > Patch against 1.4.13-stable. Applied to the patches-1.4 and master branches. Thanks! -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Threading concerns
On Sun, Nov 29, 2009 at 11:18:19PM -0500, Donghua Xu wrote: > Hi, > > Sorry if this is a stupid question. But I have not found a clear > answer in the documentations. I am just beginning to look into using > libevent for our project. We will need to schedule millions of timed > events, and thousands of them may be triggered at the same time. So we > have to use thread pool to handle all these events. Let's say we have > 10 threads running at the same time, so how do I know which thread > will the call-back function be running when an event is triggered? Is > it always the thread that calls event_base_set() for the event? Or > could it be some other thread? When you schedule an event, it will be run in the thread that is running the loop for that event base. So if you say event_assign(event2, base99, ...); And later, you say event_base_dispatch(base99); Then the thread that called event_base_dispatch() for base99 will be the one to run event2's callback. Note that Libevent 1.4.x does not provide thread-safety for access to an event_base from another thread: if you want to be able to add or remove an event from an event_base that another thread is using, please check out the 2.0.x series. Note also that if you have millions of timer events scheduled (wow!), Libevent's default min-heap timer implementation might not be as fast as you want. The "common timeout" logic added in 2.0.3-alpha might be a viable choice for this kind of scenario, if it applies to your actual use-case. Right now Libevent has no built-in thread-pool support. Generally speaking, if you want a thread-pool style structure with today's Libevent, I think you have two basic options: - Run an event_base loop in each thread, and try to assign events to bases fairly and/or - Have the callback from each work-intensive event assign the work to a worker thread rather than running it immediately. though other people can probably add more info about what has worked for them. (I'd like to get better thread pool support in a future version, but as things stand, it is probably too big a feature for the 2.0.x series. Maybe 2.1.x, if somebody wants to write it.) yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] get next timeout from libevent
On Tue, Nov 24, 2009 at 03:31:18PM +0100, Filippo Giunchedi wrote:
[rewrapped]
> Hi,
>
> we are trying to build a library which uses libevent but also has
> the capability to be integrated into an application's main loop.
>
> Our solution so far includes three functions:
> - get the list of file descriptions handled by our library
> (IOW the list of fds in libevent)
> - register a callback to be called whenever a new fd is added
> - perform a step (i.e. basically call libevent loop once)
>
> This way the application can poll/select on our lib fds and then make a step
> whenever necessary.
>
> So far so good (please correct me if there's something missing :)).
Hm. I'm not sure this is such a great design. If the application is
*already* calling poll()/select() for you, then it's redundant to have
Libevent do it as well. Worse, if the application is actually using
poll or select, then you'll lose all the performance benefits that
you'd get from Libevent's use of fast backends like kqueue, epoll, or
evport.
Also, how are you getting "the list of fds in Libevent"? FWICT, there
isn't a public interface that exposes that.
> At this point though the application won't be able to get the
> timeout for the next event because AFAICS libevent's timeout_next()
> isn't supposed to be called from the outside, is this correct?
Right.
I wouldn't object to adding one, if there turn out to be good uses for
it, or as part of a general extension scheme.
> Also, what would be the correct approach to build a library upon
> libevent which can be then integrated into another main loop?
This are just my initial thoughts, so I don't promise they're very
good ones:
1. If all your library wants from Libevent is read/write operations on
fds and timeout operations, and you want to give your fds to an
application main loop that handle select()/poll()/kqueue()/epoll()
itself, then I wouldn't tie your library so tightly to Libevent at
all. Instead, I'd design the library so you could register
callbacks for the following situations:
- The library starts caring about read/write on an fd.
- The library stops caring about read/write on an fd.
- The library wants to register a timeout.
Then, you'd want the application's main loop to call you whenever
an fd should make progress, and whenever a timeout is ready.
This way, you wouldn't be tied to any specific main loop. Of
course, you'd probably want to include a libevent-implementation of
these functions in a separate library, so that applications that
*didn't* want to do their own select()/poll() would just be able to
use the Libevent-based default.
Of course, with this approach, you're not actually using the
Libevent main loop at all, so you're stuck with the performance
characteristics of whatever the application's main loop does.
1a. Alternatively, if you want to hack on Libevent some, you could
give Libevent the ability to build an event_base that used some
other event loop API as its backend. This could be tricky, but it
might be a fun project, and you'd be able to use all the features
of Libevent, either with Libevent's main event loop, or an
application-supplied event loop.
With this approach you're not actually using the Libevent main
loop at all either, so the same caveats apply as above.
2. You might be able to get away with just running all your Libevent
code in a separate thread, so that it doesn't need to interfere
with the application main loop at all. This way Libevent can be as
fast as it can, while the application can keep its loop. Of
course, you'd need a good way to communicate back from the Libevent
thread to the application thread(s).
2a. You could probably add a feature to Libevent so that the main
event loop could run in one thread, but it wouldn't actually run
callbacks itself: instead, it would mark events active and signal
the main thread to wake up by whatever means you like, and then
the main thread could call another function.
I'd probably do this with two new event_base_loop flags: one to
say "only activate events; do not process active events", and
another saying "run all the active events; do not actually do an
event loop". Then your Libevent thread would say something like:
while(1) {
event_base_loop(base, EVLOOP_ACTIVATE_ONLY);
alert_application();
}
And your application's main loop would just need to be set up so
that it ran "event_base_loop(base, EVLOOP_PROCESSS_ONLY)" whenever
the Libevent thread called 'alert_application()'.
Of course, you'd need to do this based on Libevent 2.0.x, since
earlier versions don't support manipulating an event_base from
outside the thread it's running in.
This is my favorite approach so far, since it lets you actually
use Libevent's event loop (for performance) and the application's
eve
Re: [Libevent-users] question about poll()
On Wed, Nov 25, 2009 at 02:44:37PM -0500, Patrick Galbraith wrote: > Hi there all! > > I'm working on trying to port libmemcached to Windows. It is not an easy > task! I've just found that libmemcached uses poll() and this is a bit of > a road-block! What do you guys do to get cross-platform support? How do > you transparently use poll() or other mechanisms to provide the > functionality that is required depending on platform? I'm not sure what exactly you're asking; I'll answer two possible variants of your question. If you meant something else, my apologies. If you mean, "how do Libevent users transparently use poll() or other mechanisms?" then the answer is "by using Libevent": Libevent knows about lots of different platform-specific poll()-like things, and tries to use whichever is fastest among the available options. You can probably find examples of how to use Libevent by looking in the regular memcached source. If you mean, "how do Libevent users transparently use poll() or other mechanisms?" then the answer is, "by having a separate implementation for each backend mechanism, compiling as many as will compile on a given platform, and selecting whichever one of the compiled ones works best at runtime." The responsibility of each backend is basically: - Know how to start caring about an fd for reading/writing - Know how to stop caring about an fd for reading/writing - Know how to wait up to a given number of seconds for fds to be ready to read/write, and For an example of one of the simpler such backends, see select.c in the Libevent source. See kqueue.c or epoll.c for an example of what we do for a fancier backend. hope this helps, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Threading concerns
On Mon, Nov 30, 2009 at 02:49:09AM -0500, Donghua Xu wrote: > Thanks Nick. How do I enable the "common timeout" logic? Or is it > enabled by default in 2.0.3-alphra? Check out the documentation for event_base_init_common_timeout() in include/event2/event.h in Libevent 2.0.3-alpha. Basically, if you have many many events that all use timeouts of the same duration, you can use the new function to make sure that they get put in a queue rather than the minheap, so that you get O(1) performance for adding and running each one rather than O(lg n). If your million timeout durations are not clustered like this, this patch won't help. The original patch at https://sourceforge.net/mailarchive/message.php?msg_name=E1N7Xqx-0001rz-7M%403kljzd1.ch3.sourceforge.com might also be good to look at. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Assertion in event_base_free
On Sun, Dec 06, 2009 at 01:36:27AM +, Matthew Wild wrote: > Hi all, > > I'm getting the following assertion fail when my application shuts down: > >event.c:241: event_base_free: Assertion > `((&base->timetree)->rbh_root == ((void *)0))' failed. > > Now from what I can gather in the source, it is trying to make sure > there are no timers still active? What version are you using? This issue was fixed in November of 2007, and should be resolved in every version of Libevent 1.4.x. > If this is the case, is there a way to easily close all timers? Upgrade. In 1.4.x and later, calling event_base_free() will call event_del() on every pending event. > Otherwise, what do people usually do to shut down a libevent > application cleanly? Must I track all timers and close manually? If you can't upgrade to 1.4.x, your easiest choice is not to call event_base_free() at all on shutdown. (Yes, it's ugly, but it won't actually hurt anything.) The only other approach, if for some reason you can't require a version newer than 1.3e, is indeed to call event_del() on every pending event yourself. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Lib and sample's compilation on Windows
On Mon, Dec 21, 2009 at 05:32:02PM +0800, Tao Feng wrote: > Hello all, > > I just started looking into libevent. On Linux the lib and samples > compile fine and works as expected. But on Windows, how do you > compile the lib and samples? Is there a script/makefile for this? I either use mingw or nmake. The project files are very out of date. We should remove them from the release, unless somebody wants to update them to match what Makefile.nmake does. > I see there is a folder WIN32-Prj which contains VS project. The > 2.0.3-alpha one does not compile. I managed to convert it into VS > 2005 project and was able to compile it on winxp 32bits (there were > some changes). However the event_test sample, the only one I tried, > did not work constantly as it does on Linux. It did not block io, no > event was notified. Is this expected behavior? The samples could use some love; the real unit tests are in test/*.c . If somebody wants to update sample/*.c to conform to good style, use Libevent 2.0.x correctly, and work more portably, that would be great. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Patch for mingw64 (64-bit windows)
On Fri, Dec 18, 2009 at 04:53:41PM -0500, Patrick Galbraith wrote:
[...]
> @@ -11,6 +11,6 @@
> Requires:
> Conflicts:
> Libs: -L${libdir} -levent
> -Libs.private:
> +Libs.private: -lws2_32
> Cflags: -I${includedir}
Hm. Is there some way to do this conditionally? We only want to link
against -lws2_32 when we're on Windows.
> Only in libevent-2.0.3-alpha.win64: libtool
> diff -ur libevent-2.0.3-alpha/log.c libevent-2.0.3-alpha.win64/log.c
> --- libevent-2.0.3-alpha/log.c2009-11-07 13:12:14 -0500
> +++ libevent-2.0.3-alpha.win64/log.c 2009-12-11 17:55:27 -0500
> @@ -101,7 +101,7 @@
> }
>
> void
> -event_sock_err(int eval, evutil_socket_t sock, const char *fmt, ...)
> +event_sock_err(int eval, int sock, const char *fmt, ...)
I've changed both instances to be evutil_socket_t (rather than int.)
Thanks!
> {
> va_list ap;
> int err = evutil_socket_geterror(sock);
> Only in libevent-2.0.3-alpha.win64/sample: Makefile
> diff -ur libevent-2.0.3-alpha/select.c libevent-2.0.3-alpha.win64/select.c
> --- libevent-2.0.3-alpha/select.c 2009-11-07 13:12:14 -0500
> +++ libevent-2.0.3-alpha.win64/select.c 2009-12-11 18:07:06 -0500
> @@ -168,7 +168,11 @@
> event_debug(("%s: select reports %d", __func__, res));
>
> check_selectop(sop);
> +#ifndef WIN32
> i = random() % (nfds+1);
> +#else
> + i = rand() % (nfds+1);
> +#endif
Hang on, what's going on here? select.c isn't even supposed to get
built on windows; it only works for a posix-style bitfield-based
select(). The windows version is in win32select.c (or
WIN32-Code/win32.c in older versions of Libevent).
Is there a problem in the build process that is making select.c get
built on mingw64?
yrs,
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] patch to released 2.0.4 for Windows
On Thu, Mar 11, 2010 at 1:31 PM, Patrick Galbraith wrote: > Hi there! I have a patch for 2.0.4 in order to compile on Windows (tested on > Windows 2008 64-bit), if not already addressed. Small issue. See attached. > The select.c backend is not for Windows; it will not work there, even if you make it compile. (It assumes that the fdset type is implemented as a bitfield, as it is on non-Windows platforms.) Windows uses win32select.c instead. Are you building with mingw, with the nmake Makefile, or with your own project files or something? If it's one of the first two, select.c shouldn't get built on win32; that's the bug there. If it's the latter, please have a look at the nmake Makefile for a list of the C modules that should build on Windows. hth, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] patch to released 2.0.4 for Windows
On Thu, Mar 11, 2010 at 2:10 PM, Patrick Galbraith wrote: > Nick, > > I'm using mingw-w64, using msysgit as my shell setup. My compile options > are: > > ./configure --host=x86_64-w64-mingw32 --build=i686-pc-mingw32 --prefix=/usr > > Would the issue be Makefile.am - start looking there? Either Makefile.am or configure.in. In particular, we check for windows by this stanza in configure.in: dnl - check if the macro WIN32 is defined on this compiler. dnl - (this is how we check for a windows version of GCC) AC_MSG_CHECKING(for WIN32) AC_TRY_COMPILE(, [ #ifndef WIN32 die horribly #endif ], bwin32=true; AC_MSG_RESULT(yes), bwin32=false; AC_MSG_RESULT(no), ) So if your configure is saying, "checking for WIN32... no", then the bug is in configure.in. But if it says "yes" and then tries to build select.c anyway, the bug's probably in Makefile.am -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] FD_SETSIZE - max connections in Windows - help!
On Mon, Mar 15, 2010 at 12:03 PM, Patrick Galbraith wrote: > Hi all, > > We have hit a brick wall with memcached - namely max connections with > select. > > Namely, select from libvent use by memcached on windows, according to the > documentation from MSDN you need to > set FD_SETSIZE to the maximum number of sockets to use. By default > FD_SETSIZE > == 64. We set FD_SETSIZE to 2048 in win32select.c at the top prior to > including winsock2.h, but still no success. It seems to fail at 41 > connections. I'm not sure why you're seeing this 41-connections issue, but I don't think it has anything to do with FD_SETSIZE. FD_SETSIZE affects the declared size of the fd_set structure that select() uses. But Libevent doesn't believe the size of fd_set from the winsock2.h header; instead, it allocates its own. Look at the code for FD_SET_ALLOC_SIZE. (For me, win32select doesn't top out at 41 connections. It can handle hundreds or thousands of connections easily, though the scaling gets poor with thousands.) yrs, -- Nick -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] FD_SETSIZE - max connections in Windows - help!
On Tue, Mar 16, 2010 at 6:50 AM, W.C.A. Wijngaards wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > This problem is 64 max for the windows event notification calls, > WSAWaitForMultipleEvents. select() probably calls one of the others > behind the scenes. > > Best regards, > Wouter > I don't think that's true, Wouter: sure, the WaitForMultiple functions have a 64-handle limit, but select() honestly doesn't. At least, it doesn't on the versions of Windows I've tested. I just tried out 96 fds with select on Windows select in Libevent 2's "main/many_events" test, and it worked just fine on my Windows XP environment. (I just bumped the definition of MANY in that test to 70 so that it will be easier for the unit tests to catch any Windows installations that do break select in this way.) A quick trip to codesearch.google.com shows that we're not the only ones to use more than 64 fds with Windows select: check out w32poll.c in nspr, or WindowsSelectorImpl.c from JDK7. I don't know what's going on with Patrick's situation, but I pretty sure that it's more complicated than Windows select() having a universal 64-socket hard limit. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Pass in __FILE__ __LINE__ __func__ for better logging?
On Fri, Mar 19, 2010 at 10:27 PM, Shuo Chen wrote: > Hi there, > > Now the log callback takes two parameters: > > typedef void (*event_log_cb)(int severity, const char *msg); > > How about adding another two or thee: > > typedef void (*event_log_cb)(int severity, const char *msg, const > char* file, int line /*, const char* func*/); > > Given many logging libraries log file and line info, this may make > logging message more useful. > That would break every program written to use this functionality in Libevent 1.4 and earlier. We try not to break backward compatibility with stable versions. It would be neat, though, to make our logging format more consistent: some messages include function names, and some don't. Maybe we should just have all of the log functions get the function name via the appropriate compiler macro, so we stop having to include it as an argument to every internal log call. There's no reason IMO a patch like that against current git master couldn't go into the next version of 2.0, assuming somebody takes the time to do it. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] levent 2.x multithreaded?
On Thu, Mar 18, 2010 at 12:52 PM, Roman Puls wrote: > Hi Folks, > > I thought libevent would work multithreaded in the sense we can create > (let's say) 4 worker threads that poll the same event_base. Is that > assumption wrong? Yes. Each event base can only have one thread invoking event_base_dispatch() on it (or event_base_loop, etc). Libevent's multithreading support in 2.0 is such that it's possible to manipulate an event_base (or most things) from multiple threads (assuming you've got locking enabled) but the main loop for each event_base still runs in just a single thread. If you want to have multiple worker threads, you need to either 1) have a separate event_base for each one 2) have an event_base running in one thread whose callbacks assign work (through some reasonable model) to the other threads. 3) get Libevent's event_base_dispatch() function to support thread pools. Long-term, we do indeed hope to do 3, but it almost certainly won't happen for Libevent 2.0; maybe in 2.1 or 2.2. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Using timerfd and signalfd on Linux
On Fri, Mar 19, 2010 at 8:42 AM, Shuo Chen wrote: > Hi there, > > Is it possible that libevent 2.0 utilizes timerfd and/or signalfd > provided by recent Linux kernels? > > - timerfd features a better timing resolution (microseconds) than > epoll (milliseconds), and I wouldn't mind support for this if it turns out to run reasonably efficiently. It _would_ add a syscall to every pass through the event loop (as we adjusted the timer), but perhaps that's acceptable to do. You'd want it to be an option in the event_base_cfg, so that people who don't care about sub-msec timings don't take the performance hit, I'd think. > - signalfd makes signal handling like reading a file descriptor It's clean, but would it make a user-visible difference in our current behavior? For both of these, I wouldn't mind a good patch. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] how to flush output immediately?
On Wed, Mar 24, 2010 at 11:24 AM, Jarod Liu wrote: > I want to implement a server which listen on a port. When new > connection come, write out something, then close the connection. > I try use bufferevent_write put data to output buffer then > bufferevent_free, but I found the connection close without flush > output first. > How can I do that with the bufferevent interfaces? Don't free the bufferevent until it's done writing. You can do this trivially in your case by adding a write callback to the bufferevent. Have the write callback check the size of the output buffer, and free the bufferevent when the output buffer becomes empty. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Recuring timers
On Mon, Mar 29, 2010 at 12:36 PM, M P wrote:
> Is there a way to install timers that are not one-shot ? currently I
> have to re-add the timer in the callback to make it restart, however,
> it creates obvious drift.
> I'm looking for a timer that restarts itself automatically...
If you're using Libevent 2.0.x, set the EV_PERSIST flag on the timer event.
If you're using Libevent 1.4.x or earlier, you'll have to fake it as
you describe above. If you care about drift, you could implement a
quick-and-dirty persistent timer system with something like the
following (completely untested!) code:
#if !defined(LIBEVENT_VERSION_NUMBER) || LIBEVENT_VERSION_NUMBER < 0x02000100
struct recurring_timer {
struct event ev;
struct timeval tv_scheduled;
struct timeval tv_interval;
void (*timer_cb)(int fd, short what, void *user_data);
void *user_data;
};
static void recurring_timer_cb(int fd, short what, void *arg)
{
struct recurring_timer *timer = arg;
struct timeval now, delta;
evutil_gettimeofday(&now, NULL);
/* pick the next time to run this event at based on its last scheduled
* time. This logic may or may not be what you want. */
do {
evutil_timeradd(&timer->tv_scheduled, &timer->tv_interval,
&timer->tv_scheduled);
} while (evutil_timercmp(&now, &timer->tv_scheduled, <));
/* delta = scheduled - now */
evutil_timersub(&timer->tv_scheduled, &now, &delta);
if (event_add(&timer->ev, &delta) < 0) {
/* handle this error */
}
/* Run the user's callback */
timer->timer_cb(-1, EV_TIMEOUT, timer->user_data);
}
void free_recurring_timer(struct recurring_timer *timer)
{
event_del(&timer->ev);
free(timer);
}
struct recurring_timer *new_recurring_timer(
struct event_base *base,
struct timeval *interval,
void (*user_cb)(int, short, void *),
void *user_data)
{
struct recurring_timer *timer;
if (!(timer = malloc(sizeof(*timer
return NULL;
evutil_gettimeofday(&timer->tv_scheduled, NULL);
evutil_timeradd(&timer->tv_scheduled, interval, &timer->tv_scheduled);
memcpy(&timer->tv_interval, interval, sizeof(*interval));
timer->timer_cb = user_cb;
timer->user_data = user_data;
event_set(&timer->ev, -1, 0, recurring_timer_cb, timer);
if (base)
event_base_set(base, &timer->ev);
if (event_add(&timer->ev, interval) < 0) {
free(timer);
return NULL;
}
return timer;
}
#else
/* Looks like we have Libevent 2.0. */
#define recurring_timer event
#define free_recurring_timer(timer) event_free(timer)
struct recurring_timer *new_recurring_timer(
struct event_base *base,
struct timeval *interval,
void (*user_cb)(evutil_socket_t, short, void *),
void *user_data)
{
struct event *ev;
if (!(ev=event_new(base, -1, EV_PERSIST, user_cb, user_data)))
return NULL;
if (!(event_add(ev, interval))) {
event_free(ev);
return NULL;
}
return ev;
}
#endif
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] evbuffer - how to read without draining data?
On Wed, Mar 31, 2010 at 7:10 AM, Sebastian Sjöberg wrote: > On Mar 31, 2010, at 10:02 AM, Jarod Liu wrote: > >> Hi, >> I writing a eventbuffer socket app. My app packet have a length >> header, I want to read data only when input buffer length > packet >> length(I need to read the length header first). But it seems like I >> can't read from evbuffer without draining data. Is there a way to do >> that? > > Hi, > > You can do evbuffer_pullup() with the size of your length header and reading > the length header from the returned pointer. You can also use evbuffer_peek() (in Libevent 2.0) if you want to look at the header without moving any data around in the evbuffer internally. *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Unable to construct simple socket client with bufferevent
On Wed, Mar 31, 2010 at 11:53 AM, Anomit Ghosh wrote: > I'm basically trying to a write a very simple echo server/client > combination, just for trying out libevent. The server doesn't use > libevent's bufferevent, but the client does. > > Code for server: http://codepad.org/nP0KsfPY > Code for client: http://codepad.org/XRJAVg5m > > The problem seems to be that even though the connection is established > but the server isn't receiving any data from the client and it quits. > Where am I going wrong? Your client is never actually entering an event loop (as by calling event_base_loop()), so it will exit immediately. You need to call event_base_loop at some point, or libevent will never actually try to get and respond to events. Also, your socket needs to be nonblocking. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] evutil_closesocket() macro portability.
On Wed, Mar 24, 2010 at 6:21 AM, Sebastian Sjöberg wrote: > Hi, > > The EVUTIL_CLOSESOCKET() macro requires you to include unistd.h in your > source for linux and I don't know what on windows. To ease portablilty > wouldn't it be nice if we either turned the macro into a function or included > the appropriate header where we define the macro? > > I'd be happy to provide a patch for whatever behavior you would prefer. Sounds plausible. We might as well turn it into a function: an extra function call is going to be cheap in comparison with the system call. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] A patch for evrpc, allowing extra argument of request_new and reply_new
On Mon, Mar 29, 2010 at 11:42 AM, Shuo Chen wrote: > Another patch to make libevent2 working with Google protocol buffers. > It provides access to the evrpc_req_generic struct members. > http://evproto.googlecode.com/svn/trunk/libevent-2.0.4-alpha-get-request-reply.patch > > To be applied after this patch: > http://evproto.googlecode.com/svn/trunk/libevent-2.0.4-alpha-msg-new-with-arg.patch Hi! I've chatted with Niels, and checked in your patches. I'm not as familiar with the rpc code as Niels is, but it looks like this one changes the arguments of evrpc_register_generic(). We've asked people not to use evrpc_register_generic in their code, of course, so it shouldn't break source compatibility, but it _will_ break binary compatibility. Not a big deal; we're already breaking some binary compatibility between 1.4 and 2.0 for other stuff. Once 2.0 is stable, though, we're going to try to keep the ABI as stable as we can. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] When will libevent 2.x be released?
On Tue, Apr 13, 2010 at 10:03 PM, Angus Salkeld wrote: > Hi > > We are quite keen to use libevent for corosync (corosync.org) > but wish to use the 2.x branch as our daemon is very threaded. > > When do you hope to release the 2.x branch? > > Note: there is no pressure we just want to plan when to do this work. I was hoping for the first quarter of 2010, but that seems unlikely unless the Python developers lend us their time machine. ;) Failing that, I'd be quite sad if the release candidate for 2.0.x comes out later than May. If this means pushing features out to Libevent 2.1, so be it; with luck, Libevent 2.1 will have a much shorter development cycle than 2.0. Fortunately, Git makes it pretty easy for us to hold off on merging code that isn't ready until it really is ready for integration. Our current tentative schedule (assuming I didn't misunderstand my last conversation with Niels and Chris) is to declare a feature freeze for the end of April and try to get a release candidate out by the end of May. > Do you think this will make it into Fedora 14? I have no idea; I don't know Fedora policies. The Fedora maintainers would probably be better people to ask about that. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] evutil_closesocket() macro portability.
On Wed, Apr 14, 2010 at 5:05 AM, Sebastian Sjöberg wrote: > On Apr 14, 2010, at 7:39 AM, Nick Mathewson wrote: > >> On Wed, Mar 24, 2010 at 6:21 AM, Sebastian Sjöberg >> wrote: >>> Hi, >>> >>> The EVUTIL_CLOSESOCKET() macro requires you to include unistd.h in your >>> source for linux and I don't know what on windows. To ease portablilty >>> wouldn't it be nice if we either turned the macro into a function or >>> included the appropriate header where we define the macro? >>> >>> I'd be happy to provide a patch for whatever behavior you would prefer. >> >> Sounds plausible. We might as well turn it into a function: an extra >> function call is going to be cheap in comparison with the system call. > > Here's the patch. Not tested on windows though. Thanks; applied! -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] evhttp_connection_set_timeout granularity
On Thu, Apr 15, 2010 at 4:44 PM, Ron Combs wrote: > Hello, > > What is the best way to set millisecond timeout to a evhttp_connection using > the evhttp framework? Looks like the best long-term solution is to write a patch to add support for having http timeouts as struct timeval rather than as int. You'd need to add new functions to take the place of evhttp_set_timeout() and evhttp_connection_set_timeout(); I'd suggest calling them evhttp_[connection_]set_timeout_tv() . The feature freeze for Libevent 2.0 is the end of April, but this shouldn't be too hard to do. > I tried evcon->ev.ev_timeout = tv but its giving me error: dereferencing > pointer to incomplete type Well, evhttp_connection isn't a public type; the definition in http-internal.h isn't meant to be messed with outside libevent. Even if the definition of evhttp_connection were exposed, altering its events by hand would be a pretty bad idea. It's not safe to modify an event that might be added, and it's a bad idea to use a means other than event_assign to do so. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] event_add fails with ENOENT when on epoll
On Mon, Apr 19, 2010 at 11:24 PM, Denis Bilenko wrote: > Hi, > > I've run into an issue where event_add() fails with ENOENT. > This happens when a socket reuses a descriptor that was recently > closed and was used before with another event. > > The details are below. The question I have is - what are the ways to > work around this? > You must never modify an event that is currently pending. If you want to reassign it, you need to call event_del(), then modify it, then call event_add(). For some backends (like epoll and kqueue) you need to call event_del() on the event when you close the socket, and then event_add() again, even if the fd didn't change: as far as the kernel is concerned, it is no longer interested in tracking fd, once the socket has closed. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] registering the same filehandle with EV_READ (without EV_PERSIST) multiple times
2010/4/24 Péter Szabó : > Hi, > > I'd like to register two struct event's for the same filehandle, with > EV_READ (without EV_PERSIST), with multiple callbacks, and whenever > the file becomes readable, I'd like to get both of my callbacks be > called Libevent 1.4.x and earlier don't support this. Libevent 2.0.x does. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] 100% cpu utilization with openssl bufferevent.
On Thu, Apr 29, 2010 at 5:19 AM, Sebastian Sjöberg wrote: > Hi, > > I've encountered a problem with openssl bufferevents where libevent reports > fd:s as writeable but no action is being taken. [...] > There is no problem when I'm connecting without tls so I think this is an > issue with openssl bufferevents and my guess is that somehow the write events > that openssl bufferevents sets up sometimes doesn't get removed or disabled > properly. > > Is this an issue that someone else has seen and does anyone have any pointers > on how to debug this problem? I haven't run into this myself yet, but the openssl code is relatively new, and probably has some bugs left. To clarify, it seems that the problem is that Libevent bufferevent openssl code never deletes the relevant read events, even though it isn't actually interested in reading? Or the problem is that epoll is returning immediately but not making any events active? If it's the first problem, I'd try adding debugging messages to the points in bufferevent_openssl that call event_add, event_del, and _bufferevent_add_event, along with debugging statements to display the return values of SSL_read and SSL_write, to see at what point we're supposed to be deleting the relevant read event but not really doing it. If it's the second problem, I'd start by testing whether stuff begins to work when you set the EVENT_NOEPOLL environment variable. If so, then the bug is probably with the epoll backend -- or at least, it requires the epoll backend to appear. To debug this, I'd add debugging messages to the loop in epoll_dispatch that calls evmap_io_active to tell me whenever it decided not to call evmap_io_active, and I'd have evmap_io_active tell me whenever it made 0 events become active. With any luck, the debugging output should help figure out exactly what's going wrong here. I'm afraid I'm about to be away from the internet for tomorrow and the weekend, so I won't be able to help much more until early next week. Good luck! yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] 100% cpu utilization with openssl bufferevent.
On Thu, Apr 29, 2010 at 2:14 PM, Sebastian Sjöberg wrote: [...] > Cheers, I haven't yet been able to reproduce it when disabling epoll. > > I've started to debug the evmap_io_active calls and after a while there are a > no events being activated as you said so I guess at some point the there's a > mismatch between what's in epoll and in the event map. Hm. In that case, I'd suspect a problem in the new(ish) changelist code. I'll let you know if I have any luck chasing it down. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] resuming libevent2 after closing all file descriptors
2010/5/3 Péter Szabó : > Hi, > > I'm doing the following with libevent 2.0.4 on Linux: > > strcut event_base *base = event_init() > ... /* not registering any events */ > for (int fd = 3; fd < 256; ++fd) close(fd); > event_reinit(base); > ... > This eventually reports the following warning: > > [warn] Epoll ADD on fd 7 failed. Old events were 0; read change was > 1; write change was 0.: Bad file descriptor > > From ls -l /pro/self/fd, it seems that libevent opens a socketpair on > fds 7 and 8, which I close with close(fd), but event_reinit(base) > doesn't create the new socketpair. Do you think it would be possible > to recreate the socketpairs? Can you say more about why you closing all these fds, including ones that belong to the event base? Libevent doesn't like you to close its socketpair fds any more than it will like it if you're closing its epoll fd, or any more than it would like it if you call free() on the internals of the event_base. In any case, re-grabbing *all* resources isn't the point of event_reinit(). The event_reinit() function is only supposed to be called after a fork() to re-acquire or re-register any resources that aren't inherited by a child process; it is not an alias for calling event_base_free() and event_base_new(). > Please note that technically it would be very cumbersome for me to > call event_init() after the close(fd) loop. What about the newer functions that obsolete event_init(), like event_base_new()? yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] ANN: Libevent 2.0.5-beta released; Libevent 2.0 now in feature-freeze
Hi everyone! Thanks to many people's hard work, Libevent 2.0 has now had its first beta release. You can download it from: http://www.monkey.org/~provos/libevent-2.0.5-beta.tar.gz http://www.monkey.org/~provos/libevent-2.0.5-beta.tar.gz.sig Don't forget to validate the signature. The complete list of changes is available in the ChangeLog file, included with the distribution. *** What's new: Some highlights include: - The evdns module now uses the regular logging system - The evbuffer backend is now more efficient in how it packs bytes when reading. - The bufferevent rate-limiting logic has a few more features to help mix it with existing code. - Debugging mode now catches attempts to mix edge-triggered and non-edge triggered events. - There's a new evbuffer_copyout() function that acts like evbuffer_remove(), but without draining the buffer. - The request and reply members of rpc_req_generic are now exposed, for use by extensions to the RPC protocol. [Shuo Chen] - DNS errors that occur during bbufferevent_connect_hostname are now visible to user code. [Christopher Davis] - There's a new mode on bufferevents where, if you're using deferred callbacks, you can have the bufferevent callbacks executed without holding the lock on the bufferevent. [Joachim Bauch] And of course, - Bugfixes far too numerous to mention here. Check out the ChangeLog file for full details. Interface changes since 2.0.4-alpha are: - The singal_assign() and signal_new() macros are gone. These were added in 2.0.1 as an analogy to the new event_assign() and event_new(), but they were deprecated as soon as they were added in favor of evsignal_assign() and evsignal_new(). - The maximum number of events that can be added to read or write on a single fd is now limited to 65535. In 2.0.4-alpha, the the limit was INT_MAX. In 1.4.x, the limit was "1 reading, 1 writing". - The internal API for generated evrpc files is slightly different, to accommodate Shuo Chen's changes. *** Future: Libevent 2.0.x-rc and beyond. Libevent 2.0 is now in feature-freeze (we don't want to add any more features to it), hack-freeze (no clever backend rewrites, refactorings, or major performance hacks), and API freeze (all code written to work with the documented APIs of Libevent 2.0.5-beta should continue to work with future versions of Libevent). We might change our minds about any of the above if there turn out to be exceptionally good reasons to do so; this is an aspiration, not a promise. ;) I'm currently hoping to spend my (sadly limited) Libevent time over the next month or so hunting bugs and trying to improve the documentation, so we can get a release candidate out. After that, we'll keep fixing reported bugs in the release candidates until one is finally ready, and that will be the first official stable Libevent 2.0.x release. Somewhere in there, we'll fork off a new bugfix-only branch for work on 2.0.x, and we'll start accepting feature patches again for 2.1. My apologies to everybody who's been working on features that didn't get included in 2.0, especially Christopher Davis, Etienne Samson, Martin Scholl, and Valery Kholodkov. Mostly this is our fault for not replying to big complex patches fast enough. I hope we can get stuff straightened out enough to get the features you want into 2.1. When we fork the 2.1 branch, I'll send out some email trying to figure out where everybody's please-merge-this stuff stands. Also, I hope we can get 2.1 out on a faster dev cycle than we managed with the (much-delayed) 2.0. *** Acknowledgements Many thanks to everybody who contributed code, suggestions, or bug reports, including but not limited to Brodie Thiesfield, Christopher Davis, Dagobert Michelsen, Doug Cuthbertson, Frank Denis, Gernot Tenchio, Gilad Benjamini, Giuseppe Scrivano, Jardel Weyrich, Joachim Bauch, Patrick Galbraith, Pierre Phaneuf, Sebastian Hahn, Sebastian Sjöberg, Shuo Chen, Tao Feng, Trond Norbye, William Ahern and Zack Weinberg. peace, -- Nick Mathewson *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Server closes connection to client just after accept()
On Wed, May 12, 2010 at 2:55 PM, Anomit Ghosh wrote: > This is the code in question: http://codepad.org/7eD9n14B > Run the code as `./server 127.0.0.1 ` > > I registered a callback(sock_callback) on the listening socket that > puts the new connections after accepting in a list. Now the problem > seems to be that even after doing event_add() in L73 the connection > closes immediately. I tested this by using telnet to connect to the > server. The printf() on L62 gets printed properly but nothing else > works properly. Where am I going wrong? Even a little help would be > appreciated (I'm kind of on a tight schedule :-| ). Hm. It works for me, so I'm suspecting something platform-dependent. My first guess is that you're calling accept() wrong: you need to initialize addrlen to sizeof(client_addr) before you call it, if I'm reading the accept() documentation right. But try checking for errors on all of your syscalls to catch stuff like that, and see if that's what the problem is? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] evhttpd_cmd_type and support for other HTTP methods
On Tue, Apr 6, 2010 at 1:04 PM, Graham Leggett wrote: > Hi all, > > I need to implement the OPTIONS method in a small evhttp based webserver, > and have discovered that the API only allows me to receive GET, HEAD and > POST requests. > > Is there a way to receive a method outside of this set such as OPTIONS? Looking at the code, I don't think there is, but we'd be glad to take a good patch to do this in Libevent 2.1.x. I've reopened the feature request entry at https://sourceforge.net/tracker/?func=detail&aid=2653030&group_id=50884&atid=461325 yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Assertion error + Segmentation Fault
On Thu, Apr 8, 2010 at 7:46 AM, Sherif Fanous wrote: > Hi > I am using libevent 2.0.4 alpha in my C application. > The below snapshot of my code occasionally encounters the following 2 > problems > > [err] event.c:1961: Assertion !(ev->ev_flags & ~EVLIST_ALL) failed in > event_del_internal > Aborted > Segmentation Fault. I've traced down that it occurs in the event_base_loop > call > > In a nutshell, I have a linked list of servers. I loop through every server, > if I'm connected to that server then I create an event for the socket. I > then run the event loop with a timeout. I use the timeout to break out of > the event loop and check if the termination condition has been encountered > in another thread. I also run the event loop with EVLOOP_ONCE to break out > of the loop as soon as 1 event triggers to re-evaluate the linked list of > servers (new connections established, current connections terminated, ...) > When I break of the event loop, I delete the events and free them. > Can anyone please tell me what I'm doing wrong here? Is it an internal > libevent problem, or is my code messing up somewhere? Try running 2.0.5-beta with the debugging options. (That is, call event_enable_debug_mode() before any other libevent call.) I think the problem in the code you posted might be that your cleanup loop is calling event_del() and event_free() on every event[i], but your setup loop is only setting event[i] if servers_array[i]->is_socket_connected. This will make some of the entries in event[i] be uninitialized, which will make event_del() very unhappy. [Also, fwiw, the code you posted is going to be pretty inefficient: it loops over all the servers every time you call the event loop, so it's going to be O(N) in the number of servers, even if you're on an OS with an efficient O(1) backend for Libevent to use.] hth, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] 100% cpu utilization with openssl bufferevent.
On Thu, Apr 29, 2010 at 2:14 PM, Sebastian Sjöberg
wrote:
[...]
>> If it's the second problem, I'd start by testing whether stuff begins
>> to work when you set the EVENT_NOEPOLL environment variable. If so,
>> then the bug is probably with the epoll backend -- or at least, it
>> requires the epoll backend to appear. To debug this, I'd add
>> debugging messages to the loop in epoll_dispatch that calls
>> evmap_io_active to tell me whenever it decided not to call
>> evmap_io_active, and I'd have evmap_io_active tell me whenever it made
>> 0 events become active.
>
> Cheers, I haven't yet been able to reproduce it when disabling epoll.
>
> I've started to debug the evmap_io_active calls and after a while there are a
> no events being activated as you said so I guess at some point the there's a
> mismatch between what's in epoll and in the event map.
Hi, Sebastian! Have you had any luck tracking this down? I'd really
like to try to get this bug fixed before 2.0 goes into a release
candidate status.
I've looked through the code, and tried to reproduce the behavior
you're seeing, but I haven't been able to figure out where the trouble
is. I'm attaching a patch that should give a warning assertion
failure when the error condition hits; together with debugging log
output [build libevent with -DDEBUG to get that ], it might be enough
to track the bug down.
Also, is there some code I can run to try to reproduce this bug
myself? I've tried writing examples with openssl, but I can't make
the warnings trigger.
yrs,
--
Nick
From 2a8a0d0a81fda941bb221f04c4bc6e0cd269ab77 Mon Sep 17 00:00:00 2001
From: Nick Mathewson
Date: Tue, 18 May 2010 15:22:48 -0400
Subject: [PATCH] Warning statements to catch possible bug in epoll
---
epoll.c |5 +++--
evmap.c |8 +++-
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/epoll.c b/epoll.c
index 6c3cf33..32aff2c 100644
--- a/epoll.c
+++ b/epoll.c
@@ -319,8 +319,9 @@ epoll_dispatch(struct event_base *base, struct timeval *tv)
ev |= EV_WRITE;
}
- if (!events)
- continue;
+ if (ev == 0) {
+ event_warnx("%s: epoll gave us no actionable events for fd %d!", __func__, events[i].data.fd);
+ }
evmap_io_active(base, events[i].data.fd, ev | EV_ET);
}
diff --git a/evmap.c b/evmap.c
index cb28002..4e4d2e9 100644
--- a/evmap.c
+++ b/evmap.c
@@ -389,6 +389,7 @@ evmap_io_active(struct event_base *base, evutil_socket_t fd, short events)
struct event_io_map *io = &base->io;
struct evmap_io *ctx;
struct event *ev;
+ int n_activated = 0;
#ifndef EVMAP_USE_HT
EVUTIL_ASSERT(fd < io->nentries);
@@ -397,8 +398,13 @@ evmap_io_active(struct event_base *base, evutil_socket_t fd, short events)
EVUTIL_ASSERT(ctx);
TAILQ_FOREACH(ev, &ctx->events, ev_io_next) {
- if (ev->ev_events & events)
+ if (ev->ev_events & events) {
event_active_nolock(ev, ev->ev_events & events, 1);
+ ++n_activated;
+ }
+ }
+ if (n_activated == 0) {
+ event_warnx("%s: We found no activateable events for fd %d, events %d!", __func__, fd, (int)events);
}
}
--
1.6.6.1
Re: [Libevent-users] ANN: Libevent 2.0.5-beta released; Libevent 2.0 now in feature-freeze
On Thu, May 13, 2010 at 4:57 AM, Aleksandar Lazic wrote: > Hi Nick, > > On Mit 12.05.2010 21:15, Nick Mathewson wrote: >> >> Hi everyone! >> >> Thanks to many people's hard work, Libevent 2.0 has now had its first >> beta release. You can download it from: >> >> http://www.monkey.org/~provos/libevent-2.0.5-beta.tar.gz >> http://www.monkey.org/~provos/libevent-2.0.5-beta.tar.gz.sig >> >> Don't forget to validate the signature. The complete list of changes >> is available in the ChangeLog file, included with the distribution. > > I have run the clang analyzer, just for my interst and want to share the > result. > > http://none.at/scan-build-2010-05-13-1/ > > I have produce this output with the following commands: > Neat, thanks! The null error in evrpc.c is a false positive; clang didn't realize that event_err never returns. I've tried annotating event_err with __attribute__((noreturn)); let's see if that helps. Similarly, the null dereferences seem to be mostly a matter of clang drawing the wrong conclusion from a redundant check in one of our macros; I've removed the redundant check. Maybe that will make stuff better. The dead assignments look real albeit mostly harmless. I think we should clean up the ones in the main codebase, and probably leave the ones in the unit tests alone. Sadly, a few of them look like places where we're not propagating an internal error sensibly. I've patched a couple, but if anybody else would like to take a crack at more of them, that would be groovy. peace, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] ev_send_error() and friends
On Mon, May 24, 2010 at 8:16 PM, Felix Nawothnig wrote: > Forwarding this discussion from Bug #3006553 to the ML: For reference, that's https://sourceforge.net/tracker/?func=detail&aid=3006553&group_id=50884&atid=461322 > If you have no objections, feel free to apply it. Done, with minor tweaking to make it C90-compliant. > With that issue resolved there are other problems however: > > 1. The reason parameter is now redundant, everywhere, at all send > routines. You should probably always set it to NULL, both for > semi-correctness and to save some bytes of space (it will still do the > same thing as before if you give it a value though). > > That's unfortunate but unavoidable if you want to keep compatibility. Yup; not a huge disaster. > 2. Having a function like evhttp_send_error() with the new behaviour > just asks people to abuse the HTTP status phrase for detailed error > messages (in fact your documentation says that this is what it is > for...). > > It should therefore be deprecated and supplemented by another function - > I'm calling it evhttp_send_reply_html() but you might have a better > name. > > That's what the second patch is. Okay; I'm going to upload this back to the bug artifact on sourceforge and move the artifact to the Patches tracker so we don't lose track of it. The first patch is IMO a bug fix, but the second patch is definitely a new feature, and I don't want to break the feature-freeze for 2.0.x even for probably harmless stuff. [...] > P.S.: It's 2:15 AM now and although I tested my code I obviously can't > be sure that I didn't break anything. Also when doing "make verify" > regress always just says "FAILED" to me... But it did that even before I > changed anything. I've changed the behavior of "make verify" in Git so that it should be better at displaying what test actually fails for you. You can also get more verbose output from older versions of Libevent by running ./test/regress manually. Could you please add a bugtracker entry for the unit test that's failing for you? many thanks, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] ev_send_error() and friends
On Wed, May 26, 2010 at 3:54 PM, Felix Nawothnig wrote: > On Wed, 2010-05-26 at 13:50 -0400, Nick Mathewson wrote: >> > If you have no objections, feel free to apply it. >> Done, with minor tweaking to make it C90-compliant. > > Great. Sorry about the C99 thing - why don't you add the appropriate > compiler switch to the build system? Wouldn't have happened then. :-) We have a lot more switches than that; just pass --enable-gcc-warnings to configure when you build. [...] > Agreed. What's the time-table for opening up a 2.1 branch? I'm just > starting development - currently with my patch for all the error > responses - of an application, and I have no problem dog-food-testing > other people's contributions. Probably some time in June, I hope, when the first 2.0.x-rc comes out. [...] > On a side note - as I have been doing mostly proprietary work in the > last few years... I really enjoy the F/OSS "bazaar style" development > process again... It's a pleasure to have you onboard. ;) If you're looking for fun http exercises, btw, check out the bugtracker entries on sourceforge for http-related bugs. Niels is supposed to be looking for those to see about fixing them as appropriate before 2.0.x-rc can come out, but he's pretty busy these days and I'm sure he'd be glad of any bugfixing time you can spare. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] libevent with EV_WRITE question
On Sat, May 29, 2010 at 6:44 AM, Sherif Fanous wrote:
> Hi
> I have the following scenario, and want to verify the expected outcome
> pthread_wrlock_rdlock(&(server->lock));
> if (server->is_connected) {
> struct send_packets_args *send_packets_args;
> send_packets_args = malloc(sizeof (struct send_packets_args));
> // Initialize send_packets_args to contain buffer to send, and length of
> buffer to send
> event_base_once(base, server->socket, EV_WRITE, send_packets, (void *)
> send_packets_args, NULL);
> pthread_wrlock_unlock(&(server->lock));
> event_base_loop(base, EVLOOP_NONBLOCK);
> } else
> pthread_wrlock_unlock(&(server->lock));
> In my callback after I'm done with the sending I perform the cleaning up
> (free the send_packets_args)
>
> Now my question, if at the moment I release my read lock above, the
> connection to the server is lost and server->socket is closed. Will the
> callback send_packets get called? I want to know the expected outcome
> because I need to make sure I'm not leaking memory allocated for the
> send_packets_args that gets freed as the final step of the send_packets
> callback.
First off, to clarify, if you want to make sure you're not leaking
memory when an event isn't called, you pretty much need to use
event_add() and and event_del() manually, and not trust to
event_base_once(). If you use event_base_once() and the event never
triggers, then there's no way to delete it.
But, will it trigger? I believe so; Libevent seems to report remotely
closed sockets as EV_READ and EV_WRITE on all the backends I can test.
I'm adding a unit test for this to be sure that it holds everywhere.
Stylistically, though, I believe that it's most common to treat the
availability of the EOF as a -read- event, not a write event. (That's
because you detect a close by calling read() or recv() and having it
return 0.) I believe some old versions of Libevent may have screwed
this up on Windows and reported EOF as write instead, though, so be
sure to test for the behavior you want here if you're using an old
version.
yrs,
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] socket message boundary problem
On Mon, May 31, 2010 at 1:14 PM, Hor Meng Yoong wrote:
> Hi:
>
> I am using libevent2. Is that a wrapper function/library to send and
> receive fixed length message, or variable length message with message length
> embedded in the first 2 bytes with libevent2? Thank
Got all three of your messages. No need to send so many!
There's no built-in function to do this, but it's very easy to do with
bufferevents. Here's some example demo code to show you the general
idea of how it's done:
#include
#include
#define MSG_LEN 512
int
send_fixed_msg(struct bufferevent *bev, const char *msg)
{
bufferevent_write(bev, msg, MSG_LEN);
return 0;
}
void
fixed_msg_readcb(struct bufferevent *bev, void *ctx)
{
char msg_buf[MSG_LEN];
struct evbuffer *inp = bufferevent_get_input(bev);
while (evbuffer_get_length(inp) >= MSG_LEN) {
evbuffer_remove(inp, msg_buf, MSG_LEN);
/* Here's where you add code to handle the contents of
* msg_buf */
}
}
void
setup_fixed_msg_bufferevent(struct bufferevent *bev)
{
/* Set up the read callback to be invoked when we have some bytes that
have arrived. In practice, you'd also want to set the other
callbacks, and maybe provide some data too. */
bufferevent_setcb(bev, fixed_msg_readcb, NULL, NULL, NULL);
}
/* Now, variable-length messages. Let's assume that the message format
* is a two-byte length value encoded in big-endian (network) order,
* plus that number of bytes worth of message */
int
send_var_msg(struct bufferevent *bev, const char *msg, int length)
{
ev_uint16_t len_encoded;
if (length < 0 || length > 65535)
return -1;
len_encoded = htons(length);
bufferevent_write(bev, &len_encoded, 2);
bufferevent_write(bev, msg, length);
return 0;
}
void
var_msg_readcb(struct bufferevent *bev, void *ctx)
{
struct evbuffer *inp = bufferevent_get_input(bev);
char msg_buf[65536];
ev_uint16_t len_encoded;
int msglen;
while (1) {
int len_in_buf = evbuffer_get_length(inp);
if (len_in_buf < 2)
return;
/* We have enough data to read a length field. For now,
* just inspect it but don't remove it.
*/
evbuffer_copyout(inp, &len_encoded, 2);
msglen = ntohs(len_encoded);
if (len_in_buf < 2 + msglen) {
/* The entire message hasn't arrived yet. */
return;
}
/* The message is here; pull it out of the buffer. */
evbuffer_drain(inp, 2); /*discard the length field */
evbuffer_remove(inp, msg_buf, msglen);
/* Here's where you add code to handle the contents of
* msg_buf */
}
}
void
setup_var_msg_bufferevent(struct bufferevent *bev)
{
/* Set up the read callback to be invoked when we have some bytes that
have arrived. In practice, you'd also want to set the other
callbacks, and maybe provide some data too. */
bufferevent_setcb(bev, var_msg_readcb, NULL, NULL, NULL);
}
Note that in practice you might want to use watermarks to avoid
extraneous calls to the _readcb functions. For more info on the
bufferevent API, check out chapter Ref6 of
http://www.wangafu.net/~nickm/libevent-book/ .
hope this helps,
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] How to close bufferevent connection?
On Mon, May 31, 2010 at 11:07 PM, Felix Nawothnig wrote: > Hey. > > What's the "suggested" way to close a bufferevent connection? > > Currently I do: > > fd = bufferevent_getfd(bev); > evutil_closesocket(fd); > > ... followed by a manual: > > my_event_handler(bev, BEV_EVENT_EOF, NULL); > > ... because it doesn't seem to be called otherwise. > > Is this the "correct" way? Is the fact that the event handler not called > a bug? > Two points to expand on hcpark's reply: 1) Calling bufferevent_free() will only close the socket if the bufferevent was set up with the BEV_OPT_CLOSE_ON_FREE option. [I already mentioned this to Felix on Sourceforge, but it's good to keep in mind. The default behavior by analogy with event_del and event_free, neither of which close their underlying fd.] 2) The BEV_EVENT_EOF event is only generated for an EOF that you *receive*, not when you close the socket yourself. Since most backends (select, poll, etc) don't provide a notice when *you* close an fd, there's no event generated for that on the bufferevent. *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] BUG: libevent 1.4.14, wrong version info
On Sat, Jun 12, 2010 at 9:29 AM, Juergen Daubert wrote: > Hello, > > The libraries of libevent 1.4.13 and older are versioned by > libtool as > > *-1.4.so.2.1.3 > > whereas libevent 1.4.14 is using > > *-1.4.so.1.2.3 > > This breaks every application linked against 1.4.13 if you > upgrade to version 1.4.14. I don't think that this is the > desired behaviour, but a bug caused by the wrong VERSION_INFO > string in Makefile.am. It is VERSION_INFO = 3:3:2 but should > probably be VERSION_INFO = 3:4:1 and resulting library-names > like *-1.4.so.2.1.4. > Ugh, how annoying! I think this warrants a 1.4.14b, if Niels agrees. (1.4.14b should contain this fix, and nothing else.) Actually, I think the VERSION_INFO should be 3:4:2, since 1.4.14 is binary compatible not only with 1 but 2 previous releases. This raises a related question: were we right to drop the use of the -release flag in libevent 2.0? We took a flag from Zack Weinberg to do so back in 2009 [1], but I'm not so sure right now. What do people with more libtool experience think? Libevent is the only library I've used with libtool. [1] The patch and rationale are in git commit 0cc10e419c6d1bca; non-git-users can see them here: http://levent.git.sourceforge.net/git/gitweb.cgi?p=levent/libevent;a=commit;h=de0096f4f3fa35fe961c4b0e59d85617d5aa2689 yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] setting bufferevent without heap-allocation
On Thu, Jun 17, 2010 at 2:35 AM, Avi Bab wrote: > > > Hi All, > > Is there an option to set up a bufferevent without heap-allocation > (something like event_assign() for events) ? > I'm afraid not; bufferevents need to be able to do heap allocation internally for their evbuffers, so they don't really support being stack- or statically- allocated. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] BUG: libevent 1.4.14, wrong version info
On Sat, Jun 12, 2010 at 2:10 PM, Niels Provos wrote: > On Sat, Jun 12, 2010 at 7:29 AM, Nick Mathewson wrote: >> Ugh, how annoying! I think this warrants a 1.4.14b, if Niels agrees. >> (1.4.14b should contain this fix, and nothing else.) > > Fine by me. Okay, I've spent a while talking with people who seem to know what they're talking about, and getting libtool versions explained to me over and over until I think I might finally get it. (Special thanks to "vorlon" of the Debian project for answering my questions from a packager's point of view.) The correct version info for 1.4.14b should be 4:0:2, not 3:4:2 or whatever fool thing I was saying before. Niels, I've tagged a new "release-1.4.14b-stable"; it changes nothing but the VERSION_INFO, the version number, and some explanatory comments in Makefile.am. Whenever you can get it uploaded, that would rock. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Thread safety with Evbuffers and Bufferevents
On Sun, Jun 20, 2010 at 5:51 AM, Avi Bab wrote: > Hello all, > > > > I’d like to use a socket based bufferevent to perform asynchronous writing > to the underlying socket: > > One thread, X, adds data to the output evbuffer (either by > evbuffer_reserve_space+ evbuffer_commit_space or by evbuffer_add_reference) > and another thread, Y, dispatches on the bufferevent’s event_base (i.e., > actually sends the data via the socket). > > My questions are – > > Does thread X need to perform locking on the evbuffer? > > If so then how can I have thread Y synchronize on the same locks? > > I assume that setting the bufferevent option BEV_OPT_THREADSAFE will not > affect the “direct” access to the evbuffer by X. Setting BEV_OPT_THREADSAFE will make the evbuffers and the bufferevents all get locks, which should work fine for most operations. As of Libevent 2.0, a bufferevent and the two evbuffers associated with it all share the same lock. (Some future version might get smarter here; but in practice the current approach seems good enough.) There's one wrinkle here, though: sometimes you want a *set* of operations on a bufferevent or evbuffer to be atomic. For example, even though evbuffer_reserve_space() is locked, and evbuffer_commit_space() is locked, you still don't want another thread modifying an evbuffer in between when you call evbuffer_reserve_space() and evbuffer_commit_space(). You can prevent this by calling evbuffer_lock() before you start doing the operations you want to be atomic, and evbuffer_unlock() when the atomic operations are finished. Libevent 2.0.6-rc will introduce a bufferevent_lock()/bufferevent_unlock() pair to perform the same function on bufferevents. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] deadlock in libevent-2.0.5-beta
On Thu, Jul 1, 2010 at 6:44 AM, Avi Bab wrote: > > > Running on Linux with pthreads. > > > > One thread (CBTcpProxyListenerThread below) adds bufferevents (with option > BEV_OPT_THREADSAFE) to an event_base. > > A second thread (CBTcpProxySenderThread) dispatches on the event_base. > > > > bufferevents are removed from the event_base either by a third thread or by > the CBTcpProxySenderThread by calling bufferevent_free (without calling > bufferevent_disable first – is this a misuse?). > > > > The deadlock happens on pretty high load: ~6000 bufferevents are added and > removed per second. Each one is triggered for write ~10 times per seconds > (which gives ~60,000 triggeres per-second). The stack traces look like they aren't the whole story. It seems the two threads you listed are both trying to acquire the lock for the event base, and blocking on it.. But what's the stack of the thread that's actually holding the lock? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] deadlock in libevent-2.0.5-beta
On Sat, Jul 3, 2010 at 2:10 AM, Zhou Li wrote: > I met such deadlock too. It happened under very high load just as you said. > I think the cause is that the call write(th_notify_fd[1]) got blocked ( > sorry I didn't remember the exact position of this call to write > th_notify_fd). > In event.c line 2597: > /* > This can't be right, can it? We want writes to this socket to > just succeed. > evutil_make_socket_nonblocking(base->th_notify_fd[1]); > */ > When I uncommented this block of code, the deadlock disappeared. > This change isn't correct, though. th_notify_fd[1] is used to tell the main thread (the one running event_base_loop) to wake up. The code that writes to it doesn't check for EAGAIN, so making that socket nonblocking means that some attempts to wake up the main thread will just get lost. *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] deadlock in libevent-2.0.5-beta
On Sun, Jul 4, 2010 at 4:29 AM, Avi Bab wrote: > > Indeed it seems that someone, some when, failed to release the lock. > At the time of the deadlock the third thread (The ReceiverThread) is > dispatching on a different eventbase. > > This third thread does do some manipulation on bufferevents that are > registered with the Sender's event_base: [..] > > This is the only interaction with the third thread. > I do not see a relation to the deadlock. > Confusing! I don't see how this would cause the deadlock either. Have you tried configuring the lock debugging feature? Just after you call evthread_use_pthreads(), call evthread_enable_lock_debugging(): it will put wrappers around all the locks and locking callbacks to track each lock's current owner and recursion count. If there are gross errors, you'll get an assertion failure with a hopefully useful stack trace. If not, you can inspect the recursion count and owner in the debugger by casting the lock to "struct debug_lock" and looking at the count and held_by variables. [The owner is just a pthread_t as an unsigned long, as returned by pthread_self().] yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] deadlock in libevent-2.0.5-beta
On Mon, Jul 5, 2010 at 3:03 AM, Avi Bab wrote: > > I now have a clearer view of things. > Like Zhou said - there is a thread adding an event to the sender's event_base > which is blocked on writing to base->th_notify_fd[1]: Oho! Indeed, this is absolutely right. My apologies to Zhou Li for not getting the point before, and thanks to you and Scott Lamb for explaining it. I think that reinstating the commented-out line is indeed correct, so I'll check it in. One point that confusese me still: were you saying that even with Zhou's fix, there's still a deadlock? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] When to use libevent?
On Sun, Jul 4, 2010 at 10:06 AM, Stef Bon wrote: > Hello, > > I'm building a construction serving the user in his/her homedirectory > easy access to various resources like > a SMB share, USB stick, partition on harddisks etc. See: > > http://linux.bononline.nl/projects/mount.md5key.new/ > > I'm wondering, when writing a fuse module to get access to an Internet > service like Flickr or a webservice > like a cloud, is libevent required?? Well, Libevent is only a library that provides compatibility across different operating systems' asynchronous IO facilities. As such, it is basically never *required* for a new project; you could always replace it with direct calls to the underlying OS facilities. Since FUSE is Linux-only, you could probably just write your networking code to use epoll or aio and forget about portability layers. As for the question of whether you *should* or *could* use Libevent in implementing a user-space filesystem... I don't know why you couldn't. You can get an intro to what libevent can do for you at http://www.wangafu.net/~nickm/libevent-book/ . Only you can decide whether that functionality is a good fit for your program, though. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] the usage of ev_pncalls in struct event
On Thu, Jul 1, 2010 at 11:31 AM, xiaobing jiang wrote:
> hi all:
> what's the usage of ev_pncalls in struct event? why not direct
> use ev_ncalls?
>
> two question:
> 1. in libevent 1.4.14, it seems only used in event_process_active(),
> but in libeven2 it used in event_signal_closure. why?
Because the ncalls logic really only makes sense for signals, so in
2.0.x it's made more signal-specific.
> 2. ev->ev_pncalls = &ncalls; // from event_process_active()
> here, ncalls is a stack variable.
>
> if (ev->ev_ncalls && ev->ev_pncalls) {
> /* Abort loop */
> *ev->ev_pncalls = 0; // ev.ev_pncalls point to a
> stack variable, will this cover the stack?
> }
>
A fine question. I think the answer is, "The code in 1.4.x is safe
and will not clobber the stack: if ncalls is nonzero, then we are
either inside event_process_active() where pncalls points to ncalls,
or we have called event_active, which sets pncalls to &ev->ev_ncalls."
Nevertheless, I'd feel a bit safer if we didn't leave the pointer to
an unused stack location lying around; what do you think of the
attached patch?
The idea behind ev_ncalls was to allow signal handlers to get run once
for each signal received. This is important for some programs,
especially ones that do things like treating one SIGINT as meaning
"shut down at your own convenience" and a second SIGINT as meaning
"shut down immediately."
The idea of the ev_pncalls pointer, as I understand it, was to provide
a good way for an event_del() called from inside an event callback for
a running signal event to safely delete that event. Once a callback
has called event_del, it's allowed to reassign that event, or free it,
or whatever. But this means that event_process_active() can't just
loop on ev->ev_ncalls, since any call to the callback might invalidate
ev->ev_ncalls. Instead, we loop on a stack variable, and the
ev_pncalls pointer gives the event_del() to modify that stack pointer.
But the 2.0.x situation is a little trickier. In fact, I think the
code in 2.0.x is not threadsafe wrt the ev_pncalls pointer. I've got
a bug report with a patch at
http://sourceforge.net/tracker/?func=detail&aid=3008667&group_id=50884&atid=461322
, and I'm really hoping somebody will have a chance to look it over
before I merge it.
peace,
--
Nick
0001-Clean-up-a-pointer-to-stack-in-event_process_active.patch
Description: Binary data
[Libevent-users] Re: cmake build harness
2010/5/31 Brodie Thiesfield : > Hi all, > > I've updated the cmake build harness that Alexey Ozeritsky supplied, and > I've updated it to build the latest source both released and in git. > I've tested the build on both Windows (VC 2008) and Linux (gcc 4.4) and > apart from warnings it builds. > > It now does full detection of header files, functions, symbols, etc on > all platforms. It also generates the RPC files from the source on > Windows (requires python to be installed). The regress test harness > runs, but there are a number of failures on both Windows and Linux. > > This build harness is still a work in progress, but it shows promise. By > using cmake, the Visual Studio 2008 projects are generated automatically > (or makefiles on Linux). Hi, Brodie, and sorry for the delay. :/ This looks like something worth considering for Libevent 2.1. (Libevent 2.0.x is in feature-freeze.) I think the sensible first target here is to see if we use this just for generating visual studio stuff only, and continue to ship autotools for unix-like systems. Replacing autotools on unix/mac/mingw would be a harder sell for me because, as it stands, it works okay, and it does lots of stuff that your cmake script doesn't. For example, there are plenty of --enable-XYZ flags your script doesn't support, and there are some tricky cases that I don't see where it considers. (For example, how you need magic options in order to make some libcs build thread-safe code.) This is not to say "never", but rather to say "I fear regressions from replacing autotools on the platforms where it currently works, but I like the idea of generating project files that Visual Studio users will find worthwhile, since apparently none of them agree with me that Makefile.nmake is better than project files." ;) > This is also some changes required to the bench_cascade.c file in order > to have it compile on Windows. It looks like these already happened in git master somewhere along the line? I'm not seeing a difference between your bench_cascade.c and the one in git head. > Question: > Since all of the header files are now in include/event2/. wouldn't it be > better if the event-config.h header file was also generated into that > directory so that all libevent headers are always loaded from > ? That sounds like a good idea. I've got a git branch on my github repo that tries to do this; if nobody objects, I'll merge it soon. (git://github.com/nmathewson/Libevent.git ; branch is move_eventconfig_h.) many thanks, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] [PATCH] uri parsing helpers
On Wed, Jul 14, 2010 at 8:43 AM, Pavel Plesov wrote: > This patch introduces evhttp_uri_* functions to deal with URI parsing. > > See evhttp_uri_parse(), evhttp_uri_clear() and evhttp_uri_join() for details. > > Patch is made against Nick's master branch @ github > Thanks for the patch, Pavel! Could you upload it to the sourceforge patch tracker at https://sourceforge.net/tracker/?group_id=50884&atid=461324 ? Libevent 2.0.x is currently in feature-freeze, so we won't be merging new externally visible functions (except for severe circumstances) until 2.1.x development forks off. Also, the official Libevent repo is not on github; it's the sourceforge one at git clone git://levent.git.sourceforge.net/gitroot/levent/levent . yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] [PATCH] http: Allow wildcards in request callback paths
On Sat, Jul 10, 2010 at 4:05 AM, Bobby Powers wrote: > this allows you to register callbacks with paths like "/artist*". > > based on the patch here: > http://www.mail-archive.com/libevent-us...@monkey.org/msg00985.html Neat! There's a tracker entry for this feature request on the sourceforge tracker at https://sourceforge.net/tracker/?func=detail&aid=1826562&group_id=50884&atid=461325 ; can you upload (or paste) your patch there so we don't lose track of it between now and forking Libevent 2.1.x? yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Ability to shutdown() socket with bufferevents?
On Tue, Jul 27, 2010 at 12:55 PM, Corey Stup wrote: > I'm needing the ability to shutdown(fd, 1) on a bufferevent managed > socket [created with bufferevent_socket_new() and > bufferevent_socket_connect()] as part of my protocol. The flow of > which looks somethign like this: > > Client opens connection to server > Client sends data. > Client closes the sending side of the socket to indicate EOF. > Server sees EOF and closes the receive side of the socket > Client > > For my first pass at this client, I'm using a bufferevent, but without > an obvious way to send a EOF, it seems like I'm stuck. It appears > that at some point in the future, bufferevent_flush(..., EV_WRITE, > BEV_FINISHED) may provide what I'm looking for, but its not yet > implemented. > > Anyone have any suggestions? Right now there's no way to make a bufferevent call shutdown() itself, but what if you have a write handler on the client bufferevent just call shutdown() manually on the fd to send a FIN when the write buffer is empty? You can get the fd with bufferevent_get_fd(). yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] Re: [Libevent-users] error: aggregate ‘evkeyvalq q uery’ has incomplete type and cannot be defined
On Sun, Jul 25, 2010 at 11:12 AM, Mihai Draghicioiu wrote: > Hi guys. I'm trying to extract parameters from the url query, but i > can't seem to figure out why i'm getting this error. Here is the code: What version of libevent? What order are you including the headers in? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] SSL support in evhttp.h
On Wed, Jul 28, 2010 at 9:10 AM, Graham Leggett wrote: > Hi all, > > Is SSL support planned for the event driven http server described in > evhttp.h? > It would be good to add, if somebody has the time to implement it. It won't go into Libevent 2.0, since that's in feature-freeze, but I'd bet a good implementation would get accepted into 2.1 pretty easily. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] running libevent with multiple bases
On Tue, Jul 27, 2010 at 5:16 PM, Amy Hwang wrote: > I have an architecture that reads and writes to a large number of > sockets, where the sockets are spread over a number of threads, each > with its own libevent base. Each socket is "owned" by a single > libevent base, and the bases don't interact with each other. I opted > for this architecture rather than a thread pool/job distribution one > because most of the heavy lifting should be interacting with the > sockets. I've read a couple of posts stating that multiple threads > with bases aren't really supported in libevent, but what I have so far > seems to be working. > > Does what I've described seem like a reasonable thing to do in > libevent 2.0.5? It sounds reasonable to me. > I've also enabled pthreads just to be safe, although > I'm not exactly sure pthreads is actually contributing to anything. Probably nothing for the architecture you describe, so long as you're right about the kind of isolation you're getting. Remember that making sure that threads never interact with each other's bases is not (alas!) as simple as making sure that they don't refer to the bases directly. Each thread must also never touch (e.g., add, delete, or activate) any event associated with another event base, or touch any object (bufferevent, evdns_base) whose events are associated with other event bases, or so on. Personally, I think it's safer just to enable threads. peace, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] 100% cpu utilization with openssl bufferevent.
On Thu, Apr 29, 2010 at 5:19 AM, Sebastian Sjöberg wrote: > Hi, > > I've encountered a problem with openssl bufferevents where libevent reports > fd:s as writeable but no action is being taken. I think Mike Smellie finally tacked this down in his comments in bug http://sourceforge.net/tracker/?func=detail&aid=3009983&group_id=50884&atid=461322 ; I applied a later version of his change to master as commit cf249e7d997. If one of the original reporters here can try the fixed code out, that would be great. *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] lock question
On Thu, Jul 29, 2010 at 4:23 AM, Avi Bab wrote: > > > Does writing to an output evbuffer (specifocally I use > evbuffer_add_reference) of a bufferevent attempt to acquire the event_base’s > lock? Usually not. It does acquire the bufferevent's lock. It *might* have to tell the bufferevent to start writing if it wasn't previously writing, which (if the bufferevent is using events to write) will make the bufferevent's write event have to get added. > I.e., if thread A calls evbuffer_add_reference(bufferevent_get_output(bev), > data, data_len, cleanupfn, extra) - will it block while thread B that > dispatches on bev’s event_base is handling active events? Handling active events doesn't block; the event_base releases its lock when it's running callbacks, and only grabs it again when it needs to find the next callback. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] SSL support in evhttp.h
On Wed, Jul 28, 2010 at 11:28 PM, Zhuang Yuyao wrote: > hi, > > here is my patch and sample code for libevent 2.0 to add ssl support to > evhttp. > the trick is to allow the caller to set bufev of struct evhttp_connection. > > Best regards, > Zhuang Yuyao Thanks! Two quick questions: 1) Would you mind uploading this to the patch tracker at https://sourceforge.net/tracker/?group_id=50884&atid=461324 ? Because Libevent 2.0 is feature-frozen for now, I can't merge anything for a while, and I wouldn't want to forget about this between now and then. 2) Does this "callback to set the bufferevent" trick have other realistic applications? If the only use is getting https to work, we might as well just hide the details from the user and provide an interface to do https directly. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Small patch for grabbing underlying event_base from an evhttp_request
On Tue, Jul 27, 2010 at 3:34 PM, Mark Ellzey wrote: > Greetings, > > Attached is a small patch which allows a user to grab the underlying > event_base from an evhttp_request. I have patched this against the > current master on github. > > If there is a better way to do this, I am open to suggestions. > > Thanks in advance! Hi! Could you either upload this to the patch tracker at https://sourceforge.net/tracker/?group_id=50884&atid=461324 ? Because Libevent 2.0 is feature-frozen for now, I can't merge anything for a while, and I wouldn't want to forget about this between now and when 2.1 branches. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] Re: [Libevent-users] Re: [Libevent-users] Re: [Libev ent-users] error: aggregate ‘evkeyvalq query’ has incomp lete type and cannot be defined
On Thu, Jul 29, 2010 at 4:02 AM, Mihai Draghicioiu wrote: > Thanks to your suggestion, i've fixed it by prepending #include > to the other libevent headers. Thanks! Ugh. That's what I suspected, but it really shouldn't be necessary. I've added a bug about it, targeting 2.1: https://sourceforge.net/tracker/?func=detail&aid=3036645&group_id=50884&atid=461322 (I'd try to get it fixed in 2.0, but historically the handling of TAILQ in user-visible headers has lead to some hard-to-diagnose bugs in older versions of Libevent, so rather than take the risk of making things much worse for some people in order to remove this (not so bad but kind of annoying) issue in a stable release, I think we're better off trying to fix it in the next alpha series.) yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Measuring load
On Sat, Jul 31, 2010 at 8:21 AM, Matthew Wild wrote: > Hi all, > > I've been wondering if there is some way I can add load measurement to > my application when using libevent. > > How to define "load" is open to interpretation, but generally I refer > to the case where events are being fired faster than my application is > processing them. Similar to the loadavg found on most *nix systems, I > guess this would be based on the average number of outstanding events > over a certain period of time. > > Assuming this is the best approach, I searched for a way to retrieve a > list of events awaiting processing from libevent, but couldn't find > anything. Am I missing it, or is there no way to get this information? There isn't an exposed interface to get the list of pending events from an event base right now, but it would be interesting to think about ways to get load information in future versions of Libevent without hurting performance. Other measurements you might care about could include: * Average delay between getting a result from the dispatch function and calling it again. * Average number of low-priority events starved for attention while high-priority events are getting processed. * Average delay between an event becoming active and its callback finishing. What else would folks find interesting? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] libevent with python segfaulting
On Sun, Aug 1, 2010 at 3:01 PM, Michael Gale wrote: > Hello, > I am using the latest release of libevent-python and the latest release > of libevent-1.4. > After I am done writing data and close the socket I attempt to remove the > write event callback from the event loop. If I do this the server segfaults > between 500 and 600 connections. > If I do not attempt to remove the write event for the fd, the server appears > to scale with out issues accept the connection objects are never garbaged > collected and it eventually runs out of memory. > Any suggestions on how to resolve / find the segfault issue or a better > method to use libevent with python? So, if you can attach a debugger and get a stack trace, or run in a debugger and get a stack trace, or get a core file and use it to generate a stack trace, that's the usual way to debug segfault issues. You could also try coming up with a minimal program that demonstrates this, then show it to others and see if they can spot the problem. You might also try contacting the libevent-python maintainers; I don't know if they're on this list, but debugging libevent-and-python issues is likely more their expertise. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] bufferevents and write signalling
On Thu, Aug 5, 2010 at 2:13 PM, Mark Ellzey wrote:
>
> This is an odd question that hopefully someone can help me out with. I
> have a function right now that looks a bit like this:
>
> void
> read_data(struct bufferevent *bev, void *args) {
> evbuffer *data;
>
> data = bufferevent_get_input(bev)
>
> if (evbuffer_get_length(data) < some_size) {
> /* waiting for some specific amount of data */
> return;
>
> process data here
>
> evbuffer_drain(data, some_size);
>
> if (evbuffer_get_length(data)) {
> return read_data(bev, args);
> }
> }
>
> If more data is present at the time it has finished reading one block of
> data it was interested in, if you were to return, you never get notified
> unless something was added.
Right. The read callback is invoked when more data *arrives,* not
whenever there is data.
If you want it to handle all the data, why not just use a loop:
while (evbuffer_get_length(data) >= some_size) {
/* ... */
}
That way you avoid the arbitrary recursion.
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
[Libevent-users] ANN: Libevent 2.0.6-rc released
Hi everyone! Thanks to many people's hard work, Libevent 2.0 has now had its first release candidate. You can download it from the Sourceforge site at: https://sourceforge.net/projects/levent/files/libevent/libevent-2.0/libevent-2.0.6-rc.tar.gz/download There's also a pgp signature of it (by me) there, at: https://sourceforge.net/projects/levent/files/libevent/libevent-2.0/libevent-2.0.6-rc.tar.gz.asc/download Don't forget to validate the signature. The complete list of changes is available in the ChangeLog file, included with the distribution. Niels will upload this to monkey.org as soon as he has a chance; he's been super-busy with his work lately, which is why I've been doing the "lead developer" stuff with this release. *** A note on http status With regret, I need to say that this release is really only "-rc" quality with respect to the non-evhttp parts of the code; the evhttp module should still be considered "beta" quality. I hope we can bring it up to snuff in the next release or two. Personally, I haven't been comfortable enough in the evhttp code to merge patches in it without review from Niels, and Niels has been to busy with work to get the time to fix bugs. I hope that we can change that with the next release or two, but we're going to need lots of help. *** What's new in 2.0.5-rc: Some highlights include: o Avoid event_del on uninitialized event in event_base_free (6d19510) o Correctly recognize .255 addresses as link-local when looking for interfaces (8c3452b) o Use generic win32 interfaces, not ASCII-only ones, where possible (899b0a3) o Fix the default HTTP error template (06bd056 Felix Nawothnig) o Close the file in evutil_read_file whether there's an error or not (0798dd1 Pierre Phaneuf) o Fix possible nullptr dereference in evhttp_send_reply_end() (29b2e23 Felix Nawothnig) o Fix a deadlock related to event-base notification. Diagnosed by Zhou Li, Avi Bab, and Scott Lamb. (17522d2) o Possible fix to 100% cpu usage with epoll and openssl (cf249e7 Mike Smellie) o Don't race when calling event_active/event_add on a running signal event (fc5e0a2) o Suppress a spurious EPERM warning in epoll.c (e73cbde) o Fix wrong size calculation of iovec buffers when exact=1 (65abdc2 niks) o Change bufferevent_openssl::do_write so it doesn't call SSL_write with a 0 length buffer (c991317 Mike Smellie) o Fix compilation of sample/le-proxy.c on win32 (13b912e Trond Norbye) o Fix rate-limit calculation on openssl bufferevents (009f300) o Remember to initialize timeout events for bufferevent_async (de1f5d6 Christopher Davis) o Move event-config.h to include/event2 (ec347b9) And others too numerous to list; see the ChangeLog file for the full details, and the git log for the even more full details. *** Fun facts about Libevent 2.0.5-rc: This release changes fewer lines than any other release in the 2.0 series: 2.0.6-rc: 104 files changed, 1749 insertions(+), 1192 deletions(-) 2.0.5-beta: 122 files changed, 3283 insertions(+), 1194 deletions(-) 2.0.4-alpha: 122 files changed, 8112 insertions(+), 4190 deletions(-) 2.0.3-alpha: 98 files changed, 10131 insertions(+), 2213 deletions(-) 2.0.2-alpha: 90 files changed, 5677 insertions(+), 1619 deletions(-) 2.0.1-alpha: 113 files changed, 23909 insertions(+), 7670 deletions(-) Unit test coverage is at 79.76%, up from 79.35% in Libevent 2.0.5-beta. The most tested nontrivial module is evmap.c at 92.44%. The least tested is bufferevent_ratelim.c at 58.05%. (Numbers taken from my Linux desktop; your coverage may vary.) The documentation at http://www.wangafu.net/~nickm/libevent-book/ now covers 66% of the exposed functions, constants, and structures in the Libevent headers. *** Status: Libevent 2.0.x-rc and beyond. Libevent 2.0 is *still* in feature-freeze (we won't add any more features to it), hack-freeze (no clever backend rewrites, refactorings, or major performance hacks), and API freeze (all code written to work with the documented APIs of Libevent 2.0.5-beta should continue to work with future versions of Libevent). We might change our minds about any of the above if there turn out to be exceptionally good reasons to do so; this is an aspiration, not a promise. ;) Please upload bug reports and patches to the sourceforge site at https://sourceforge.net/projects/levent/ . Feature requests and non-bugfix enhancements will be acknowledged but politely deferred till we fork Libevent 2.0 into a stable maintenance branch and start development on 2.1. *** Acknowledgements Many thanks to everybody who contributed code, suggestions, or bug reports to this release, including but absolutely not limited to Niels Provos, Christopher Davis, Trond Norbye, Joachim Bauch, Mike Smellie, Zhou Li, Avi Bab, Scott Lamb, Sebastian Hahn, Felix Nawothnig, Pierre Phaneuf, and Brodie
Re: [Libevent-users] To upgrade or not to upgrade - that is the question
On Tue, Aug 10, 2010 at 7:27 PM, Gilad Benjamini wrote: > We have working Linux application using libevent 1.4 > We are now beginning work on a new version, which would include, among other > things porting our code to Windows. > > Theoretically, it looks like a good point in time to upgrade our libevent to > 2.0 > The question is should we ? > > What are the major benefits of 2.0 ? Hi, Gilad! I just updated the "what's new in Libevent 2.0" document. You can see the latest version here: http://levent.git.sourceforge.net/git/gitweb.cgi?p=levent/levent;a=blob_plain;f=whatsnew-2.0.txt;hb=HEAD . It's still not complete, but it could be a good place to start. You can also peruse the draft book-in-progress at http://www.wangafu.net/~nickm/libevent-book/ ; APIs that are new in 2.0.x are noted throughout. Generally, Libevent 2.0 has better designed APIs (while retaining backwards compatibility), better thread-safety, better portability, better windows support, some key performance improvements, and numerous new features. It's (I think) easier to use, easier to debug with, more tested, and generally better written throughout. Another reason to consider upgrading is that once Libevent 2.0 has been stable for a while, we're not going to backport bugfixes to 1.4 unless they are truly critically important. If you have more specific questions not answered in the documents above, please don't hesitate to ask here. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] To upgrade or not to upgrade - that is the question
On Wed, Aug 11, 2010 at 1:21 AM, Gilad Benjamini wrote: > Nick, > Thanks for the answer and the detailed document. > Out of the different open source projects out there using libevent, which > ones have already moved to 2.0 ? Tor supports some but not all of the new 2.0 stuff, and will move to use most of it with Tor 0.2.3.x. There are a few new projects here and there that use 2.0.x exclusively (the "shim" http proxy, etc), but most old projects still work fine with the 1.4.x APIs. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Include order in Makefile.nmake
On Thu, Aug 12, 2010 at 3:18 PM, Gilad Benjamini wrote: > Makefile.nmake in the tarball has this line > CFLAGS=/Iinclude /Icompat /IWIN32-Code /DWIN32 /DHAVE_CONFIG_H /EHsc /I. > > event-config.h exists in both include/event2 and WIN32-Code/event2 > > Therefore, the file is taken from the wrong place. > I believe the "/IWIN32-Code" should come before "/Iinclude" Hm! There should actually be no event-config.h in include/event2 as shipped; it should only show up there if you previously built using mingw before you tried building with MSVC. The fact that a copy was included with the 2.0.6-rc tarball is itself a bug. I'll try to fix both of these problems, if my automake fu is strong enough today. (I'm calling the Makefile.nmake include order a bug too, since sometimes people _do_ want to test libevent with mingw, then with msvc.) yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Compiling libevent on Windows
On Fri, Aug 13, 2010 at 3:35 AM, Gilad Benjamini wrote: > I tried compiling libevent on Windows, using WinDDK. > I initially had some issues with Makefile.nmake, and then abandoned it in > favor of the simple "sources" file, used by the "build" utility. Hi, Gilad! FYI, it is *way way easier* for us to track bugs and patches and make sure we don't forget about any of them if you upload them to the appropriate tracker at sourceforge: https://sourceforge.net/tracker/?group_id=50884 > There were then a few issues that prevent compilation on Windows. I'm > presenting them as-is and not as a patch, since I don't know Windows well > enough to vouch for a proper patch > - signal handler function are defined as "__cdecl" functions This sounds like it's solvable just by declaring the evsig_handler function to be __cdecl if WIN32 is defined? > - fstat doesn't exist, but _fstat does The usual fix for this one is to include something like #ifdef WIN32 #define fstat _fstat #endif near the start of the appropriate file. > > There was also a decent number of warnings, though nothing major. I'll send > a patch for these later on. Neat! yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] deadlock in version 2.0.5
On Thu, Aug 12, 2010 at 2:31 AM, Avi Bab wrote: > The scenario – Ick. I've added this one to the bugtracker as https://sourceforge.net/tracker/index.php?func=detail&aid=3044479&group_id=50884&atid=461322 ; further discussion there. >From what I can see now, there doesn't seem to be a better solution than moving current_event_lock from being a lock to being a condition or something. More thought is probably needed, though. :/ yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Ability to shutdown() socket with bufferevents?
On Tue, Jul 27, 2010 at 2:42 PM, Corey Stup wrote: >> Right now there's no way to make a bufferevent call shutdown() itself, >> but what if you have a write handler on the client bufferevent just >> call shutdown() manually on the fd to send a FIN when the write buffer >> is empty? You can get the fd with bufferevent_get_fd(). > How can I check to see if the write buffer is empty? Obviously I'd > only want to indicate a shutdown() after all my data had been > processed *and* the buffer is empty. > > Inside my write callback, when I call > struct evbuffer *output = bufferevent_get_output(bev); > len=evbuffer_get_length(output); > > I always get a return length of 0. (I also tried to check the input > buffer, but its 0 as well). I assumed it would indicate how much > data was left in the backing evbuffer that hadn't been written to the > O/S socket buffers. Yes, that's correct. If bufferevent_get_length(bufferevent_get_output(bev)) is 0, then all the data has been written to the TCP stack (with writev or write or sendfile or whatever was appropriate). > In that case I'd want to indicate my shutdown > after I had written all data to the backing evbuffer *and* the > evbuffers were empty. Am I missing something obvious? I don't think so. > Will the flush() sequence I mentioned before be what I'd want to use > once its implemented? I'm not sure. Right now, the behavior of flush() on socket bufferevents is not only unimplemented, but also unspecified. The behavior you mentioned before is one reasonable thing for it to do, I think. > Any timeline if thats the case? (2.1 I assume > since 2.0 features are locked). 2.1 or later. Right now I don't believe anybody has committed to any particular list of features for 2.1, and I don't think we even have a timeline or list of planned stuff for 2.1. It'd be nice to have one. Of course, if somebody were to commit to write a good flush implementation for socket bufferevents, that would improve the odds of it going into 2.1 immensely. ;) peace, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Compiling libevent on Windows
On Fri, Aug 13, 2010 at 3:59 AM, Gilad Benjamini
wrote:
> and one more to end the day...
> event_struct.h has this code
> #ifndef TAILQ_ENTRY
> #define _EVENT_DEFINED_TQENTRY
> #define TAILQ_ENTRY(type) \
> struct { \
> struct type *tqe_next; /* next element */ \
> struct type **tqe_prev; /* address of previous next element */ \
> }
> #endif /* !TAILQ_ENTRY */
>
> Obviously, there is a missing ";" at the end of the struct.
Well, not quite. It would be an error if the code were saying
TAILQ_ENTRY(foo)
TAILQ_ENTRY(abr)
all over, but that's not how TAILQ_ENTRY is used.
See comments on
https://sourceforge.net/tracker/index.php?func=detail&aid=3044492&group_id=50884&atid=461324#
(BTW, when you create artifacts on sourceforge, it helps if you log in
to do it. That way you get email updates whenever somebody comments
on the ticket. You can also do this by logging into sourceforge and
clicking the "monitor" button.)
peace,
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] deadlock in version 2.0.5
On Fri, Aug 13, 2010 at 1:17 PM, Nick Mathewson wrote: > On Thu, Aug 12, 2010 at 2:31 AM, Avi Bab wrote: >> The scenario – > > Ick. I've added this one to the bugtracker as > https://sourceforge.net/tracker/index.php?func=detail&aid=3044479&group_id=50884&atid=461322 > ; further discussion there. > > From what I can see now, there doesn't seem to be a better solution > than moving current_event_lock from being a lock to being a condition > or something. More thought is probably needed, though. :/ With help from Chris Davis, I've got some patches committed to master that stand an okay chance of solving this issue. It would be neat if somebody has a chance to test them out before we release 2.0.7-rc. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Event after close
On Wed, Aug 18, 2010 at 10:44 AM, Gilad Benjamini wrote: > I have a read event on a socket. At some point I realize the connection is > broken. > I delete the event and close the socket. > If the socket had pending data at that point, is there a chance that I will > get a read event ? I.e. an event on a socket that's already closed ? If you have deleted the event, you should never get it. Only events that are "pending" (i.e., added) can get their callbacks called. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] 1.4.13 (and lower): bug in configure: kqueue test-case w/ x86_64
On Thu, Aug 19, 2010 at 1:24 AM, Christopher Layne wrote: [...] > Just by adding a simple "memset(&ev, 0, sizeof ev)" above the initial ev > struct assignments we're doing in the test-case, I see the following > results: [...] > The reason I think this is kind of an annoying bug is that if compiling with > x86_64 on Darwin 10.5, one ends up with poll() behavior as the test-case > claims kqueue is broken (because of the above issue). As far as know > kqueue() doesn't have any significant issues under 10.5 (unlike 10.4). It's > fairly common to use x86_64 - especially when one may be linking against > other x86_64 libraries/binaries. > > I haven't verified differences under 10.6 yet. Seems like an easy enough fix; I've checked in the memset to the 1.4 and 2.0 git branches. Thanks! yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] To upgrade or not to upgrade - that is the question
On Wed, Aug 11, 2010 at 3:16 PM, Thor Lancelot Simon wrote: > On Tue, Aug 10, 2010 at 04:27:47PM -0700, Gilad Benjamini wrote: >> >> What are the major benefits of 2.0 ? > > From my point of view, a lot of convenience code was added which many > large programs already written to the libevent 1.4 APIs would already > have, some APIs were cleaned up, and the library was bloated almost > beyond all recognition. Sorry to hear it; I've heard this offline from a couple of other people too. The APIs do not seem bloaty to me, but I suppose that any API that your projct doesn't need is necessarily bloat to you. As it stands, there's nothing stopping you from just linking with the libevent_core library, and omitting most of the stuff you don't like. The protocol stuff is all in libevent_extra, and the ssl support is in libevent_openssl. Those parts you can just not use if you don't want or need them. The bufferevent code stays in libevent_core (since it was there in 1.4.x) but nothing inherently stops libevent_core from splitting in some future release, maybe into a libevent_base and a libevent_buffered or something. If you're building for a very-low-memory embedded environment, you can also chop out some more code with --disable-thread-support and --disable-malloc-replacement and --disable-debug-mode. I'm amenable to making the libraries even more fine-grained in 2.1 or later if somebody else wants to do the refactoring work on that, or adding configure flags to disable API components (e.g., --disable-bufferevents, --disable-listener) if people would find that useful. The splitting could be a bit tricky, though, since old code that linked with -levent or -levent_core would need to keep working. I'm afraid I don't see the point of splitting the source distribution, though. What would be gained by that, other than having less source to download? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] bufferevent_flush on bufferevent_sock
On Tue, Aug 24, 2010 at 11:14 AM, Andrei Chirila wrote: > Hello, > > I'm using libevent 2.0.6 on a startup project, I just updated from 2.0.5. > I've written a server and I had the surprise, after testing it for a while, > that, for some messages, if I'm using the zlib filter, the last fragment of > data doesn't get send. The current way to solve the problem you describe above is to not close the underlying bufferevent immediately, but rather to disconnect it from the filter, and give it a write callback of its own, to close it once it is finally done writing its data. > Using tcpdump on the port I'm using I can see that > the data never leaves the server. I'm calling in client_write function > bufferevent_flush(client->event), but, as I saw in bufferevent_sock.c, the > flush method is not implemented. Yeah; the behavior for bufferevent_flush() is not currently specified (or implemented) for sockets that connect to the network. For a filtering bufferevent, flushing is no problem: there is another bufferevent, one level lower, that's ready to take all the data you give it, and you can tell bufferevent_flush() to explicitly ignore its high-water mark. But for a socket, it's a bit trickier: there is no guarantee that you actually _can_ flush all the data waiting for a network socket without blocking. (For example, if the amout of data queued in the bufferevent's outbuf exceeds what the kernel can buffer for writing, calling send() on that data will eventually tell you "EAGAIN", unless the user is willing to block until the data is written -- and they probably aren't, since they're using Libevent.) So, having "flush a socket-based bufferevent" mean "write all the data to the network right now" is basically out, since that isn't (in the general case) possible. Here are some other things it *could* mean that wouldn't solve your original problem: * "Write as much data to the network as you can without blocking". * "Write as much data to the network as you can without blocking, and if you empty the buffer, send a FIN packet" > Where should I start writing if I want to write this function? I'd start with specifying what exactly you think it should do. peace, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] loopbreak vs. loopexit
On Sun, Aug 29, 2010 at 2:01 PM, Scott Lamb wrote: [...] > By an active event, I meant one which has triggered and whose callback > needs to be called. [I see this explanation in the docstring for the > activequeues but unless I'm missing it, not anywhere prominent in the > documentation. And unfortunately, the main doxygen page seems to use a > contradictory meaning in the phrase "The event structure must remain > allocated as long as it is active".] I'd love documentation patches to fix this up. I tried to be consistent in the book, but I'm not going to have a chance to do a major editing pass on the doxygen stuff for a while. :/ I've cleaned up the book to avoid saying "Currently processing" and instead say "running callbacks". > event_base_loopbreak(evbase) will cause a running > event_base_loop(evbase) to return without calling any additional > callbacks. In constrast, event_base_loopexit(evbase, NULL) will cause > a running event_base_loop(evbase) to return after calling the > callbacks of all active events (but without blocking again). > > [I'm not being precise about the behavior when multiple threads are > interacting with the same event_base. Can you do that with libevent2? > I think there have been changes regarding threading, but I haven't had > time to follow the details.] You sure can. The semantics for loopbreak and loopexit are still the same as before. [...] > It'd make sense to switch pyevent after it drops support for any > libevent version prior to event_base_loopbreak's introduction. > Probably when it adopts the libevent2 API. event_base_loopbreak first appeared in Libevent 1.4.3 , if my ctags post-processing can be trusted. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Cant compile for armel
On Wed, Sep 1, 2010 at 7:20 AM, Alexander Seibel wrote: > Hi > I am a noob when it comes to compiling. When I want to compile > libevent-1.4.14b-stable for armel, i get errors like this : > > //./libevent-1.4.14b-stable/signal.c:117: undefined reference to > `event_warn' > //./libevent-1.4.14b-stable/signal.c:118: undefined reference to > `event_warn' The "undefined reference to event_warn" thing means that the linker is trying to find event_warn -- which is defined in log.c -- and it isn't there. Maybe for some reason you aren't linking log.c into Libevent when you build it? The thing you linked to below looks like a copy of your config.log file, which explains what the "configure" script is doing. But the error above isn't coming from there; it looks like it's coming as part of your compiler output. Could you please upload somewhere the entire output of the compilation process? Just copy-and-pasting the results of "make" from your terminal should do fine. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Libevent 2.0.6-rc libtool
On Fri, Sep 3, 2010 at 11:59 AM, Ralph Castain wrote: > Hi folks > > I've been working on integrating 2.0.6-rc into the Open MPI code base and ran > across an issue. You distribute an aclocal.m4 that was generated by libtool > 2.2.6b, and you don't whack it at the beginning of autogen.sh. Thus, those of > us who have updated to libtool 2.2.10 get an error. > > If you change your autogen.sh to be nothing more than "autoreconf -ivh", it > will do everything you currently do in autogen.sh -and- regenerate aclocal.m4 > with the current libtool version, thus avoiding this error. Are you sure about the "h" ? I just tried "autoreconf -ivh" on my laptop, and it gave me a help message, claiming that the '-h' switch means "print this help, then exit". Did you maybe mean "-ivf" ? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] A patch to test/bench.c
On Mon, Sep 6, 2010 at 8:09 AM, Shuo Chen wrote:
> Hi there,
>
> I found that test/bench not working in 2.0.6rc. here's a simple fix:
>
> diff --git a/test/bench.c b/test/bench.c
> index 76717d7..de4e814 100644
> --- a/test/bench.c
> +++ b/test/bench.c
> @@ -85,7 +85,8 @@ run_once(void)
> static struct timeval ts, te;
>
> for (cp = pipes, i = 0; i < num_pipes; i++, cp += 2) {
> - event_del(&events[i]);
> + if (events[i].ev_base)
> + event_del(&events[i]);
> event_set(&events[i], cp[0], EV_READ | EV_PERSIST,
> read_cb, (void *) i);
> event_add(&events[i], NULL);
> }
>
Good catch; I'm going to check it in, but using "event_initialized" in
place of looking at the event internals.
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] What's the story with cygevent-1-4-2.dll?
On Mon, Sep 6, 2010 at 1:36 AM, Chris Brown wrote: [...] > The logfile contains the following ... > > /usr/local/sbin/gearmand.exe: error while loading shared libraries: > cygevent-1-4-2.dll: cannot open shared object file: No such file or directory > > Can anyone explain why gearmand.exe runs from the commandline but not as a > service? I'd guess that the DLL search path is different when you're invoking it from the cygwin command line and when it's getting run by the NT service code. This isn't a libevent issue per se. To get you started, you can check out the msdn article at http://msdn.microsoft.com/en-us/library/7d83bc18(VS.80).aspx , and try to figure out which of these options is under your control when running as a service. Alternatively, if disk space isn't too tight, you could just link your program against libevent statically, and ignore the DLL issues completely. > I've tried running the service using my login credentials (just in case there > is something about my environment I'm overlooking) but this fails with a > login error. > > Also, the cygevent-1-4-2 dlls are in /usr/local/bin; is this correct? Somebody else might be able to answer this one; I haven't used cygwin in ages. Re the issues in your other message: it looks like we've got some timing/DNS portability issues on cygwin. None of them look like they're going to break typical programs, but it would be neat if somebody wanted to track them down sometime and fix them. We've been doing our windows testing on mingw and msvc, but this is supposed to be a portability library after all. hth, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Something to consider for future
Hi, Ralph! On Tue, Sep 7, 2010 at 11:04 AM, Ralph Castain wrote: [...] > (b) we need to modify it somewhat to get things working correctly across the > broad range of our installed base. Any chances of getting the patches here upstreamed? "Working correctly across a broad install base" sounds like a feature worth merging into the main distribution. ;) > It is my understanding that quite a few software packages also embed libevent > in their distributions, so this is a fairly common problem. It would make > life much easier, therefore, if you only declared the true public APIs as > public symbols, defining all internal-only functions as "static". Pretty much every symbol that's used only in a single module *is* declared static right now. (If we missed some, that's worth fixing: just send in a patch.) The remaining non-static symbols are ones shared between modules, and of course C doesn't let you make those static. The right solution here is probably to manually tell the linker about what symbols to export and what symbols not to export. On windows, this is the dllexport/dllimport stuff stuff. With gcc4 on other platforms, this is the __attribute__((visibility(...))) stuff. ( http://gcc.gnu.org/wiki/Visibility ) I'd like to move this way in Libevent 2.1, but I'm not sure on the timeframe, since it's a fair amount of work. If somebody commits to writing up and testing the code here, that'll improve the odds of it getting done. peace, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
[Libevent-users] ANN: Libevent 2.0.7-rc is released
Hello again! Thanks to many people's hard work, Libevent 2.0 has now had its second release candidate. You can download it from the Sourceforge site at: https://sourceforge.net/projects/levent/files/libevent/libevent-2.0/libevent-2.0.7-rc.tar.gz/download There's also a pgp signature of it (by me) there, at: https://sourceforge.net/projects/levent/files/libevent/libevent-2.0/libevent-2.0.7-rc.tar.gz.asc/download Don't forget to validate the signature. The complete list of changes is available in the ChangeLog file, included with the distribution. Niels will upload this to monkey.org as soon as he has a chance; he's been super-busy with his work lately, which is why I've been doing the "lead developer" stuff these days. *** What's new in Libevent 2.0.7-rc Thanks to everybody who reported and fixed bugs in Libevent 2.0.6-rc, Libevent 2.0.7-rc should be much more stable and portable, especially for people using IOCP, Windows, rate-limiting, or threads. There are also numerous small bugfixes thoughout the codebase (though still not, alas, in the http stuff). For a complete list of changes, just see the ChangeLog included with the source distribution. *** Fun facts about Libevent 2.0.7-rc: This release changes more lines than 2.0.6-rc, but fewer than any other in the 2.0 series: 2.0.7-rc: 61 files changed, 2459 insertions(+), 685 deletions(-) 2.0.6-rc: 104 files changed, 1749 insertions(+), 1192 deletions(-) 2.0.5-beta: 122 files changed, 3283 insertions(+), 1194 deletions(-) 2.0.4-alpha: 122 files changed, 8112 insertions(+), 4190 deletions(-) 2.0.3-alpha: 98 files changed, 10131 insertions(+), 2213 deletions(-) 2.0.2-alpha: 90 files changed, 5677 insertions(+), 1619 deletions(-) 2.0.1-alpha: 113 files changed, 23909 insertions(+), 7670 deletions(-) The big changes are mostly accounted for by the new condition-variable code that we needed to introduce to solve the deadlock bug discussed on the list last month, by Chris's IOCP fixes, and by new threading-related unit tests. Unit test coverage (for me, on Fedora 13) is at 79.25%, down from 79.76% in Libevent 2.0.5-beta. Let's try to get back up to 80% for the release! *** Status: Libevent 2.0.x-rc and beyond. Libevent 2.0 is *still* in feature-freeze (we won't add any more features to it), hack-freeze (no clever backend rewrites, refactorings, or major performance hacks), and API freeze (all code written to work with the documented APIs of Libevent 2.0.5-beta should continue to work with future versions of Libevent). We might change our minds about any of the above if there turn out to be exceptionally good reasons to do so; this is an aspiration, not a promise. ;) Please upload bug reports and patches to the sourceforge site at https://sourceforge.net/projects/levent/ . Feature requests and non-bugfix enhancements will be acknowledged but politely deferred till we fork Libevent 2.0 into a stable maintenance branch and start development on 2.1. *** Acknowledgements Many thanks to everybody who contributed code, suggestions, or bug reports to this release, including but absolutely not limited to Niels Provos, Christopher Davis, Avi Bab, Gilad Benjamini, Ralph Castain, Shuo Chen, Mihai Draghicioiu, Sebastian Hahn, Christopher Layne, Nicholas Marriott, and Ralf Schmitt. yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] bufferevent_setwatermark() enables reading
On Thu, Sep 9, 2010 at 5:40 PM, Simon Perreault wrote: > On 2010-09-09 17:28, Simon Perreault wrote: >> Would this be the right fix? > > Argh, git-send-email didn't do what I wanted it to do. Here's the patch > in attachment. Sorry. This looks good to me. Merging it. Thanks for tracking this down! BTW, any I could talk you into a regression test for this, so the behavior can't come back in the future? :) yrs, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] ANN: Libevent 2.0.7-rc is released
On Fri, Sep 10, 2010 at 3:55 PM, Mihai Draghicioiu wrote: > Good job. > > However, i seem to be getting a lot of different crashes with the http > code. I'm not sure if it's my application or the libevent code, i'll > have to debug it, but i hope you've planned a http bug hunt until the > next tarball :) I would love to! :) It will depend when the next tarball comes out, though. I have some killer deliverables due for my Real Actual Job at the end of September, so unless my workload goes much better than expected, I'm probably not going to be able to get into http myself before then. So the likeliest-to-get-fixed-soon stuff is probably going to be stuff where the reporter provides not only the bug report, but also a new unit test test case, and ideally a fix too. Barring that, well, maybe my work _will_ go faster than I planned. :/ peace, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Write quirks in default eventbuffer implementation?
On Thu, Sep 9, 2010 at 3:09 PM, Kelly Brock wrote: [...] > So, anyway, it is at the challenge point where I'm getting a little > problem with my initial test. I grab the output evbuffer from event buffer > for the socket and then reserve/write/commit space into the evbuffer. I > then set the EV_WRITE event. At which point write does not happen until > some other event happens. Since the client is patiently waiting for the > server to challenge it, this means nothing ever happens. I added a > persistent timer to the event base and now the write happens when the timer > triggers. > > Now, to somewhat prove that I'm not just doing something bad, the > same code under OsX and Linux seems to function as expected and the data > gets written immediately without needing the extra timer code. I'm just > guessing but maybe the select (I believe?) based version is not working > correctly in this case? That does sound like a bug, if I'm understanding you right. If you get the same issue with 2.0.7-rc, could you maybe open a ticket on the bugtracker [1]? I'd like to get this tracked down. [1] https://sourceforge.net/tracker/?group_id=50884&atid=461322 -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Configure changes
On Tue, Sep 7, 2010 at 2:31 PM, Ralph Castain wrote: > Hi folks > Continuing the discussion re pushing some of Open MPI's experiences > upstream, I have attached our opal_setup_libevent.m4 script for building > libevent as part of OMPI. Note that the comments describe several scenarios > where the existing libevent tests aren't quite adequate for detecting when > various functionality should be "on" vs "off". These have been rather > thoroughly tested by the respective developers, so we offer them to you for > consideration. > I also have added a set of options to your configure.in and associated files > that might prove useful. Nothing profound - they just allow users to > "deselect" certain support options (e.g., DNS, http). I've attached a diff > (options.diff) that shows the changes, though I suspect you'll want to clean > them up. > HTH > Ralph Thanks, Ralph! It would be great if you can upload these to the patch tracker at sourceforge [1] ; I would hate to lose track of them between now and when we fork 2.1.x. Re licensing: The opal_setup_libevent.m4 file has a huge pile of copyright statements on it, but no actual license. FWICT, that means I am have permission to use it. Could you slap one on? It's also pretty clearly made (in parts) by changes to Libevent's configure.in (many of the lines are identical), but it looks like when the preamble got chopped off, Dug Song's credit for the original version got chopped off too. Once that's cleared up, it would be cool to start picking this apart to figure out which parts are improvements or new features and which parts are just porting Libevent's configure.in to be embedded. Anything that constitutes a bug in 2.0.x should get fixed now (if feasible). Anything else can wait for 2.1. Oh, and a random code strategy thought: it seems to me that testing at compile-time for poll and epoll and kqueue behavior is subtly suboptimal. Consider a program built to use one linux kernel where epoll works that's run with another where epoll doesn't work. That's pretty much a poster case for a run-time check. (Yes, I know that libevent already does some compile-time checks for backend correctness, but I'm not sure that they're actually the right thing to do.) In 2.1.x, we should see if we can make more of these checks happen at startup time (as feasible). This should also improve correctness in the cross-compilation case, where you can't do a compile-time check that involves running code. So, to summarize: * Thanks! * Patch tracker [1]. * License. * Let's figure out what's a bugfix. * I think we should migrate to doing fewer "does this backend work" checks at compile-time. [1] https://sourceforge.net/tracker/?group_id=50884&atid=461324 peace and thanks again, -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] epoll erros
On Thu, Sep 16, 2010 at 2:59 PM, Gilad Benjamini
wrote:
> My Linux code, which uses libevent 2.0.x occasionally prints these error
> messages
> Epoll ADD on fd 14 failed. Old events were 0; read change was 1 (add);
> write change was 0 (none).: File exists
> When they appear, it can be as frequent as once a second.
>
> Can someone shed some light on what the issue might be ?
First, make sure you're on 2.0.7-rc, not just any "2.0.x"; there was
an error like this that got fixed in 2.0.6-rc.
So the message means that libevent's epoll backend told the kernel to
start listening for reads on fd 14. Libevent believed that it was not
previously watching fd 14, so it used EPOLL_ADD rather than EPOLL_MOD
to add it. The kernel, though, disagreed, and gave an EEXIST error
("file exists") to say, "no, I was already watching events on that fd.
Two possibilities here: one is that Libevent is confused about which
fds it was listening on. Another one is that something in your code
screwed up Libevent's view of which fds it was listening on.
The first thing to do here is to use event_enable_debug_mode() to have
libevent track event adds/deletes/etc to make sure that there's
nothing screwy going on there.
yrs,
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] Re: help with debugging
On Fri, Sep 17, 2010 at 5:22 AM, Mihai Draghicioiu
wrote:
[...]
> Core was generated by `/home/mihai/frogmod-justice/frogserv'.
> Program terminated with signal 11, Segmentation fault.
> [New process 14829]
> #0 evhttp_request_get_input_buffer (req=0x99d90f8) at http.c:3014
> 3014 {
> (gdb) up
> #1 0x08091a0e in _bufferevent_run_eventcb (bufev=0x20, what=0) at
> bufferevent.c:264
> 264 bufev->errorcb(bufev, what, bufev->cbarg);
> Current language: auto; currently c
> (gdb)
> #2 0x080928cd in bufferevent_connect_getaddrinfo_cb (result=-4,
> ai=0x99e3100, arg=0x99d90f8) at bufferevent_sock.c:444
> 444 _bufferevent_run_eventcb(bev, BEV_EVENT_ERROR);
> (gdb) list 434
Hm. Looks like we're missing a couple levels of stack here. Is it
doable to compile with inlining turned off so we can see what's
calling what going on for real? (-fno-inline should do it under gcc,
I think)
Also, maybe consider running under valgrind if you can. It often
notices screwy memory stuff before a crash actually occurs.
[...]
> It seems that by the time _bufferevent_run_eventcb() is called, its
> bufev argument ends up being 0x20. No idea what to do next.
This might or might not be for real. It could just be that by the
time the debugger is invoked, the memory that used to have 'bufev' has
already in _bufferevent_run_eventcb() has become an unused ("dead")
variable, and so the compiler has decided it can be overwritten by
something else. You could toss in an "assert(bufev != 0x20)" there if
you want to be sure , but I wouldn't expect it to trigger.
hth,
--
Nick
***
To unsubscribe, send an e-mail to majord...@freehaven.net with
unsubscribe libevent-usersin the body.
Re: [Libevent-users] epoll erros
On Fri, Sep 17, 2010 at 3:06 PM, Gilad Benjamini wrote: >> The first thing to do here is to use event_enable_debug_mode() to have >> libevent track event adds/deletes/etc to make sure that there's >> nothing screwy going on there. (Did this turn anything up? You don't need to build with any special options. There is a build option to *disable* debug mode entirely, but it's available by default.) > Here is the chain of events I was able to identify > - An event for fd 14 is created > - At some later point, my code deletes the event. strace does not show a > call to epoll_ctl to delete fd 14 from the set. > - fd 14 is closed by my code > - Shortly afterwards, there is a call to epoll_ctl to delete fd 14 > - The call fails, of course, and a libevent warning says "DEL was > unnecessary". The comment in that place in the code says "... that's fine > too ... " > - It's not fine. AFAIK, since the file descriptor is closed, epoll did not > actually remove it > - Further down the road, attempts to add fd 14 fail since epoll did not > remove fd 14 in the first place. > > It seems like libevent delays the call to epoll_ctl, and by the time it > actually does that, it's too late. But that's not how epoll behaves, I think. As near as I can tell, closing an fd does indeed remove it from the fds watched by an epoll, and so you can indeed re-add a new file with the same fd. I am basing this on actual testing, with the attached test code. It adds a file to an epoll_fd, then closes the file, then arranges for a new file to get made with the same epoll_fd, then adds the new file. It does not get an EEXIST for me. (I tried this on a 2.6.26 kernel and on a 2.6.34 kernel.) If you can come up with a setup where that sequence of events gives an EEXIST, or if you can tweak the test here until it gives an EEXIST, that would help figuring out how the sequence of events described above could cause an EEXIST. (If you re-enable the #if 0'd code, you'll see that deleting a closed fd from an epoll_fd failes with EBADF.) yrs, -- Nick epoll_test.c Description: Binary data
Re: [Libevent-users] Stop listening on http server when accept() returns Too Many Open Files
On Mon, Sep 20, 2010 at 12:22 PM, Simon Perreault wrote: > Does anyone have an answer to that? We just hit this problem in real > life. It's not just theory. > > Thanks, > Simon > > On 2010-09-03 03:49, Yee Keat Phuah wrote: >> Hi, >> >> I am using the http server part of libevent 2.0.6, and currently testing >> it under all sorts of situation. One of the situation I ran into is when >> there are too many connections, there might be "Too Many Open Files" >> returned from the accept() call, that is ok, but the problem is because >> in listener.c:listener_read_cb() function, there's no way I could >> intercept this error message, and maybe stop accepting new connections, >> what happen subsequently is that event_sock_warn just gets called >> thousands of time. >> >> Is there a way that I can intercept this error and act accordingly? >> Probably stop listening for a while? >> >> Thanks in advance. >> >> Cheers, >> Phuah Yee Keat It looks like we need to add a way for listeners to report errors to the user, rather than warning, and we need to make http respect this error. One challenge is that we've promised not to break existing programs that ignore listener errors, so whatever change we make can't be backwards-incompatible. Here are two possible APIs; I'm not sure which is better. One possibility is that, when a non-retriable error occurs, the callback is invoked with fd set to -1, and errno (LastSocketError) set to the appropriate error. This behavior would break programs that don't expect listeners to ever call the listener callback with a bad fd, so it would have to be off by default, and enabled via an LEV_OPT_* option. Another possibility is to allow code to register a separate error callback for each listener that would get invoked when an error occurred. This seems a little cleaner to me. Thoughts? -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Stop listening on http server when accept() returns Too Many Open Files
On Mon, Sep 20, 2010 at 12:50 PM, Simon Perreault wrote: > Thanks a lot for your reply! > > On 2010-09-20 12:29, Nick Mathewson wrote: >> One possibility is that, when a non-retriable error occurs, the >> callback is invoked with fd set to -1, and errno (LastSocketError) set >> to the appropriate error. This behavior would break programs that >> don't expect listeners to ever call the listener callback with a bad >> fd, so it would have to be off by default, and enabled via an >> LEV_OPT_* option. >> >> Another possibility is to allow code to register a separate error >> callback for each listener that would get invoked when an error >> occurred. This seems a little cleaner to me. > > I'd be fine with both, but I would also prefer the second option. A > separate callback for errors is how the rest of libevent already works. > > Would the attached patch work? Looks okay to me. Probably, you'd want to rename "ecb" to "errorcb" or something; we don't use "ecb" to mean "error callback" anywhere else. Is there any way to write a unit test for this? I don't see a good one beyond passing in a bogus fd to provoke an ENOTSOCK or something like that, which is not the exact situation we're facing here, but which at least would make sure the callback is getting run properly. *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] Stop listening on http server when accept() returns Too Many Open Files
On Mon, Sep 20, 2010 at 2:09 PM, Simon Perreault wrote: > On 2010-09-20 13:53, Nick Mathewson wrote: >>> Would the attached patch work? >> >> Looks okay to me. Probably, you'd want to rename "ecb" to "errorcb" >> or something; we don't use "ecb" to mean "error callback" anywhere >> else. > > Sure, no problem. Also, I noticed a small typo: > > -void evconnlistner_set_error_cb(struct evconnlistener *lev, > +void evconnlistener_set_error_cb(struct evconnlistener *lev, Thanks. Should I expect a revised patch, or should I try to remember to fix this up myself. >> Is there any way to write a unit test for this? I don't see a good >> one beyond passing in a bogus fd to provoke an ENOTSOCK or something >> like that, which is not the exact situation we're facing here, but >> which at least would make sure the callback is getting run properly. > > Having no idea what your test harness is capable of, couldn't one set > ulimit -n some_low_value, start a listener, create connections until the > error callback is called, and then ...? Ick; that approach *would* work, but it's mostly launched from C, so instead of doing a ulimit -n, you're looking at a setrlimit, which implies that we need to do the necessary autoconf stuff to see if we _have_ getrlimit, etc. I guess if we've gotta, we've gotta. :p -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] epoll erros
2010/9/21 Gilad Benjamini : [...] > Any chance you can make something out of the attached log ? That's pretty helpful! It's almost but not quite enough information to figure out what's up here. There are a couple of epoll debugging messages that don't give enough detail. I just added a couple more detailed debugging messages in the git master branch to help try to figure out what exactly we're doing to epoll fd, and why. If you can get the same output with the new stuff from git master, that would be great. (Also, was this with invoking event_enable_debug_mode() or not?) -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
Re: [Libevent-users] epoll erros
On Wed, Sep 22, 2010 at 11:53 AM, Gilad Benjamini wrote: >> >> 2010/9/21 Gilad Benjamini : >> [...] >> > Any chance you can hmake something out of the attached log ? >> >> That's pretty helpful! It's almost but not quite enough information >> to figure out what's up here. There are a couple of epoll debugging >> messages that don't give enough detail. >> >> I just added a couple more detailed debugging messages in the git >> master branch to help try to figure out what exactly we're doing to >> epoll fd, and why. If you can get the same output with the new stuff >> from git master, that would be great. > > > I can, but it would take me a few days. > In the mean time, I do have a hunch, based on reading the code. > In epoll_apply_changes, when precautionary_add is turned on, based on > equality between old and new events, shouldn't that only be done if their > value is non-zero ? I'm not sure how that part of the code could be reached with new_events equal to zero. It's inside an if block that gets invoked when at least one of read_change and write_change is EV_CHANGE_ADD. Given that, I think we're bound to call one of the blocks that sets EV_READ or EV_WRITE in new_events. That said, you could try adding an EVUTIL_ASSERT(new_events != 0) inside the block that turns on precautionary add, and see whether it ever triggers. If it does, we'll know I'm wrong. -- Nick *** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-usersin the body.
