> On Apr 23, 2018, at 11:12, Jason Madden <[email protected]> wrote: > >> On Apr 23, 2018, at 09:55, Jameson Nash <[email protected]> wrote: >> >> Instead of task switching deep inside some callback, could you run the libuv >> event processing as a separate task, via `uv_run(uv_default_loop, >> UV_RUN_ONCE)`, then task switch from outside the library? That would let >> libuv finish updating all of its internal structures and remove itself from >> the stack before anything else happens. > > The short answer is that yes, that could probably be done. I thought about it > before and concluded it would be a lot of work on gevent's side and it might > be slow. But maybe it's worth it. > > In order to do something like that, gevent would have to implement a second > callback queue on top of the libuv loop. For each prepare/check/io/timer/idle > watcher that libuv ran, instead of it actually calling the callback the > application needed, libuv would run a gevent callback that caused gevent to > queue the real application callback on gevent's internal queue. Then when > libuv returned control, gevent would have to start iterating over that > internal queue and running the application callbacks. (Application callbacks > can do anything, including arbitrary coroutine switches.) > > That's probably a viable approach, it just isn't the way gevent is structured > now.
[snip] > I think I'll take another look at that option and try to get a better idea > about the complexity and overhead. Thanks for reminding me! I've just released gevent 1.3b2 that implements this policy, so the patch to libuv is no more! Performance was not really measurably different for the benchmarks we have, and the implementation complexity wasn't too bad either. There is one small downside, and that's the need to run callbacks both before and after the loop blocks to poll for IO. That is, the timers that expired at the top of the loop and were queued need to be run before the arbitrary amount of time the loop blocks for IO. We currently do this by running queued callbacks in a prepare watcher as well as after uv_run(UV_RUN_ONCE) returns. That prepare watcher means that in order to preserve the property that no manipulation of on stack pointers can happen, we disabled all user-level access to prepare watchers (there's only one prepare callback, the system callback, and we never stop it so the QUEUE is never manipulated during arbitrary coroutine switches). We can emulate prepare watchers at the gevent level if necessary, so that's not a big issue. (We already emulate some libev behaviour in gevent on libuv, e.g., fork and child watchers, so there's precedent.) Thanks everyone for the productive discussion and suggestions. Jason -- You received this message because you are subscribed to the Google Groups "libuv" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/libuv. For more options, visit https://groups.google.com/d/optout.
