[issue40485] Provide an abstraction for a select-able Event
Faidon Liambotis added the comment: Thanks so much for all the work on os.eventfd(), it's exciting to see it come to fruition. An eventfd variant of Event (compatible with the threading & multiprocessing APIs) is now as simple as: class Event: _ONE = (1).to_bytes(8, byteorder=sys.byteorder) def __init__(self): self._event_fd = os.eventfd(0, os.EFD_NONBLOCK) self._selector = selectors.DefaultSelector() self._selector.register(self._event_fd, selectors.EVENT_READ) def is_set(self): return self.wait(timeout=0) def set(self): try: os.write(self._event_fd, self._ONE) except BlockingIOError: pass def clear(self): try: os.read(self._event_fd, 8) except BlockingIOError: pass def wait(self, timeout=None): return bool(self._selector.select(timeout=timeout)) def fileno(self): return self._event_fd Given this now has a fileno() method, it is now possible to wait for the event as part of a broader selector, among other events (e.g. a file or socket becoming available to read or write). I don't know where (or how) such a variant would fit into stdlib. It's simpler and more lightweight (less fds) than both threading's and multiprocessing's, and could be used from both threads and processes. I'd love some guidance here. (If a maintainer or anyone else reading this wants to use the above code in a PR, feel free -- no copyright claimed or expected for this trivial piece of code above) -- ___ Python tracker <https://bugs.python.org/issue40485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40485] Provide an abstraction for a select-able Event
Faidon Liambotis added the comment:
I missed that there is now also an os.eventfd_{write,read}(), so in the above
os.write(self._event_fd, self._ONE)
can become
os.eventfd_write(self._event_fd, 1)
and:
os.read(self._event_fd, 8)
can become:
os.eventfd_read(self._event_fd)
(the _ONE declaration will then be unnecessary and can be removed)
--
___
Python tracker
<https://bugs.python.org/issue40485>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40485] Provide an abstraction for a select-able Event
Faidon Liambotis added the comment: Not sure if I understand the question! I'd like to see that happen, I don't particularly care if it makes to 3.10 or a later version, although of course the earlier the better :) As an idea of a path forward, would it make sense to take the code above and push it to multiprocessing as an alternative to Event with an if hasattr(os, "eventfd") guard? -- ___ Python tracker <https://bugs.python.org/issue40485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40485] Provide an abstraction for a select-able Event
New submission from Faidon Liambotis : In certain codebases, it's useful to be able to wait for input from one or more file descriptors (e.g. a socket), while at the same time waiting for an event triggered by another thread, or perhaps multiprocessing process. To wait for one or more file descriptors to get ready, the select module can be used. However, neither threading.Event() nor multiprocessing.Event() are select-able, i.e. they provide no fileno() method. The standard way one can implement this on Unix is with os.pipe(), but it can be tricky (forgetting to use non-blocking I/O etc.). It is also limited to a pair of processes at a time. On Linux systems from the past decade, one can also implement this much more efficiently using the eventfd() system calls. I think similar functionality exists in other Unixes with kqueue etc. It'd be great if stdlib provided an abstraction over this mechanism. In fact, multiprocessing.Event() itself could probably be a thin abstraction over eventfd() on Linux? Perhaps even multiprocessing.Semaphore with EFD_SEMAPHORE, although I admit I'm less familiar with how all that works. (Select-able Queues would be even neater, but that's a story for a different issue :) -- components: Extension Modules messages: 367975 nosy: paravoid priority: normal severity: normal status: open title: Provide an abstraction for a select-able Event type: enhancement versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue40485> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
