On Tue, May 27, 2014 at 10:34 AM, Andrius Bentkus <[email protected]> wrote: > Hey guys! > > I was lurking around on linux and found an article about creation filesystem > creation which ultimately led to the discovery that linux has aio_read and > aio_write which are asynchronous versions for write and read for files. Here > is the entire man page http://man7.org/linux/man-pages/man7/aio.7.html > > There seems to be also http://man7.org/linux/man-pages/man2/io_submit.2.html > > Further interesting links talking about this: > http://stackoverflow.com/questions/20973754/linux-aio-poor-scaling , > https://code.google.com/p/kernel/wiki/AIOUserGuide . Is there a reason why > this is not being used in libuv?
aio(7) and io_submit(2) are two different things. aio(7) is a POSIX standard for asynchronous file I/O and quite limited in scope; it supports asynchronous reads, writes and fsync. On Linux, it's implemented in glibc using a thread pool, if it's implemented at all. You can compile glibc without AIO support and in that case the aio_*() functions are stubs that raise ENOSYS. To make matters more confusing, there's also a libaio but that library does _not_ implement the POSIX AIO functions. It's a thin wrapper around the kernel's native io_*() system calls. libaio's raison d'être is that glibc doesn't export wrappers for them. io_submit() and friends have severe limitations. They require direct I/O (bad for read-heavy workloads) and have been buggy in the past: kernel memory leaks, data races, silently falling back to synchronous I/O, etc. I have investigated (several times, actually) the feasibility of using native AIO in libuv but I haven't been able to get to a point where I was confident that I could make it would work reliably under all circumstances. If I were to try again, I would probably sniff the kernel version and fall back to traditional thread pool-based AIO if the kernel is e.g. < 3.12 (exact version TBD). However, embedded kernels are quite often nothing like the mainline kernel so that might complicate matters. It's going to require a fair amount of code and may involve chicanery like whitelisting/blacklisting file paths based on what statfs() returns. The kernel's asynchronous I/O infrastructure is, in theory, unified but, in practice, YMMV. I'd initially restrict it to btrfs, ext[34], XFS and maybe JFS and NFS. Hope that answers your question, Andrius. -- You received this message because you are subscribed to the Google Groups "libuv" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/libuv. For more options, visit https://groups.google.com/d/optout.
