On Tue, Jul 20, 2010 at 10:47 PM, Anthony Liguori <anth...@codemonkey.ws> wrote: > On 07/20/2010 01:34 PM, Ot ten Thije wrote: >> >> Hello, >> >> I am working on fixing the savevm/loadvm functionality in the Android >> emulator, and the two issues I've encountered so far both appear to stem >> from the asynchronous I/O (AIO) code. In both cases, the emulator busy-waits >> indefinitely for an operation that never signals completion. >> >> Unfortunately I am not really familiar with AIO, so I was hoping one of >> the emulator devs could point me some resources (design docs, general >> introduction, etc.). I've done some searching myself and found some docs for >> the Linux kernel AIO implementation >> (http://lse.sourceforge.net/io/aio.html), but I'm not sure to what extent it >> applies to the QEMU code. >> >> Tips for debugging AIO would also be greatly appreciated. I can trace the >> execution until I am within the (emulated) device driver (i.e. >> block/qcow2.c:qcow_aio_writev()), but haven't been able to pinpoint the >> exact location where the actual async call is made. This makes it difficult >> to identify the code that should signal completion back to the main process >> (and apparently fails to do so). I know this code is called though, because >> some asynchronous calls *do* signal completion. > > TCG translates guest code into small sequences of host code (basic blocks). > These basic blocks can be chained together such that one block directly > jmps to the next block. The effect is that a guest can run a tight loop > whereas guest code continuously runs without a chance for QEMU to do any > work. > > To allow qemu to make forward progress in such a scenario, we program > signals to fire. Currently, the signals fire in a number of circumstances > including when AIO operations complete, or when a periodic timer needs to > fire. > > When dealing with multiple threads, it's very easy to screw things up by not > masking signals properly. Often times, this is hidden because the periodic > timer runs often enough that it doesn't matter if you miss a signal. An > exception, however, would be emulation of synchronous code. This tends to > happen in qcow2 metadata operations since they are still synchronous. To > complete this emulation, we have to block the current thread until the I/O > operation completes. But since qemu isn't re-entrant, we can't run the full > main loop as that could trigger re-entrancy in qcow2. To work around this, > we implement "idle bottom halves" which are special bottom halves that are > run by the normal io loop but also by a special I/O used exclusive for > emulating synchronous writes. > > To further complicate matters, non-x86 platforms (like ARM) are more likely > to not use a periodic timer which makes these bugs much more obvious. >
Thank you for the explanation, this makes the high-level structure of the system more understandable. However, as I replied to Stefan just now, I still have some difficulty finding the exact code that is executed while the main loop busy-waits. That is, the code executed asynchronously that should raise the signal indicating that the operation completed. Could you point me to the code doing this in mainline QEMU? Hopefully I can use that to find the corresponding code in the Android emulator. Regards, Ot ten Thije >> >> I realize that the Android emulator is a rather heavy fork of QEMU, so >> giving specific advice will probably be difficult. However, the overall >> approach is still the same, so I hope you can help me get a better >> understanding of that. > > This is the problem with forking. This is very hairy code that requires > careful attention to detail. If you're introducing any type of threading, > disk emulation, or changes to the block subsystem, chances are you've done > it wrong. > > Regards, > > Anthony Liguori > >> >> Ot ten Thije > > >
