Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-31 Thread Roni Luxenberg
- Original Message - > Il 30/05/2012 14:34, Geert Jansen ha scritto: > > > > On 05/29/2012 02:52 PM, Paolo Bonzini wrote: > > > >>> Does the drive-mirror coroutine send the writes to the target in > >>> the > >>> same order as they are sent to the source? I assume so. > >> > >> No, it doe

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-31 Thread Paolo Bonzini
Il 31/05/2012 13:08, Geert Jansen ha scritto: > 1) Target flushes are not guaranteed to happen at all. If the latency of > the target is higher than the maximum interval between writes to the > source, the bitmap will always be dirty when a write to the target > returns, and a target flush will nev

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-31 Thread Geert Jansen
On 05/30/2012 05:06 PM, Paolo Bonzini wrote: I think it's beginning to dawn on me that what you have is correct, when i combine this: 2) target flushes do not have to coincide with a source flush. Writes after the last source flush _can_ be inconsistent between the source and the destinatio

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-31 Thread ronnie sahlberg
On Fri, May 25, 2012 at 11:25 PM, Paolo Bonzini wrote: > Il 25/05/2012 14:09, Stefan Hajnoczi ha scritto: >>> > >>> > Perhaps that be simply a new qemu-img subcommand?  It should be possible >>> > to run it while the VM is offline.  Then the file that is produced could >>> > be fed to blockdev-dir

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-31 Thread Paolo Bonzini
Il 31/05/2012 10:44, Roni Luxenberg ha scritto: > a continuous replication application expects to get all IOs in the same order > as > issued by the guest irrespective of the (rate of the) flushes done within the > guest. Does real hardware give such consistency, unless you disable caching? > T

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-30 Thread Paolo Bonzini
Il 30/05/2012 14:34, Geert Jansen ha scritto: > > On 05/29/2012 02:52 PM, Paolo Bonzini wrote: > >>> Does the drive-mirror coroutine send the writes to the target in the >>> same order as they are sent to the source? I assume so. >> >> No, it doesn't. It's asynchronous; for continuous replicatio

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-30 Thread Geert Jansen
On 05/29/2012 02:52 PM, Paolo Bonzini wrote: Does the drive-mirror coroutine send the writes to the target in the same order as they are sent to the source? I assume so. No, it doesn't. It's asynchronous; for continuous replication, the target knows that it has a consistent view whenever it

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-29 Thread Paolo Bonzini
Il 29/05/2012 13:57, Geert Jansen ha scritto: > I assume the target can be any QEmu block driver including e.g. NBD? A > networked block driver would be required for a continuous replication > solution. Yes. > Does the drive-mirror coroutine send the writes to the target in the > same order as th

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-29 Thread Geert Jansen
Hi, On 05/24/2012 04:19 PM, Paolo Bonzini wrote: Here is how the bitmaps are handled when doing I/O on the source: - after writing to the source: - clear bit in the volatile in-flight bitmap - set bit in the persistent dirty bitmap - after flushing the source: - msync the persistent b

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Luiz Capitulino
On Thu, 24 May 2012 15:41:29 +0200 Paolo Bonzini wrote: > * block-stream: I would still like to add on_error to the existing > block-stream command, if only to ease unit testing. Concerns about the > stability of the API can be handled by adding introspection (exporting > the schema), which is n

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Eric Blake
On 05/25/2012 02:48 AM, Paolo Bonzini wrote: >>> * block-job-complete: force completion of mirroring and switching of the >>> device to the target, not related to the rest of the proposal. >>> Synchronously opens backing files if needed, asynchronously completes >>> the job. >> >> Can this be made

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Paolo Bonzini
Il 25/05/2012 14:09, Stefan Hajnoczi ha scritto: >> > >> > Perhaps that be simply a new qemu-img subcommand? It should be possible >> > to run it while the VM is offline. Then the file that is produced could >> > be fed to blockdev-dirty-enable. > For both continuous replication and incremental

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Stefan Hajnoczi
On Fri, May 25, 2012 at 01:17:04PM +0200, Paolo Bonzini wrote: > Il 25/05/2012 11:43, Stefan Hajnoczi ha scritto: > > On Thu, May 24, 2012 at 03:41:29PM +0200, Paolo Bonzini wrote: > >> Persistent dirty bitmap > >> === > >> > >> A persistent dirty bitmap can be used by managemen

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Paolo Bonzini
Il 25/05/2012 11:43, Stefan Hajnoczi ha scritto: > On Thu, May 24, 2012 at 03:41:29PM +0200, Paolo Bonzini wrote: >> Persistent dirty bitmap >> === >> >> A persistent dirty bitmap can be used by management for two reasons. >> When mirroring is used for continuous replication of

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Stefan Hajnoczi
On Thu, May 24, 2012 at 03:41:29PM +0200, Paolo Bonzini wrote: > Persistent dirty bitmap > === > > A persistent dirty bitmap can be used by management for two reasons. > When mirroring is used for continuous replication of storage, to record > I/O operations that happened while

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Paolo Bonzini
Il 24/05/2012 17:32, Dor Laor ha scritto: > I didn't understand whether the persistent dirty bitmap needs to be > flushed. This bitmap actually control the persistent known state of the > destination image. Since w/ mirroring we always have the source in full > state condition, we can choose to laz

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Paolo Bonzini
Il 24/05/2012 18:57, Eric Blake ha scritto: > On 05/24/2012 07:41 AM, Paolo Bonzini wrote: >> changes from v1: >> - added per-job iostatus >> - added description of persistent dirty bitmap >> >> The same content is also at >> http://wiki.qemu.org/Features/LiveBlockMigration/1.2 >> > >> * query-blo

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Kevin Wolf
Am 25.05.2012 10:28, schrieb Stefan Hajnoczi: > On Thu, May 24, 2012 at 03:41:29PM +0200, Paolo Bonzini wrote: >> changes from v1: >> - added per-job iostatus >> - added description of persistent dirty bitmap >> >> The same content is also at >> http://wiki.qemu.org/Features/LiveBlockMigration/1.2

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-25 Thread Stefan Hajnoczi
On Thu, May 24, 2012 at 03:41:29PM +0200, Paolo Bonzini wrote: > changes from v1: > - added per-job iostatus > - added description of persistent dirty bitmap > > The same content is also at > http://wiki.qemu.org/Features/LiveBlockMigration/1.2 > > > QMP changes for error handling >

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-24 Thread Eric Blake
On 05/24/2012 07:41 AM, Paolo Bonzini wrote: > changes from v1: > - added per-job iostatus > - added description of persistent dirty bitmap > > The same content is also at > http://wiki.qemu.org/Features/LiveBlockMigration/1.2 > > * query-block-jobs: BlockJobInfo gets two new fields, paused and

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-24 Thread Dor Laor
On 05/24/2012 05:19 PM, Paolo Bonzini wrote: Il 24/05/2012 16:00, Ori Mamluk ha scritto: The dirty bitmap is managed by these QMP commands: * blockdev-dirty-enable: takes a file name used for the dirty bitmap, and an optional granularity. Setting the granularity will not be supported in the

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-24 Thread Paolo Bonzini
Il 24/05/2012 16:00, Ori Mamluk ha scritto: > >> The dirty bitmap is managed by these QMP commands: >> >> * blockdev-dirty-enable: takes a file name used for the dirty bitmap, >> and an optional granularity. Setting the granularity will not be >> supported in the initial version. >> >> * query-bl

Re: [Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-24 Thread Ori Mamluk
On 24/05/2012 16:41, Paolo Bonzini wrote: The dirty bitmap is managed by these QMP commands: * blockdev-dirty-enable: takes a file name used for the dirty bitmap, and an optional granularity. Setting the granularity will not be supported in the initial version. * query-block-dirty: returns sta

[Qemu-devel] Block job commands in QEMU 1.2 [v2, including support for replication]

2012-05-24 Thread Paolo Bonzini
changes from v1: - added per-job iostatus - added description of persistent dirty bitmap The same content is also at http://wiki.qemu.org/Features/LiveBlockMigration/1.2 QMP changes for error handling == * query-block-jobs: BlockJobInfo gets two new fields, paused an