Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-09-03 Thread Stefan Hajnoczi
On Mon, Sep 02, 2013 at 02:57:23PM +0200, Benoît Canet wrote: > > I don't see the point of using hashes. > Using hashes means that at least one extra read will be done on the target to > compute the candidate target hash. > It's bad for a cloud provider where IOs count is a huge cost. > > Another

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-09-02 Thread Benoît Canet
I don't see the point of using hashes. Using hashes means that at least one extra read will be done on the target to compute the candidate target hash. It's bad for a cloud provider where IOs count is a huge cost. Another structure to replace a bitmap (smaller on the canonical case) would be a bl

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-24 Thread Dietmar Maurer
> Maybe the best approach is to maintain a dirty bitmap while the guest is > running, which is fairly cheap. Then you can use the dirty bitmap to only > hash > modified clusters when building the Merkle tree - this avoids reading the > entire disk image. Yes, this is an good optimization. > > >

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-24 Thread Stefan Hajnoczi
On Thu, May 23, 2013 at 08:11:42AM +, Dietmar Maurer wrote: > > > I also consider it safer, because you make sure the data exists (using > > > hash keys > > like SHA1). > > > > > > I am unsure how you can check if a dirty bitmap contains errors, or is > > > out of > > date? > > > > > > Also,

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-23 Thread Dietmar Maurer
> > I also consider it safer, because you make sure the data exists (using hash > > keys > like SHA1). > > > > I am unsure how you can check if a dirty bitmap contains errors, or is out > > of > date? > > > > Also, you can compare arbitrary Merkle trees, whereas a dirty bitmap is > > always > re

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-23 Thread Stefan Hajnoczi
On Wed, May 22, 2013 at 03:34:18PM +, Dietmar Maurer wrote: > > That sounds like more work than a persistent dirty bitmap. The advantage > > is that > > while dirty bitmaps are consumed by a single user, the Merkle tree can be > > used > > to sync up any number of replicas. > > I also consi

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-22 Thread Dietmar Maurer
> That sounds like more work than a persistent dirty bitmap. The advantage is > that > while dirty bitmaps are consumed by a single user, the Merkle tree can be used > to sync up any number of replicas. I also consider it safer, because you make sure the data exists (using hash keys like SHA1).

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-22 Thread Dietmar Maurer
> > That way you can also implement async replication to a remote site (like MS > do). > > Sounds like rsync. yes, but we need 'snapshots' and something more optimized (rsync compared the whole files). I think this can be implemented using the backup job with a specialized backup driver.

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-22 Thread Stefan Hajnoczi
On Tue, May 21, 2013 at 10:58:47AM +, Dietmar Maurer wrote: > > >> True, but that would happen only in case the host crashes. Even for > > >> a QEMU crash the changes would be safe, I think. They would be > > >> written back when the persistent dirty bitmap's mmap() area is > > >> unmapped, d

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-21 Thread Dietmar Maurer
> >> True, but that would happen only in case the host crashes. Even for > >> a QEMU crash the changes would be safe, I think. They would be > >> written back when the persistent dirty bitmap's mmap() area is > >> unmapped, during process exit. > > > > I'd err on the side of caution, mark the per

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-21 Thread Paolo Bonzini
Il 21/05/2013 12:34, Stefan Hajnoczi ha scritto: > On Tue, May 21, 2013 at 10:30:22AM +0200, Paolo Bonzini wrote: >> Il 21/05/2013 09:31, Stefan Hajnoczi ha scritto: >>> On Mon, May 20, 2013 at 09:23:43AM +0200, Paolo Bonzini wrote: Il 20/05/2013 08:24, Stefan Hajnoczi ha scritto: >>> You

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-21 Thread Stefan Hajnoczi
On Tue, May 21, 2013 at 10:30:22AM +0200, Paolo Bonzini wrote: > Il 21/05/2013 09:31, Stefan Hajnoczi ha scritto: > > On Mon, May 20, 2013 at 09:23:43AM +0200, Paolo Bonzini wrote: > >> Il 20/05/2013 08:24, Stefan Hajnoczi ha scritto: > > You only need to fdatasync() before every guest flush, n

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-21 Thread Paolo Bonzini
Il 21/05/2013 09:31, Stefan Hajnoczi ha scritto: > On Mon, May 20, 2013 at 09:23:43AM +0200, Paolo Bonzini wrote: >> Il 20/05/2013 08:24, Stefan Hajnoczi ha scritto: > You only need to fdatasync() before every guest flush, no? >>> No, you need to set the dirty bit before issuing the write on th

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-21 Thread Stefan Hajnoczi
On Tue, May 21, 2013 at 11:25:01AM +0800, Wenchao Xia wrote: > 于 2013-5-17 17:14, Stefan Hajnoczi 写道: > >On Fri, May 17, 2013 at 02:58:57PM +0800, Wenchao Xia wrote: > >>于 2013-5-16 15:47, Stefan Hajnoczi 写道: > >>>On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: > After checking

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-21 Thread Stefan Hajnoczi
On Mon, May 20, 2013 at 09:23:43AM +0200, Paolo Bonzini wrote: > Il 20/05/2013 08:24, Stefan Hajnoczi ha scritto: > >> > You only need to fdatasync() before every guest flush, no? > > No, you need to set the dirty bit before issuing the write on the > > host. Otherwise the image data may be modifi

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-20 Thread Wenchao Xia
于 2013-5-17 17:14, Stefan Hajnoczi 写道: On Fri, May 17, 2013 at 02:58:57PM +0800, Wenchao Xia wrote: 于 2013-5-16 15:47, Stefan Hajnoczi 写道: On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: After checking the code, I found it possible to add delta data backup support also, If an a

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-20 Thread Paolo Bonzini
Il 20/05/2013 08:24, Stefan Hajnoczi ha scritto: >> > You only need to fdatasync() before every guest flush, no? > No, you need to set the dirty bit before issuing the write on the > host. Otherwise the image data may be modified without setting the > appropriate dirty bit. That would allow data

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-19 Thread Stefan Hajnoczi
On Fri, May 17, 2013 at 12:17 PM, Paolo Bonzini wrote: > Il 16/05/2013 09:47, Stefan Hajnoczi ha scritto: >> On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: >>> After checking the code, I found it possible to add delta data backup >>> support also, If an additional dirty bitmap was

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-17 Thread Paolo Bonzini
Il 16/05/2013 09:47, Stefan Hajnoczi ha scritto: > On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: >> After checking the code, I found it possible to add delta data backup >> support also, If an additional dirty bitmap was added. > > I've been thinking about this. Incremental backu

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-17 Thread Stefan Hajnoczi
On Fri, May 17, 2013 at 02:58:57PM +0800, Wenchao Xia wrote: > 于 2013-5-16 15:47, Stefan Hajnoczi 写道: > >On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: > >> After checking the code, I found it possible to add delta data backup > >>support also, If an additional dirty bitmap was adde

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-17 Thread Wenchao Xia
于 2013-5-16 15:47, Stefan Hajnoczi 写道: On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: After checking the code, I found it possible to add delta data backup support also, If an additional dirty bitmap was added. I've been thinking about this. Incremental backups need to know w

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-16 Thread Stefan Hajnoczi
On Thu, May 16, 2013 at 02:16:20PM +0800, Wenchao Xia wrote: > After checking the code, I found it possible to add delta data backup > support also, If an additional dirty bitmap was added. I've been thinking about this. Incremental backups need to know which blocks have changed, but keeping a

Re: [Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-15 Thread Wenchao Xia
于 2013-5-15 22:34, Stefan Hajnoczi 写道: > Note: These patches apply to my block-next tree. You can also grab the code > from git here: > git://github.com/stefanha/qemu.git block-backup-core > > This series adds a new QMP command, drive-backup, which takes a point-in-time > snapshot of a block devi

[Qemu-devel] [PATCH v3 0/8] block: drive-backup live backup command

2013-05-15 Thread Stefan Hajnoczi
Note: These patches apply to my block-next tree. You can also grab the code from git here: git://github.com/stefanha/qemu.git block-backup-core This series adds a new QMP command, drive-backup, which takes a point-in-time snapshot of a block device. The snapshot is copied out to a target block d