Am 23.02.2017 um 20:34 schrieb John Snow: > > On 02/23/2017 09:29 AM, Peter Lieven wrote: >> Am 22.02.2017 um 22:17 schrieb John Snow: >>> On 02/22/2017 03:45 AM, Peter Lieven wrote: >>>> Am 21.02.2017 um 22:13 schrieb John Snow: >>>>> On 02/21/2017 07:43 AM, Peter Lieven wrote: >>>>>> Hi, >>>>>> >>>>>> >>>>>> is there anyone ever thought about implementing something like VMware >>>>>> CBT in Qemu? >>>>>> >>>>>> >>>>>> https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1020128 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Peter >>>>>> >>>>>> >>>>> A bit outdated now, but: >>>>> http://wiki.qemu-project.org/Features/IncrementalBackup >>>>> >>>>> and also a summary I wrote not too far back (PDF): >>>>> https://drive.google.com/file/d/0B3CFr1TuHydWalVJaEdPaE5PbFE >>>>> >>>>> and I'm sure the Virtuozzo developers could chime in on this subject, >>>>> but basically we do have something similar in the works, as eblake >>>>> says. >>>> Hi John, Hi Erik, >>>> >>>> thanks for your feedback. Are you both the ones working primary on >>>> this topic? >>>> If there is anything to review or help needed, please let me know. >>>> >>> I've been working on incremental backups; Fam and I now co-maintain >>> block/dirty-bitmap.c. >>> >>> Vladimir Sementsov-Ogievskiy has been working on bitmap persistence and >>> migration from Virtuozzo; as well as the NBD specification amendment to >>> allow us to fleece images with dirty bitmaps. >>> >>> (Check the wiki and the whitepaper I linked!) >>> >>> Eric has been guiding the review process for the NBD side of things. >>> >>>> My 2 cents: >>>> I thing I had in mind if there is no image fleecing available, but >>>> fetching the dirty bitmap >>>> from external would be a feauture to put a write lock on a block device. >>>> Write lock means, drain all pending writes and queue all further >>>> writes until unlock (as if they >>>> were throttled to zero). This could help fetch consistent backups >>>> from storage device (thinking of iSCSI SAN) without >>>> the help of the hypervisor to actually transfer data (no load in the >>>> frontend network or the host). What would further >>>> be needed is a write generation for each block, not just only a dirty >>>> bitmap. >>>> >>>> In this case something like this via QMP (and external software) >>>> should work: >>>> ---8<--- >>>> gen = write generation of last backup (or 0 for full backup) >>>> do { >>>> nextgen = fetch current write generation (via QMP) >>> As Eric said, there's a lot of hostility to using QMP as a metadata >>> transmission protocol. >>> >>>> dirtymap = send all block whose write generation is greater >>>> than 'gen' (via QMP) >>>> dirtycnt = 0 >>>> foreach block in dirtymap { >>>> copy to backup via external software >>>> dirtycnt++ >>>> } >>>> gen = nextgen >>>> } while (dirtycnt < X) <--- to achieve this a thorttling or >>>> similar might be needed >>>> >>>> fsfreeze (optional) >>>> write lock (via QMP) >>>> backupgen = fetch current write generation (via QMP) >>>> dirtymap = send all block whose write generation is greater than >>>> 'gen' (via QMP) >>>> foreach block in dirtymap { >>>> copy to backup via external software >>>> } >>>> unlock (via QMP) >>>> fsthaw (optional) >>>> --->8--- >>>> >>>> As far as I understand CBT in VMware is not just only a dirty bitmap, >>>> but also a write generation tracking for blocks (size 64kb or whatever) >>>> >>> I think at the moment I'm worried about getting the basic features out >>> the door, but I'm not opposed to adding fancier features if there's >>> justification or demand for them. >> Sure, the basic features are most important. I was just thinking of the >> above scenario to interact with a NAS and have Qemu's "help" >> to create incremental backups. >> >> Peter > If you get the chance to read the white paper I linked to you, please > let me know which use cases we might not be able to cover that you feel > other programs might offer.
Will do. Only have had a short glimpse yet. > > I can also make a point to CC you on future upstream discussions as they > happen. Yes, please. Peter
