Re: [PATCH] file-posix: Fix aio=threads performance regression after enablign FUA

2025-06-25 Thread Eric Blake
ao > Signed-off-by: Kevin Wolf > --- > block/file-posix.c | 29 +++-- > 1 file changed, 15 insertions(+), 14 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 12/12] block/io_uring: use non-vectored read/write when possible

2025-06-23 Thread Eric Blake
x27;t hurt to have > this optimization in place. > > Suggested-by: Eric Blake > Signed-off-by: Stefan Hajnoczi > --- > block/io_uring.c | 29 - > 1 file changed, 24 insertions(+), 5 deletions(-) > > diff --git a/block/io_uring.c b/block/io_uring.c

Re: [PATCH v2 08/12] aio-posix: gracefully handle io_uring_queue_init() failure

2025-06-23 Thread Eric Blake
e in the codebase). > - It's easier to move the #ifdefs from aio-posix.h to aio-posix.c. > > Signed-off-by: Stefan Hajnoczi > --- > util/aio-posix.h | 12 ++-- > util/aio-posix.c | 29 ++--- > util/fdmon-io_uring.c | 8 ---- &g

Re: [PATCH v2 01/12] aio-posix: fix race between io_uring CQE and AioHandler deletion

2025-06-23 Thread Eric Blake
ff-by: Stefan Hajnoczi > --- > util/fdmon-io_uring.c | 26 +++--- > 1 file changed, 19 insertions(+), 7 deletions(-) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 1/4] block/graph-lock: fix typo in comment

2025-06-17 Thread Eric Blake
On Wed, Jun 04, 2025 at 02:07:14PM +0200, Fiona Ebner wrote: > Signed-off-by: Fiona Ebner > --- > include/block/graph-lock.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Eric Blake > > diff --git a/include/block/graph-lock.h b/include/block/gr

Re: [PATCH 18/18] qapi: add cross-references to misc modules

2025-06-16 Thread Eric Blake
+- > qapi/introspect.json | 22 +++--- > qapi/misc-arm.json | 4 ++-- > qapi/misc-i386.json | 2 +- > qapi/misc.json | 6 +++--- > qapi/stats.json | 8 > 7 files changed, 23 insertions(+), 23 deletions(-) > Reviewed-by: Eric Blake --

Re: [PATCH 14/18] qapi: add cross-references to sockets.json

2025-06-16 Thread Eric Blake
On Fri, Jun 13, 2025 at 04:36:16PM -0400, John Snow wrote: > Signed-off-by: John Snow > --- > qapi/sockets.json | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualizatio

Re: [PATCH 03/18] qapi: add cross-references to block layer

2025-06-16 Thread Eric Blake
0 ++--- > 4 files changed, 128 insertions(+), 128 deletions(-) Detecting whether you missed links is harder; but for the links you include here... Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 4/4] iotests/244: Add test cases for keep_data_file

2025-06-12 Thread Eric Blake
s simpler (= will always skip create) and seems safer (users may > +# expect the file to exist, and the error will warn them when it does not). > +_make_test_img \ > +-o "data_file=$TEST_IMG.doesnotexist,keep_data_file=on,data_file_raw=on" > \ > +1M I agree w

Re: [PATCH 3/4] iotests/common.filter: Sort keep_data_file

2025-06-12 Thread Eric Blake
deletion(-) Reviewed-by: Eric Blake > > diff --git a/tests/qemu-iotests/common.filter > b/tests/qemu-iotests/common.filter > index fc3c64bcb8..3744326d2d 100644 > --- a/tests/qemu-iotests/common.filter > +++ b/tests/qemu-iotests/common.filter > @@ -176,7 +176,7 @@ _do_filte

Re: [PATCH 2/4] qcow2: Simplify size round-up in co_create_opts

2025-06-12 Thread Eric Blake
On Fri, May 30, 2025 at 10:44:45AM +0200, Hanna Czenczek wrote: > Use the now-existing qcow2_opts pointer to simplify the size rounding up > code. > > Signed-off-by: Hanna Czenczek > --- > block/qcow2.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) Re

Re: [PATCH v2 0/2] reject qcow2 creation with protocol in data_store

2025-06-12 Thread Eric Blake
On Tue, May 27, 2025 at 09:32:43AM -0500, Eric Blake wrote: > v1 was here: > https://lists.gnu.org/archive/html/qemu-devel/2025-05/msg05581.html > > Since then: > - split into two patches > - also tweak vmdk [Kevin] > - rewrite commit messages for better justification [K

Re: [PATCH 1/4] qcow2: Add keep_data_file command-line option

2025-06-12 Thread Eric Blake
t; +if (!qcow2_opts->has_data_file_raw) { > +qcow2_opts->data_file_raw = false; > +} > + > +if (keep_data_file && > + qcow2_opts->preallocation != PREALLOC_MODE_OFF && > +qcow2_opts->preallocation != PREALLOC_MODE_METADATA) > +{ > +error_setg(errp, "Preallocating more than only metadata would " > + "overwrite the external data file's content and is " > + "therefore incompatible with '%s=on'", > + BLOCK_OPT_KEEP_DATA_FILE); I like this safety check. Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 12/21] block: Move qemu_fcntl_addfl() into osdep.c

2025-06-04 Thread Eric Blake
On Wed, Jun 04, 2025 at 03:28:04PM +0200, Hanna Czenczek wrote: > Move file-posix's helper to add a flag (or a set of flags) to an FD's > existing set of flags into osdep.c for other places to use. > > Suggested-by: Eric Blake > Signed-off-by: Hanna Czenczek > ---

[PULL 5/6] iotests: Improve mirror-sparse on ext4 and xfs

2025-05-29 Thread Eric Blake
Fiona Ebner Reported-by: Markus Armbruster Fixes: c0ddcb2c ("tests: Add iotest mirror-sparse for recent patches") Signed-off-by: Eric Blake Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner Message-ID: <20250523163041.2548675-7-ebl...@redhat.com> [eblake: Also fix xfs issue] Sign

[PULL 6/6] iotests: Filter out ZFS in several tests

2025-05-29 Thread Eric Blake
Reported-by: Fiona Ebner Signed-off-by: Eric Blake Message-ID: <20250523163041.2548675-8-ebl...@redhat.com> Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner --- tests/qemu-iotests/common.rc| 30 + tests/qemu-iotests/106 | 1 + te

[PULL 1/6] io: Add helper for setting socket send buffer size

2025-05-29 Thread Eric Blake
: Nir Soffer Message-ID: <20250517201154.88456-2-nir...@gmail.com> Reviewed-by: Daniel P. Berrangé Signed-off-by: Eric Blake --- include/io/channel-socket.h | 13 + io/channel-socket.c | 11 +++ 2 files changed, 24 insertions(+) diff --git a/include/io/channel

[PULL 3/6] nbd: Set unix socket send buffer on Linux

2025-05-29 Thread Eric Blake
456-4-nir...@gmail.com> Reviewed-by: Daniel P. Berrangé Signed-off-by: Eric Blake --- nbd/common.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nbd/common.c b/nbd/common.c index 9436e9d1d14..2a133a66c39 100644 --- a/nbd/common.c +++ b/nbd/common.c @@ -271,8 +271,9 @@

[PULL 2/6] nbd: Set unix socket send buffer on macOS

2025-05-29 Thread Eric Blake
1.555 | | 524288 | 1.271 | 4.528 | 1.224 | | 1048576 | 1.294 | 4.565 | 1.333 | | 2097152 | 1.299 | 4.569 | 1.344 | | 4194304 | 1.291 | 4.559 | 1.327 | Signed-off-by: Nir Soffer Message-ID: <20250517201154.88456-3-nir...@gmail.com> Reviewed-by:

[PULL 4/6] iotests: Use disk_usage in more places

2025-05-29 Thread Eric Blake
Commit be9bac07 added a utility disk_usage function, but there are a couple of other tests that could also use it. Signed-off-by: Eric Blake Message-ID: <20250523163041.2548675-6-ebl...@redhat.com> Reviewed-by: Fiona Ebner Tested-by: Fiona Ebner --- tests/qemu-iotests/125 | 2 +- test

[PULL 0/6] NBD patches through 2025-05-29

2025-05-29 Thread Eric Blake
NBD - Eric Blake: clean up mirror-sparse iotest issues ---- Eric Blake (3): iotests: Use disk_usage in more places iotests: Improve mirror-sparse on ext4 and xfs iotests: Filter out ZFS in several tests Nir Soffer (3):

Re: [PATCH] iotests: fix 240

2025-05-29 Thread Eric Blake
t; tests/qemu-iotests/240.out | 4 +--- > 2 files changed, 1 insertion(+), 5 deletions(-) This one has been annoying me - thanks! Reviewed-by: Eric Blake Tested-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [RFC 11/11] block/io_uring: use aio_add_sqe()

2025-05-29 Thread Eric Blake
ted in the sq ring until the end of the event loop iteration >and there are fewer io_uring_enter(2) syscalls. > 2. Less code duplication. > > Signed-off-by: Stefan Hajnoczi > --- Comments below, but looks sane to me. Reviewed-by: Eric Blake > include/block/aio.h

Re: [RFC 10/11] aio-posix: avoid EventNotifier for cqe_handler_bh

2025-05-29 Thread Eric Blake
ood >solution. > > So in the end I moved the qemu_bh_schedule() call from fdmon-io_uring.c > to aio-posix.c. It's ugly but straightforward. > > Signed-off-by: Stefan Hajnoczi > --- > util/aio-posix.c | 11 +++ > util/fdmon-io_uring.c | 11 ++

Re: [RFC 09/11] aio-posix: add aio_add_sqe() API for user-defined io_uring requests

2025-05-29 Thread Eric Blake
pending CQE callbacks. If you're > wondering why CqeHandler exists instead of just a callback function > pointer, this is why. > > Signed-off-by: Stefan Hajnoczi > --- Large patch. I found a couple of nits, but the overall design looks sound. Revi

Re: [RFC 09/11] aio-posix: add aio_add_sqe() API for user-defined io_uring requests

2025-05-28 Thread Eric Blake
iles changed, 200 insertions(+), 37 deletions(-) I'll have to resume my review tomorrow. (I'm having fun learning about io_uring while reviewing this) -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [RFC 08/11] aio-posix: gracefully handle io_uring_queue_init() failure

2025-05-28 Thread Eric Blake
}; > > -bool fdmon_io_uring_setup(AioContext *ctx) > +void fdmon_io_uring_setup(AioContext *ctx, Error **errp) > { > int ret; > > @@ -369,15 +370,14 @@ bool fdmon_io_uring_setup(AioContext *ctx) > > ret = io_uring_queue_init(FDMON_IO_URING_ENTRIES, &ctx-

Re: [RFC 07/11] aio: add errp argument to aio_context_setup()

2025-05-28 Thread Eric Blake
/block/aio.h | 3 ++- > util/aio-posix.c| 2 +- > util/aio-win32.c| 2 +- > util/async.c| 7 ++- > 4 files changed, 10 insertions(+), 4 deletions(-) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [RFC 06/11] aio: free AioContext when aio_context_new() fails

2025-05-28 Thread Eric Blake
QEMUBH *bh; > unsigned flags; > > +if (!ctx->initialized) { > + return; > +} > + > thread_pool_free_aio(ctx->thread_pool); Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [RFC 05/11] aio: remove aio_context_use_g_source()

2025-05-28 Thread Eric Blake
> Signed-off-by: Stefan Hajnoczi > --- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [RFC 04/11] aio-posix: integrate fdmon into glib event loop

2025-05-28 Thread Eric Blake
> --- > include/block/aio.h | 36 ++ > util/aio-posix.h | 5 +++ > tests/unit/test-aio.c | 7 +++- > util/aio-posix.c | 69 - > util/fdmon-epoll.c| 52 ++--- > util/fdmon-io_uring.c | 44 ++++++

Re: [RFC 03/11] tests/unit: skip test-nested-aio-poll with io_uring

2025-05-28 Thread Eric Blake
o it is safe to > include "util/aio-posix.h". > > Signed-off-by: Stefan Hajnoczi > --- > tests/unit/test-nested-aio-poll.c | 7 +++ > 1 file changed, 7 insertions(+) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [RFC 02/11] aio-posix: keep polling enabled with fdmon-io_uring.c

2025-05-28 Thread Eric Blake
f (timeout || ctx->fdmon_ops->need_wait(ctx)) { > ^^ > > Polling will be disabled even when timeout == 0. > > Extend the optimization to handle the case when need_wait() returns true > and timeout == 0. > > Cc: Chao Gao > Signed-off-by: Stefan H

Re: [RFC 01/11] aio-posix: fix polling mode with fdmon-io_uring

2025-05-28 Thread Eric Blake
he host kernel, the multi-shot > POLL_ADD operation completes with -ECANCELED. Adjust the code slightly > to take this into account. > > Signed-off-by: Stefan Hajnoczi > --- Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 2/3] iotests: Improve mirror-sparse on ext4

2025-05-28 Thread Eric Blake
On Wed, May 28, 2025 at 10:45:35AM -0500, Eric Blake wrote: > On Fri, May 23, 2025 at 11:27:22AM -0500, Eric Blake wrote: > > Fiona reported that an ext4 filesystem on top of LVM can sometimes > > report over-allocation to du (based on the hueristics the filesystem > > is m

Re: [PULL 13/14] tests: Add iotest mirror-sparse for recent patches

2025-05-28 Thread Eric Blake
t), du reports 3702 * 1k sectors or exactly 3M of disk usage. On the bright side, it's an easy fix: either s/-lt/-le/ or s/3/4/ in mirror-sparse for the bound of what forms a sparse file (we can easily explain how even though less than 3M is reported as a hole, disk usage can plausibly round its answers up and list every megabyte of content that contains at least some data, while still being an "accurate" measure of how sparse the file is). -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 2/3] iotests: Improve mirror-sparse on ext4

2025-05-28 Thread Eric Blake
On Fri, May 23, 2025 at 11:27:22AM -0500, Eric Blake wrote: > Fiona reported that an ext4 filesystem on top of LVM can sometimes > report over-allocation to du (based on the hueristics the filesystem > is making while observing the contents being mirrored); even though > the content

Re: [PULL 13/14] tests: Add iotest mirror-sparse for recent patches

2025-05-28 Thread Eric Blake
On Wed, May 28, 2025 at 01:39:25PM +0200, Markus Armbruster wrote: > Eric Blake writes: > > > Prove that blockdev-mirror can now result in sparse raw destination > > files, regardless of whether the source is raw or qcow2. By making > > this a separate test, it was pos

Re: [PATCH] file-posix: Tolerate unaligned hole at middle

2025-05-28 Thread Eric Blake
as follows (where lowercase respresents changed results due to rounding to alignment): |+++|+++|+++|+++|+++| DDDH DDDd ^ But the important question I don't see in your patch is whether you handle a block_status query at the point of ^ correctly (it

Re: [PULL 13/14] tests: Add iotest mirror-sparse for recent patches

2025-05-28 Thread Eric Blake
On Wed, May 28, 2025 at 01:39:25PM +0200, Markus Armbruster wrote: > Eric Blake writes: > > > Prove that blockdev-mirror can now result in sparse raw destination > > files, regardless of whether the source is raw or qcow2. By making > > this a separate test, it was pos

Re: [PATCH v2 1/4] nbd: Add multi-conn option

2025-05-27 Thread Eric Blake
;format-specific": {"type": "rbd", "data": {"encryption-format":"..."}}}, note the extra layer of nesting, due to historical reasons of being added in a time when the QMP generator was not as nice on supporting flatter union-style coding. Must I create an ImageInfoSpecificNbdWrapper type, with the sole purpose of having the same (pointless, IMO) "data":{} wrapper as all the other branches of the union type, or am I okay with my addition using the flatter style? -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 4/4] docs/about/removed-features: Move removal notes to tidy up order

2025-05-27 Thread Eric Blake
moved-features.rst | 60 - > 1 file changed, 30 insertions(+), 30 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 3/4] docs/about/deprecated: Move deprecation notes to tidy up order

2025-05-27 Thread Eric Blake
out/deprecated.rst | 32 > 1 file changed, 16 insertions(+), 16 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 1/4] docs/about: Belatedly document tightening of QMP device_add checking

2025-05-27 Thread Eric Blake
to update > documentation. Do that now. > > Cc: Stefan Hajnoczi > Signed-off-by: Markus Armbruster -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

[PATCH v2 1/2] block: Allow drivers to control protocol prefix at creation

2025-05-27 Thread Eric Blake
), but it will enable the next patch to cater to drivers that want to differ in behavior for the primary image vs. any secondary images that are opened at the same time as creating the primary image. Signed-off-by: Eric Blake --- include/block/block-global-state.h | 3 ++- block.c

[PATCH v2 0/2] reject qcow2 creation with protocol in data_store

2025-05-27 Thread Eric Blake
v1 was here: https://lists.gnu.org/archive/html/qemu-devel/2025-05/msg05581.html Since then: - split into two patches - also tweak vmdk [Kevin] - rewrite commit messages for better justification [Kevin] Eric Blake (2): block: Allow drivers to control protocol prefix at creation qcow2

[PATCH v2 2/2] qcow2, vmdk: Restrict creation with secondary file using protocol

2025-05-27 Thread Eric Blake
st:10809/': No such file or directory with datastore_nbd.qcow2 no longer created. Signed-off-by: Eric Blake --- block/qcow2.c | 2 +- block/vmdk.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index f1ed03c543e..bcf4d920946 100644 --- a/blo

Re: [PATCH v2 3/3] iotests: Filter out ZFS in several tests

2025-05-27 Thread Eric Blake
On Mon, May 26, 2025 at 11:42:24AM +0200, Fiona Ebner wrote: > Am 23.05.25 um 18:27 schrieb Eric Blake: > > Fiona reported that ZFS makes sparse file testing awkward, since: > > - it has asynchronous allocation (not even 'fsync $file' makes du see > > the desired s

Re: [PATCH 00/13] qapi: Doc fixes and improvements

2025-05-27 Thread Eric Blake
cluded For the series: Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH] qcow2: Forbid use of protocol: prefix on data_file

2025-05-23 Thread Eric Blake
On Fri, May 23, 2025 at 01:20:32PM -0500, Eric Blake wrote: > Ever since CVE-2024-4467 (see commit 7ead9469 in qemu v9.1.0), we have > intentionally treated command-line arguments as local files, and not > protocol specifications (you have to specify backing files with > full-blown

[PATCH] qcow2: Forbid use of protocol: prefix on data_file

2025-05-23 Thread Eric Blake
only place where we change the former default of true. Signed-off-by: Eric Blake --- include/block/block-global-state.h | 3 ++- block.c| 4 ++-- block/crypto.c | 2 +- block/parallels.c | 2 +- block/qcow.c | 2

Re: [PATCH v2 2/3] iotests: Improve mirror-sparse on ext4

2025-05-23 Thread Eric Blake
On Fri, May 23, 2025 at 11:27:22AM -0500, Eric Blake wrote: > Fiona reported that an ext4 filesystem on top of LVM can sometimes > report over-allocation to du (based on the hueristics the filesystem and despite my other edits, I still missed fixing my typo on heuristics > is mak

[PATCH v2 2/3] iotests: Improve mirror-sparse on ext4

2025-05-23 Thread Eric Blake
was too large by 4k, failing the test. In auditing other iotests, this is a common problem we've had to deal with. Reported-by: Fiona Ebner Fixes: c0ddcb2c ("tests: Add iotest mirror-sparse for recent patches") Signed-off-by: Eric Blake Reviewed-by: Fiona Ebner Tested-by: Fiona

[PATCH v2 3/3] iotests: Filter out ZFS in several tests

2025-05-23 Thread Eric Blake
Reported-by: Fiona Ebner Signed-off-by: Eric Blake --- tests/qemu-iotests/common.rc| 30 + tests/qemu-iotests/106 | 1 + tests/qemu-iotests/175 | 1 + tests/qemu-iotests/221 | 1 + tests/qe

[PATCH v2 0/3] iotests improvements for mirror-sparse and others

2025-05-23 Thread Eric Blake
on ext4 when I intentionally tweak the common.rc code to fail failure); and this only addresses the subset of tests in Fiona's email where I could obviously see some sort of probing of image sizes, and not every test mentioned. Eric Blake (3): iotests: Use disk_usage in more places iotests: Improve m

[PATCH v2 1/3] iotests: Use disk_usage in more places

2025-05-23 Thread Eric Blake
Commit be9bac07 added a utility disk_usage function, but there are a couple of other tests that could also use it. Signed-off-by: Eric Blake --- tests/qemu-iotests/125 | 2 +- tests/qemu-iotests/308 | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/125

Re: [PATCH v2 1/4] nbd: Add multi-conn option

2025-05-23 Thread Eric Blake
7;s always a way to tell the actual number of connections chosen. > Somewhat similar to what Richard pointed out at in nbdcopy. Warnings in QMP are difficult. It's easy to fail a command, but hard to print non-fatal messages. However, I'm not opposed to having a way to use a QMP query-* command as a followup for the user who is curious on how many connections a given block export is using (either we can already query all existing block exports, or that's something that I should add independent of the new multi-conn setting). -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

[PATCH] iotests: Improve mirror-sparse on various filesystems

2025-05-22 Thread Eric Blake
at may remain a sporadic problem for another day. Reported-by: Fiona Ebner Fixes: c0ddcb2c ("tests: Add iotest mirror-sparse for recent patches") Signed-off-by: Eric Blake --- tests/qemu-iotests/common.rc | 2 ++ tests/qemu-iotests/tests/mirror-sparse | 4 +++- 2 files ch

Re: [PATCH v2 1/4] nbd: Add multi-conn option

2025-05-22 Thread Eric Blake
On Thu, May 22, 2025 at 08:38:34PM +0300, Andrey Drobyshev wrote: > On 4/28/25 9:46 PM, Eric Blake wrote: > > From: "Richard W.M. Jones" > > > > Add multi-conn option to the NBD client. This commit just adds the > > option, it is not functional. > &g

Re: [PATCH v2 4/4] nbd: Enable multi-conn using round-robin

2025-05-22 Thread Eric Blake
On Thu, May 22, 2025 at 08:37:58PM +0300, Andrey Drobyshev wrote: > On 4/28/25 9:46 PM, Eric Blake wrote: > > From: "Richard W.M. Jones" > > > > Enable NBD multi-conn by spreading operations across multiple > > connections. > > > > (XXX) This

Re: [PULL 13/14] tests: Add iotest mirror-sparse for recent patches

2025-05-21 Thread Eric Blake
On Wed, May 21, 2025 at 11:54:03AM +0200, Fiona Ebner wrote: > Hi, > > Am 15.05.25 um 04:28 schrieb Eric Blake: > > +do_test full ignore off full > > for me, this one fails irregularly (more than 50% of the time) with: > -Destination is full; expected full > +Desti

Re: [PATCH 3/4] block/copy-before-write: reverse access bitmap

2025-05-20 Thread Eric Blake
1 for avoid) and no inverting is needed before or after the merge, or the two sources have opposite sense (in which case, I would have expected inverting one of the bitmaps before the merge to get them to agree on sense, then merging, then inverting back to the desired sense). Am I missing something? -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 2/4] hbitmap: introduce hbitmap_reverse()

2025-05-20 Thread Eric Blake
ving the order). The naming change will require respinning the series, but the concept makes sense. -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH 1/4] hbitmap: drop meta bitmap leftovers

2025-05-20 Thread Eric Blake
On Tue, May 13, 2025 at 03:32:35AM +0200, Andrey Zhadchenko wrote: > API to manipulate meta bitmap was removed with commit 0c88f1970c76 > > Signed-off-by: Andrey Zhadchenko > --- > util/hbitmap.c | 17 ++--- > 1 file changed, 2 insertions(+), 15 deletions(-) Revie

Re: [PATCH v2 24/24] block: add bdrv_graph_wrlock_drained() convenience wrapper

2025-05-20 Thread Eric Blake
ore the call, the former requires that the > variable is a counter and not just a boolean. > -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 07/24] block: mark bdrv_child_change_aio_context() GRAPH_RDLOCK

2025-05-20 Thread Eric Blake
GRAPH_RDLOCK. > > Signed-off-by: Fiona Ebner > --- > -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 06/24] block: mark change_aio_ctx() callback and instances as GRAPH_RDLOCK(_PTR)

2025-05-20 Thread Eric Blake
GRAPH_RDLOCK. > > Signed-off-by: Fiona Ebner > --- > -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 00/24] block: do not drain while holding the graph lock

2025-05-20 Thread Eric Blake
get more details on the failure as you see it, so I can work on skipping the test when the filesystem does not support the test at hand? (It might be better to reply in context to the pull request that introduced the file [2]) [2] https://lists.gnu.org/archive/html/qemu-devel/2025-05/msg03623.ht

Re: [PATCH v4 1/3] io: Add helper for setting socket send buffer size

2025-05-19 Thread Eric Blake
". (Or > "Experiments" if you really wanted a plural noun.) I'll tweak that while staging the patch for a pull request; no need for another revision. -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v4 0/3] nbd: Increase unix socket buffer size

2025-05-19 Thread Eric Blake
same issue on other platforms. It should be easy to enable > this change for more platform by defining UNIX_STREAM_SOCKET_SEND_BUFFER_SIZE. Thanks; I'm staging this through my NBD tree. -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH] docs/interop: convert text files to restructuredText

2025-05-15 Thread Eric Blake
s beyond 2 EB (61 bits); in > -the worst case, with a 512 cluster size and refcount_order of 6, it is > -unable to access beyond 32 GB (35 bits). > +.. note:: > +qemu has an implementation limit of 8 MB as the maximum refcount Should we be changing s/qemu/QEMU/ while editing this file? -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 2/2] qcow2: put discards in discard queue when discard-no-unref is enabled

2025-05-15 Thread Eric Blake
upond > --- > block/qcow2-cluster.c | 16 ++-- > block/qcow2-refcount.c | 17 + > block/qcow2.h | 4 > 3 files changed, 27 insertions(+), 10 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org

Re: [PATCH v2 1/2] qcow2: rename update_refcount_discard to queue_discard

2025-05-15 Thread Eric Blake
On Tue, May 13, 2025 at 03:26:27PM +0200, Jean-Louis Dupond wrote: > The function just queues discards, and doesn't do any refcount change. > So let's change the function name to align with its function. > > Signed-off-by: Jean-Louis Dupond > --- Reviewed-by: Er

[PULL 01/14] block: Expand block status mode from bool to flags

2025-05-14 Thread Eric Blake
, a bit more complicated to prove if the raw file was created with 'qemu-img create' since we intentionally allocate a small chunk of all-zero data to help with alignment probing). Later patches will add a generic algorithm for seeing if an entire file reads as zeroes. Signed-off-by:

[PULL 07/14] mirror: Pass full sync mode rather than bool to internals

2025-05-14 Thread Eric Blake
o s->is_none_mode=false, it is better to pass s->sync_mode throughout the entire operation. For active commit, the desired semantics match sync mode TOP. Signed-off-by: Eric Blake Message-ID: <20250509204341.3553601-22-ebl...@redhat.com> Reviewed-by: Sunny Zhu Reviewed-by: Stefan Hajnoczi ---

[PULL 12/14] iotests/common.rc: add disk_usage function

2025-05-14 Thread Eric Blake
Garcia Message-ID: <20240913163942.423050-6-andrey.drobys...@virtuozzo.com> Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-ID: <20250509204341.3553601-27-ebl...@redhat.com> --- tests/qemu-iotests/common.rc | 6 ++ tests/qemu-iotests/250 | 5 - 2 file

[PULL 13/14] tests: Add iotest mirror-sparse for recent patches

2025-05-14 Thread Eric Blake
; for raw, it is the entire file). But the important part is that the destination file ends up smaller than 3M, rather than the 20M it used to be before this patch series. Signed-off-by: Eric Blake Message-ID: <20250509204341.3553601-28-ebl...@redhat.com> Reviewed-by: Stefan Hajnoczi ---

[PULL 02/14] file-posix, gluster: Handle zero block status hint better

2025-05-14 Thread Eric Blake
uiting and advertising full allocation. This change will be utilized in later patches to let mirroring optimize for the case when the destination already reads as zeroes. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-ID: <20250509204341.3553601-17-ebl...@redhat.com> --- block/

[PULL 08/14] mirror: Allow QMP override to declare target already zero

2025-05-14 Thread Eric Blake
uot;target-is-zero":true is designed for optimizations, it is okay to silently ignore the parameter rather than erroring if the user ever sets the parameter in a scenario where the mirror job can't exploit it (for example, when doing "sync":"top" instead of "

[PULL 14/14] mirror: Reduce I/O when destination is detect-zeroes:unmap

2025-05-14 Thread Eric Blake
: Eric Blake Message-ID: <20250513220142.535200-2-ebl...@redhat.com> Reviewed-by: Stefan Hajnoczi --- block/mirror.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 724318f0371..c2c5099c951 100644 --- a/block/mirror.c

[PULL 03/14] block: Let bdrv_co_is_zero_fast consolidate adjacent extents

2025-05-14 Thread Eric Blake
ncated and didn't match the incoming bytes. While refactoring this into a loop, note that there is no need to assign pnum prior to calling bdrv_co_common_block_status_above() (it is guaranteed to be assigned deeper in the callstack). Signed-off-by: Eric Blake Reviewed-by: Stefan Hajno

[PULL 06/14] mirror: Minor refactoring

2025-05-14 Thread Eric Blake
er to initialize to -EIO. An upcoming patch wants to track two bitmaps in do_sync_target_write(); this will be easier if the current variables related to the dirty bitmap are renamed. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-ID: <20250509204341.3553601-21-ebl...@re

[PULL 11/14] mirror: Skip writing zeroes when target is already zero

2025-05-14 Thread Eric Blake
opened with "detect-zeroes": "unmap". And any time writing zeroes is skipped, the job counters are not incremented. Signed-off-by: Eric Blake Message-ID: <20250509204341.3553601-26-ebl...@redhat.com> Reviewed-by: Stefan Hajnoczi --- block/mirror.c | 107 +++

[PULL 09/14] mirror: Drop redundant zero_target parameter

2025-05-14 Thread Eric Blake
of sync ripped out of blockdev.c, the sync mode is passed through the full callstack unmolested, and we can now reliably reconstruct the same settings as what used to be passed in by zero_target=false, without the redundant parameter. Signed-off-by: Eric Blake Message-ID: <20250509204341.3553601-2

[PULL 05/14] iotests: Improve iotest 194 to mirror data

2025-05-14 Thread Eric Blake
Mirroring a completely sparse image to a sparse destination should be practically instantaneous. It isn't yet, but the test will be more realistic if it has some non-zero to mirror as well as the holes. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi Message-ID: <20250509204341

[PULL 10/14] mirror: Skip pre-zeroing destination if it is already zero

2025-05-14 Thread Eric Blake
shell filters _filter_block_job_{offset,len} in common.filter). A later test will also be added which further validates expected sparseness, so it does not matter that 194 is no longer explicitly looking at how many bytes were copied. Signed-off-by: Eric Blake Message-ID: <20250509204341.3553

[PULL 00/14] NBD patches for 2025-05-14

2025-05-14 Thread Eric Blake
2025-05-14 for you to fetch changes up to aff46b4bf556430dd3c12fa39a457f0487bb0053: mirror: Reduce I/O when destination is detect-zeroes:unmap (2025-05-14 21:27:04 -0500) NBD patches for 2025-05-14 - Eric Blake: fix blockdev-mi

[PULL 04/14] block: Add new bdrv_co_is_all_zeroes() function

2025-05-14 Thread Eric Blake
re complexity in a common manner for all block devices - when the BDS is mostly a hole, but has a small non-hole header, it is still worth the time to read that header and check if it reads as all zeroes before giving up and returning a pessimistic answer. Signed-off-by: Eric Blake Reviewed-by: St

Re: [PATCH v4 09/13] mirror: Drop redundant zero_target parameter

2025-05-14 Thread Eric Blake
On Wed, May 14, 2025 at 05:09:14PM -0500, Eric Blake wrote: > On Fri, May 09, 2025 at 03:40:26PM -0500, Eric Blake wrote: > > The two callers to a mirror job (drive-mirror and blockdev-mirror) set > > zero_target precisely when sync mode == FULL, with the one exception > > th

Re: [PATCH v4 09/13] mirror: Drop redundant zero_target parameter

2025-05-14 Thread Eric Blake
On Fri, May 09, 2025 at 03:40:26PM -0500, Eric Blake wrote: > The two callers to a mirror job (drive-mirror and blockdev-mirror) set > zero_target precisely when sync mode == FULL, with the one exception > that drive-mirror skips zeroing the target if it was newly created and > reads a

Re: [PATCH v4 11/13] mirror: Skip writing zeroes when target is already zero

2025-05-14 Thread Eric Blake
On Wed, May 14, 2025 at 11:37:54PM +0800, Sunny Zhu wrote: > On Fri, 9 May 2025 15:40:28 -0500, Eric Blake wrote: > > @@ -847,8 +887,10 @@ static int coroutine_fn GRAPH_UNLOCKED > > mirror_dirty_init(MirrorBlockJob *s) > > bool punch_holes = > >

[PATCH v4 14/13] mirror: Reduce I/O when destination is detect-zeroes:unmap

2025-05-13 Thread Eric Blake
: Eric Blake --- block/mirror.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 724318f0371..c2c5099c951 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -920,11 +920,16 @@ static int coroutine_fn GRAPH_UNLOCKED

Re: [PATCH v4 11/13] mirror: Skip writing zeroes when target is already zero

2025-05-13 Thread Eric Blake
On Fri, May 09, 2025 at 03:40:28PM -0500, Eric Blake wrote: > When mirroring, the goal is to ensure that the destination reads the > same as the source; this goal is met whether the destination is sparse > or fully-allocated (except when explicitly punching holes, then merely > reading

Re: [PATCH v4 08/13] mirror: Allow QMP override to declare target already zero

2025-05-12 Thread Eric Blake
On Sat, May 03, 2025 at 07:57:24AM +0800, Sunny Zhu wrote: > On Fri, May 09, 2025 at 03:40:25PM -0500, Eric Blake wrote: > > QEMU has an optimization for a just-created drive-mirror destination > > that is not possible for blockdev-mirror (which can't create the > > desti

[PATCH v4 07/13] mirror: Pass full sync mode rather than bool to internals

2025-05-09 Thread Eric Blake
o s->is_none_mode=false, it is better to pass s->sync_mode throughout the entire operation. For active commit, the desired semantics match sync mode TOP. Signed-off-by: Eric Blake --- v4: new patch --- block/mirror.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) d

[PATCH v4 02/13] file-posix, gluster: Handle zero block status hint better

2025-05-09 Thread Eric Blake
uiting and advertising full allocation. This change will be utilized in later patches to let mirroring optimize for the case when the destination already reads as zeroes. Signed-off-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block/file-posix.c | 3 ++- block/gluster.c| 2 +- 2 files

[PATCH v4 04/13] block: Add new bdrv_co_is_all_zeroes() function

2025-05-09 Thread Eric Blake
re complexity in a common manner for all block devices - when the BDS is mostly a hole, but has a small non-hole header, it is still worth the time to read that header and check if it reads as all zeroes before giving up and returning a pessimistic answer. Signed-off-by: Eric Blake Reviewed-by: Stefa

[PATCH v4 11/13] mirror: Skip writing zeroes when target is already zero

2025-05-09 Thread Eric Blake
opened with "detect-zeroes": "unmap". And any time writing zeroes is skipped, the job counters are not incremented. Signed-off-by: Eric Blake --- v3: also skip counters when skipping I/O [Sunny] v4: rebase to earlier changes --- block/mirror.c | 107 +

[PATCH v4 10/13] mirror: Skip pre-zeroing destination if it is already zero

2025-05-09 Thread Eric Blake
shell filters _filter_block_job_{offset,len} in common.filter). A later test will also be added which further validates expected sparseness, so it does not matter that 194 is no longer explicitly looking at how many bytes were copied. Signed-off-by: Eric Blake --- v3: add exemption for "d

[PATCH v4 09/13] mirror: Drop redundant zero_target parameter

2025-05-09 Thread Eric Blake
target_is_zero. And since we recently updated things to pass the sync mode all the way through to mirror_dirty_init(), we can now reconstruct the same conditionals without the redundant parameter. Signed-off-by: Eric Blake --- v4: new patch --- include/block/block_int-global-state.h | 3

  1   2   3   4   5   6   7   8   9   10   >