On Thu, Aug 11, 2022 at 08:02:15PM +1200, Alex King wrote:
> We are using Direct IO to slow mkfs and therefore lower I/O load. I'm not
> aware of a way to directly ask mkfs to reduce the I/O load or work more
> slowly. Using ionice at idle priority would be ideal, but we are using the
> mq-deadline scheduler rather than bfq or anything. I understand ionice is a
> noop under the deadline scheduler.
>
> In this case we want the mkfs to be lower priority that other tasks, e.g.
> other XEN guests. And sometimes we are creating large filesystems on
> spinning disk.
Ah! Thanks for the clarification. I wonder if adding the following
to your /etc/mke2fs.conf would work for you:
[options]
sync_kludge = 5
>From mke2fs.conf:
sync_kludge
If this relation is set to a positive integer, then while
writing the inode table, mke2fs will request the operating
system flush out pending writes to initialize the inode table
every sync_kludge block groups. This is needed to work
around buggy kernels that don't handle writeback throttling
correctly.
It might be that Direct I/O is still better for you, since it avoids
consuming memory in the page cache. OTOH, the advantage of using
sync_kludge is it is more adjustable, since you can specify an integer
number of block groups between fsync() calls.
We could also think about a more direct way to regulate the I/O load
by (for example) timing how long it takes to complete the fsync(2) and
then use that to make mke2fs sleep between each batch of block group
writes, so we could specify a target kb/s or mb/s that mke2fs should
try to hit....
Cheers,
- Ted
