Scalability of many operations on wide ZFS pool can be limited by requirement to prefetch indirection tables first. Recently added asynchronous indirection tables read partially helped, but did not solve the problem completely. This patch extends existing prefetcher functionality to explicitly work with indirection tables.
Before this change prefetcher issued reads for up to 8MB of data in advance. With this change it also issues indirection tables reads for up to 64MB of data in advance, so that when it will be time to actually read those data, it can be done immediately. Alike affect can be achieved by just increasing maximal data prefetch distance, but at higher memory cost. Also this change introduces indirection tables prefetch for rewrite operations, that was never done before. Previously ARC miss for indirection table regularly blocked rewrites, converting perfectly aligned asynchronous operations into synchronous read-write pairs, significantly reducing maximal rewrite speed. While being there couple of other issues were also fixed: - indirection tables prefetches were not counted as prefetches in statistics; - prefetch was done always, even if caching for the dataset was completely disabled. Testing on FreeBSD with zvol on top of 6x striped 2x mirrored pool of 12 assorted HDDs shown me such performance numbers: ------- BEFORE -------- Write 491363677 bytes/sec Read 312430631 bytes/sec Rewrite 97680464 bytes/sec -------- AFTER -------- Write 493524146 bytes/sec Read 438598079 bytes/sec Rewrite 277506044 bytes/sec https://www.illumos.org/issues/6322 https://reviews.csiden.org/r/259/ You can view, comment on, or merge this pull request online at: https://github.com/openzfs/openzfs/pull/65 -- Commit Summary -- * Improve speculative indirection tables prefetch. -- File Changes -- M usr/src/uts/common/fs/zfs/dbuf.c (10) M usr/src/uts/common/fs/zfs/dmu.c (7) M usr/src/uts/common/fs/zfs/dmu_zfetch.c (45) M usr/src/uts/common/fs/zfs/sys/dmu_zfetch.h (3) M usr/src/uts/common/fs/zfs/sys/dnode.h (5) -- Patch Links -- https://github.com/openzfs/openzfs/pull/65.patch https://github.com/openzfs/openzfs/pull/65.diff --- Reply to this email directly or view it on GitHub: https://github.com/openzfs/openzfs/pull/65
_______________________________________________ developer mailing list [email protected] http://lists.open-zfs.org/mailman/listinfo/developer
