This is the PR promised in #4265. Draft/RFC, since I'd like feedback on the format and API decisions before polishing it further.
rpm currently copies payload file content in and out of packages with a userspace read/write loop. This series switches that to cloning, using FICLONERANGE with a copy_file_range() fallback, so that on filesystems with reflink support installed files share extents with the package instead of duplicating the data. The main use case is local image building, where the same packages are installed into many roots on one filesystem. Cloning needs block-aligned source content, so the series is structured as follows: 1. Add %_payload_alignment and RPMTAG_PAYLOADALIGNMENT (v4 and v6). Large regular files start on the configured boundary in the canonical uncompressed cpio stream. Works with any compressor, the padding is cheap in transit, and it's gated by an rpmlib(PayloadAlignment) capability. 2. Add an uncompressed on-disk representation where the cpio start is zero-framed to the boundary, so alignment inside the stream becomes alignment at absolute file offsets. Primary payload digests cover the stored bytes including the framing, ALT digests cover the canonical cpio and are identical for compressed and raw representations. 3. Add an internal range copy helper: FICLONERANGE for the whole range, then bounded copy_file_range() chunks, then a userspace loop. 4. Use it on install to clone content out of raw payloads. Installed content is still read back and hashed for the usual per-file digest verification. 5. Use it in rpmbuild to clone content from the buildroot when writing a raw payload. 6. Add rpmUncompressPackage() and hook it up to rpmuncompress (rpmuncompress foo.rpm > foo-raw.rpm). The lead and signed main header are preserved byte for byte and the signature header is rebuilt without the records covering the old physical payload, so header signatures and ALT digests verify on the result. The decompressed payload is verified against the signed ALT digests while writing. On the questions from the discussion: repositories keep their compression, alignment is recorded inside the compressed payload and nothing about transport changes. The workflow @pmatilai outlined is what's implemented: packages are built aligned but compressed, repeat consumers materialize a raw copy once and rpm installs from it with reflinks, signatures intact. Managing those copies (dnf/librepo integration) is out of scope here. Materialization also works on unaligned packages that have a payload ALT digest (rpm >= 4.16), the output just installs through copy_file_range() or plain copies instead of clones. Without alignment, without copy_file_range(), or on an incapable filesystem everything falls back to the current behavior. You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/4294 -- Commit Summary -- * Support aligned file contents in compressed RPM payloads * Support block-aligned uncompressed RPM payloads * Add a clone-first range copy primitive * Install payload files with reflink-first range copies * Copy buildroot files into raw payloads with reflinks * Materialize raw payloads without invalidating signatures -- File Changes -- M CMakeLists.txt (1) M build/pack.cc (81) M config.h.in (1) M docs/man/rpmuncompress.1.scd (9) M docs/manual/format_v6.md (53) M docs/manual/tags.md (15) M include/rpm/rpmarchive.h (16) M include/rpm/rpmlib.h (22) M include/rpm/rpmtag.h (1) M lib/CMakeLists.txt (2) M lib/cpio.cc (315) M lib/cpio.hh (68) M lib/fsm.cc (51) M lib/fsm.hh (13) M lib/package.cc (314) A lib/rpmalign.hh (38) M lib/rpmchecksig.cc (97) M lib/rpmds.cc (4) M lib/rpmfi.cc (155) A lib/rpmpayload.cc (85) A lib/rpmpayload.hh (20) M lib/rpmte.cc (27) M lib/rpmte_internal.hh (5) M lib/rpmvs.cc (15) M lib/rpmvs.hh (3) M lib/signature.cc (23) M lib/signature.hh (3) M macros.in (8) M rpmio/rpmio.cc (20) M rpmio/rpmio_internal.hh (6) M sign/rpmgensig.cc (25) M sign/rpmsignverity.cc (12) M tests/CMakeLists.txt (15) A tests/data/SPECS/aligntest.spec (43) A tests/rpmalign.at (199) A tests/rpmcfr.at (132) A tests/rpmcfrprobe.c (84) A tests/rpmcfrpython.at (63) A tests/rpmcorruptpad.c (64) M tests/rpmgeneral.at (1) A tests/rpmpayloadconsumer.at (51) A tests/rpmuncompress.at (241) A tests/rpmuncompresspkg.c (217) M tools/CMakeLists.txt (2) M tools/rpm2archive.cc (60) M tools/rpmuncompress.cc (89) -- Patch Links -- https://github.com/rpm-software-management/rpm/pull/4294.patch https://github.com/rpm-software-management/rpm/pull/4294.diff -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/4294 You are receiving this because you are subscribed to this thread. Message ID: <rpm-software-management/rpm/pull/[email protected]>
_______________________________________________ Rpm-maint mailing list [email protected] https://lists.rpm.org/mailman/listinfo/rpm-maint
