rdblue opened a new pull request, #18984:
URL: https://github.com/apache/iceberg/pull/18984

   This adds `V4ManifestWriter` to produce v4 manifests. There are two inner 
classes to produce leaf and root manifests (created using `createRoot` and 
`createLeaf`). The root writer is similar to `ManifestListWriter`.
   
   Like `ManifestWriter`, this class assumes that incoming entry/tracking 
metadata is unchanged and replaces it with updated metadata produced by 
`TrackingBuilder`. This API accepts `TrackedFile` because the writer is not 
responsible for wrapping and is will be used directly by writers that handle 
`TrackedFile`. For instance, copying or filtering manifests will work with 
`TrackedFile` directly.
   
   The API for adding files is:
   - `append(TrackedFile)` adds a new file to the manifest
   - `appendAtSeq(TrackedFile, long)` adds a new file with a given data 
sequence number; equivalent to `ManifestWriter.add(file, seq)`
   - `modify(TrackedFile, DeletionVector)` adds a deletion vector to an 
existing data file
   - `existing(TrackedFile)` adds an existing file to the manifest
   - `delete(TrackedFile)` adds a file as deleted
   - `replace(TrackedFile)` adds a file as replaced
   
   These methods work with manifest files, but this PR is focused on data files 
and tests only that surface area. This also does not include support for 
updating column files, but writes with column files are validated.
   
   Test plan:
   - Test data file cases only
       - With and without DV
       - Unpartitioned and partitioned in the same file
   - Test all statuses (added, modified, existing, deleted, replaced)
       - Test append at sequence number
       - Test append with DV
       - Test modify with new DV
   - Test root and leaf files
       - Root metadata is written into the file and read unchanged
       - Leaf metadata is written as null and inherited in the manifest reader
   - Validate
       - All tracking fields
       - All data file fields except key metadata


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to