paul-bormans-pcgw opened a new issue, #1368: URL: https://github.com/apache/iceberg-python/issues/1368
### Apache Iceberg version 0.7.1 ### Please describe the bug ๐ We use pyiceberg to create a table and then append "chunks" of data (pyarrow table) to that table. This creates a new commit each time (which is expected obviously), however when listing the tables history only a single (yongest) snapshot-id is showing up? What are we missing? Where are the older versions? We are running Nessie / Rest as the catalog: quay.io/projectnessie/nessie:0.100.1 ``` $ pyiceberg --uri=http://localhost:19120/iceberg/main properties get namespace xyz location s3://demobucket/xyz/ $ pyiceberg --uri=http://localhost:19120/iceberg/main properties get table xyz.pack history.expire.max-snapshot-age-ms 7200000 nessie.commit.id 9f143150991c552ba763c976813834c8ec2b4eba716eddba132907c3899c7b8a write.metadata.previous-versions-max 10 gc.enabled false nessie.catalog.content-id f4432c4d-2059-4dcc-9b1c-9565c6d781d5 created-at 2024-11-22T14:36:39.583176654Z write.metadata.delete-after-commit.enabled true history.expire.min-snapshots-to-keep 1 nessie.commit.ref main $ pyiceberg --uri=http://localhost:19120/iceberg/main list-refs xyz.pack Snapshot Refs โโโโโโโโณโโโโโโโโโณโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโ โ Ref โ Type โ Max ref age ms โ Min snapshots to keep โ Max snapshot age ms โ โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ โ main โ branch โ forever โ 1 โ 7200000 โ โโโโโโโโดโโโโโโโโโดโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโ $ pyiceberg --uri=http://localhost:19120/iceberg/main describe xyz.pack --entity table Table format version 2 Metadata location s3://demobucket/xyz/pack_eba37841-2b06-447b-96b3-0735f47dd74d/metadata/00000-26fef904-6314-476b-afb1-a32b73f6c9da.metadata.json Table UUID 53525b03-2818-4223-a58a-4d86b92ee6b8 Last Updated 1732528970031 Partition spec [] Sort order [] Current schema Schema, id=2 <...> Current snapshot Operation.APPEND: id=3802038740629390389, schema_id=2 Snapshots Snapshots โโโ Snapshot 3802038740629390389, schema 2: s3://demobucket/xyz/pack_eba37841-2b06-447b-96b3-0735f47dd74d/metadata/snap-3802038740629390389-0-2b017922-61c9-4b2e-a25c-2d24b370cb8f.avro Properties history.expire.max-snapshot-age-ms 7200000 nessie.commit.id 9f143150991c552ba763c976813834c8ec2b4eba716eddba132907c3899c7b8a write.metadata.previous-versions-max 10 gc.enabled false nessie.catalog.content-id f4432c4d-2059-4dcc-9b1c-9565c6d781d5 created-at 2024-11-22T14:36:39.583176654Z write.metadata.delete-after-commit.enabled true history.expire.min-snapshots-to-keep 1 nessie.commit.ref main $ nessie log --number 10 --extended commit 9f143150991c552ba763c976813834c8ec2b4eba716eddba132907c3899c7b8a (main) Author: Date: Mon Nov 25 10:02:50 2024 +0000 Parent: 48320ef7df58afeaaaf01420b64ff5a421785efbf6b1117c9629fd07a4ed727e Operations: Put of xyz.pack : Iceberg table: metadata-location: s3://demobucket/xyz/pack_eba37841-2b06-447b-96b3-0735f47dd74d/metadata/00000-26fef904-6314-476b-afb1-a32b73f6c9da.metadata.json snapshot-id: 3802038740629390389 schema-id: 2 partition-spec-id: 0 default-sort-order-id: 0 Update ICEBERG_TABLE xyz.pack commit 48320ef7df58afeaaaf01420b64ff5a421785efbf6b1117c9629fd07a4ed727e Author: Date: Mon Nov 25 10:02:37 2024 +0000 Parent: a7020ff32bf2ac0c320e2686b4edf604f622f9b9245b7085ba0b2a070b4882a9 Operations: Put of xyz.pack : Iceberg table: metadata-location: s3://demobucket/xyz/pack_eba37841-2b06-447b-96b3-0735f47dd74d/metadata/00000-536316a7-485b-4540-8f3c-aa510ff86d03.metadata.json snapshot-id: 8940046520981915903 schema-id: 2 partition-spec-id: 0 default-sort-order-id: 0 Update ICEBERG_TABLE xyz.pack <etc....> ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
