Re: [PR] Spec: Add GCS and ADLS configuration to REST table load [iceberg]

munendrasn commented on code in PR #10576: URL: https://github.com/apache/iceberg/pull/10576#discussion_r1703087225 ## open-api/rest-catalog-open-api.py: ## @@ -1100,6 +1100,31 @@ class LoadTableResult(BaseModel): - `s3.session-token`: if present, this value should be used

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r1703112487 ## core/src/main/java/org/apache/iceberg/encryption/StandardEncryptionManager.java: ## @@ -51,6 +69,8 @@ public StandardEncryptionManager( this.tableKeyId = ta

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r1703113214 ## core/src/main/java/org/apache/iceberg/encryption/StandardEncryptionManager.java: ## @@ -92,13 +112,98 @@ public ByteBuffer wrapKey(ByteBuffer secretKey) { p

Re: [PR] Concurrent table scans [iceberg-rust]

liurenjie1024 commented on PR #373: URL: https://github.com/apache/iceberg-rust/pull/373#issuecomment-2267424040 > Thanks @sdd! That's awesome! BTW, can we integrate the performance test within the repository so that it will be helpful for performance improvement in the future? cc @liurenji

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r1703115064 ## core/src/main/java/org/apache/iceberg/encryption/StandardEncryptionManager.java: ## @@ -92,13 +112,98 @@ public ByteBuffer wrapKey(ByteBuffer secretKey) { p

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r1703116600 ## core/src/main/java/org/apache/iceberg/encryption/EncryptionUtil.java: ## @@ -70,31 +76,105 @@ public static KeyManagementClient createKmsClient(Map catalogPro

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r1703118933 ## core/src/main/java/org/apache/iceberg/BaseManifestListFile.java: ## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/credentials from 1.17.16 to 1.17.27 [iceberg-go]

dependabot[bot] opened a new pull request, #119: URL: https://github.com/apache/iceberg-go/pull/119 Bumps [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2) from 1.17.16 to 1.17.27. Commits https://github.com/aws/aws-sdk-go-v2/commit/b8a0918056d0a

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.3 [iceberg-go]

dependabot[bot] opened a new pull request, #120: URL: https://github.com/apache/iceberg-go/pull/120 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.48.0 to 1.58.3. Commits https://github.com/aws/aws-sdk-go-v2/commit/5102c2e4b1a28439

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.2 [iceberg-go]

dependabot[bot] closed pull request #114: build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.2 URL: https://github.com/apache/iceberg-go/pull/114 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.48.0 to 1.58.2 [iceberg-go]

dependabot[bot] commented on PR #114: URL: https://github.com/apache/iceberg-go/pull/114#issuecomment-2267432444 Superseded by #120. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] Manifest list encryption [iceberg]

ggershinsky commented on code in PR #7770: URL: https://github.com/apache/iceberg/pull/7770#discussion_r1703118933 ## core/src/main/java/org/apache/iceberg/BaseManifestListFile.java: ## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Simplify PrimitiveLiteral [iceberg-rust]

ZENOTME commented on code in PR #502: URL: https://github.com/apache/iceberg-rust/pull/502#discussion_r1703145788 ## crates/iceberg/src/spec/values.rs: ## @@ -65,24 +65,14 @@ pub enum PrimitiveLiteral { Float(OrderedFloat), /// Stored as 8-byte little-endian Doubl

[PR] S3OutputStream: Don't complete upload on finalize [iceberg]

jasonf20 opened a new pull request, #10874: URL: https://github.com/apache/iceberg/pull/10874 If a user didn't call close on the stream the file shouldn't be uploaded. This means that if the user forgets to call `close()` on a completed file it will no longer be uploaded. However, thi

Re: [I] Investigation about tracing, logging, and metrics support. [iceberg-rust]

ZENOTME commented on issue #482: URL: https://github.com/apache/iceberg-rust/issues/482#issuecomment-2267510308 > Some additional context. Renjie mentioned: > > > Cons: > > It's mixing logging with tracing. If we use tracing for logging, our logging compability will not be compatib

[PR] feat: support for gcs storage [iceberg-rust]

jdockerty opened a new pull request, #520: URL: https://github.com/apache/iceberg-rust/pull/520 Helps towards https://github.com/apache/iceberg-rust/issues/408 and https://github.com/apache/iceberg-rust/issues/410 Adds support for GCS as a storage backend. -- This is an automat

Re: [PR] feat: support for gcs storage [iceberg-rust]

jdockerty commented on code in PR #520: URL: https://github.com/apache/iceberg-rust/pull/520#discussion_r1703160577 ## crates/iceberg/tests/file_io_gcs_test.rs: ## @@ -0,0 +1,103 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license ag

Re: [PR] Manifest list encryption [iceberg]

ggershinsky closed pull request #7770: Manifest list encryption URL: https://github.com/apache/iceberg/pull/7770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] Manifest list encryption [iceberg]

ggershinsky opened a new pull request, #7770: URL: https://github.com/apache/iceberg/pull/7770 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[PR] chore(deps): Bump crate-ci/typos from 1.23.5 to 1.23.6 [iceberg-rust]

dependabot[bot] opened a new pull request, #521: URL: https://github.com/apache/iceberg-rust/pull/521 Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.23.5 to 1.23.6. Release notes Sourced from https://github.com/crate-ci/typos/releases";>crate-ci/typos's releases.

Re: [PR] chore(deps): Bump crate-ci/typos from 1.23.5 to 1.23.6 [iceberg-rust]

Xuanwo merged PR #521: URL: https://github.com/apache/iceberg-rust/pull/521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [I] Review new DangerousJavaDeserialization error-prone check [iceberg]

spotted-cat commented on issue #10853: URL: https://github.com/apache/iceberg/issues/10853#issuecomment-2267556976 Can I work on this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Simplify PrimitiveLiteral [iceberg-rust]

liurenjie1024 commented on code in PR #502: URL: https://github.com/apache/iceberg-rust/pull/502#discussion_r1703224232 ## crates/iceberg/src/spec/values.rs: ## @@ -65,24 +65,14 @@ pub enum PrimitiveLiteral { Float(OrderedFloat), /// Stored as 8-byte little-endian

[I] Some schema updates do not support dots inside a field name [iceberg]

adamhonen opened a new issue, #10875: URL: https://github.com/apache/iceberg/issues/10875 ### Apache Iceberg version 1.6.0 (latest release) ### Query engine None ### Please describe the bug 🐞 I've reproduced this with the following test added to `TestSchema

Re: [PR] Simplify PrimitiveLiteral [iceberg-rust]

ZENOTME commented on code in PR #502: URL: https://github.com/apache/iceberg-rust/pull/502#discussion_r1703299049 ## crates/iceberg/src/spec/values.rs: ## @@ -65,24 +65,14 @@ pub enum PrimitiveLiteral { Float(OrderedFloat), /// Stored as 8-byte little-endian Doubl

Re: [PR] Simplify PrimitiveLiteral [iceberg-rust]

ZENOTME commented on code in PR #502: URL: https://github.com/apache/iceberg-rust/pull/502#discussion_r1703299049 ## crates/iceberg/src/spec/values.rs: ## @@ -65,24 +65,14 @@ pub enum PrimitiveLiteral { Float(OrderedFloat), /// Stored as 8-byte little-endian Doubl

Re: [I] Review new DangerousJavaDeserialization error-prone check [iceberg]

Fokko commented on issue #10853: URL: https://github.com/apache/iceberg/issues/10853#issuecomment-2267629008 @spotted-cat sure thing! Feel free to reach out when you bump into anything -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[PR] Support execution in Windows using Local File System and NFS [iceberg-python]

rfung777 opened a new pull request, #996: URL: https://github.com/apache/iceberg-python/pull/996 When running PyIceberg in Windows, `pyiceberg.io.pyarrow.PyArrowFileIO.parse_location()` incorrectly resolve Windows path, e.g. `C:\foobar` as a scheme of `C` and causes `Unrecognized filesyste

Re: [PR] Spec: Clarify identity partition edge cases. [iceberg]

rdblue commented on code in PR #10835: URL: https://github.com/apache/iceberg/pull/10835#discussion_r1703358973 ## format/spec.md: ## @@ -241,7 +245,14 @@ Struct evolution requires the following rules for default values: Column Projection -Columns in Iceberg data file

Re: [I] Review new DangerousJavaDeserialization error-prone check [iceberg]

spotted-cat commented on issue #10853: URL: https://github.com/apache/iceberg/issues/10853#issuecomment-2267644030 > @spotted-cat sure thing! Feel free to reach out when you bump into anything Thankyou for offering to help. So I looked up into the issue, and I am unable to find

Re: [PR] Spec: Deprecate the file system table scheme. [iceberg]

rdblue commented on code in PR #10833: URL: https://github.com/apache/iceberg/pull/10833#discussion_r1703399862 ## format/spec.md: ## @@ -779,7 +779,9 @@ When two commits happen at the same time and are based on the same version, only File System Tables -An atomic swa

Re: [PR] Spec: Deprecate the file system table scheme. [iceberg]

rdblue merged PR #10833: URL: https://github.com/apache/iceberg/pull/10833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

amogh-jahagirdar commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703406710 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1015,6 +1015,11 @@ public Builder upgradeFormatVersion(int newFormatVersion) {

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

amogh-jahagirdar commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703406710 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1015,6 +1015,11 @@ public Builder upgradeFormatVersion(int newFormatVersion) {

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

amogh-jahagirdar commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703406710 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1015,6 +1015,11 @@ public Builder upgradeFormatVersion(int newFormatVersion) {

Re: [I] Docs: Add document how to export records from CDC/Upsert Stream into apache iceberg table. [iceberg]

github-actions[bot] commented on issue #3105: URL: https://github.com/apache/iceberg/issues/3105#issuecomment-2267959008 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] SparkSessionCatalog.listTables() does not list Iceberg tables [iceberg]

github-actions[bot] closed issue #3109: SparkSessionCatalog.listTables() does not list Iceberg tables URL: https://github.com/apache/iceberg/issues/3109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] after set 'write.upsert.enable'='true' in flink sql, using flink sql read iceberg table will get exception: java.lang.IllegalArgumentException: Row arity: 3, but serializer arity: 2 [iceberg]

github-actions[bot] closed issue #3114: after set 'write.upsert.enable'='true' in flink sql, using flink sql read iceberg table will get exception: java.lang.IllegalArgumentException: Row arity: 3, but serializer arity: 2 URL: https://github.com/apache/iceberg/issues/3114 -- This is an autom

Re: [I] after set 'write.upsert.enable'='true' in flink sql, using flink sql read iceberg table will get exception: java.lang.IllegalArgumentException: Row arity: 3, but serializer arity: 2 [iceberg]

github-actions[bot] commented on issue #3114: URL: https://github.com/apache/iceberg/issues/3114#issuecomment-2267959032 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Docs: we need a matrix to maintain the compatibility relationship between engines and iceberg version. [iceberg]

github-actions[bot] commented on issue #3115: URL: https://github.com/apache/iceberg/issues/3115#issuecomment-2267959044 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Read delete files in parallel. [iceberg]

github-actions[bot] closed issue #3118: Read delete files in parallel. URL: https://github.com/apache/iceberg/issues/3118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [I] Write the changelog whose primary key is DATE/TIMESTAMP type to iceberg through Flink, and throw an IllegalStateException when it is read out [iceberg]

github-actions[bot] commented on issue #3119: URL: https://github.com/apache/iceberg/issues/3119#issuecomment-2267959066 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Core: add Jackson serialization util for Trino and Presto [iceberg]

github-actions[bot] commented on PR #3210: URL: https://github.com/apache/iceberg/pull/3210#issuecomment-2267959139 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: auto compact small files [iceberg]

github-actions[bot] closed pull request #3213: Flink: auto compact small files URL: https://github.com/apache/iceberg/pull/3213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Flink: auto compact small files [iceberg]

github-actions[bot] commented on PR #3213: URL: https://github.com/apache/iceberg/pull/3213#issuecomment-2267959157 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Hive : fix vectorization test if hive it's not ok with vectorization [iceberg]

github-actions[bot] closed pull request #3242: Hive : fix vectorization test if hive it's not ok with vectorization URL: https://github.com/apache/iceberg/pull/3242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Hive : fix vectorization test if hive it's not ok with vectorization [iceberg]

github-actions[bot] commented on PR #3242: URL: https://github.com/apache/iceberg/pull/3242#issuecomment-2267959170 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: Add operator to expire snapshots [iceberg]

github-actions[bot] closed pull request #3255: Flink: Add operator to expire snapshots URL: https://github.com/apache/iceberg/pull/3255 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] add show partition with partitions [iceberg]

github-actions[bot] closed pull request #3271: add show partition with partitions URL: https://github.com/apache/iceberg/pull/3271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen

Re: [PR] add show partition with partitions [iceberg]

github-actions[bot] commented on PR #3271: URL: https://github.com/apache/iceberg/pull/3271#issuecomment-2267959198 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Parquet: avoid premature closing of FSDataInputStream object [iceberg]

github-actions[bot] commented on PR #3276: URL: https://github.com/apache/iceberg/pull/3276#issuecomment-2267959206 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: Incrementally rewrite data files in streaming. [iceberg]

github-actions[bot] commented on PR #3323: URL: https://github.com/apache/iceberg/pull/3323#issuecomment-2267959252 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Flink: Incrementally rewrite data files in streaming. [iceberg]

github-actions[bot] closed pull request #3323: Flink: Incrementally rewrite data files in streaming. URL: https://github.com/apache/iceberg/pull/3323 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Hive handler estimate file size [iceberg]

github-actions[bot] closed pull request #3328: Hive handler estimate file size URL: https://github.com/apache/iceberg/pull/3328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Docs: Add document how to export records from CDC/Upsert Stream into apache iceberg table. [iceberg]

github-actions[bot] closed issue #3105: Docs: Add document how to export records from CDC/Upsert Stream into apache iceberg table. URL: https://github.com/apache/iceberg/issues/3105 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] Hive handler estimate file size [iceberg]

github-actions[bot] commented on PR #3328: URL: https://github.com/apache/iceberg/pull/3328#issuecomment-2267959265 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] SparkSessionCatalog.listTables() does not list Iceberg tables [iceberg]

github-actions[bot] commented on issue #3109: URL: https://github.com/apache/iceberg/issues/3109#issuecomment-2267959022 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Core: enable setting table property as action config option default [iceberg]

github-actions[bot] closed pull request #3387: Core: enable setting table property as action config option default URL: https://github.com/apache/iceberg/pull/3387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] Core: enable setting table property as action config option default [iceberg]

github-actions[bot] commented on PR #3387: URL: https://github.com/apache/iceberg/pull/3387#issuecomment-2267959313 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [I] Read delete files in parallel. [iceberg]

github-actions[bot] commented on issue #3118: URL: https://github.com/apache/iceberg/issues/3118#issuecomment-2267959053 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Write the changelog whose primary key is DATE/TIMESTAMP type to iceberg through Flink, and throw an IllegalStateException when it is read out [iceberg]

github-actions[bot] closed issue #3119: Write the changelog whose primary key is DATE/TIMESTAMP type to iceberg through Flink, and throw an IllegalStateException when it is read out URL: https://github.com/apache/iceberg/issues/3119 -- This is an automated message from the Apache Git Service

Re: [PR] Add watermark in flink table [iceberg]

github-actions[bot] commented on PR #3681: URL: https://github.com/apache/iceberg/pull/3681#issuecomment-2267959398 This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull

Re: [PR] Core: add Jackson serialization util for Trino and Presto [iceberg]

github-actions[bot] closed pull request #3210: Core: add Jackson serialization util for Trino and Presto URL: https://github.com/apache/iceberg/pull/3210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] When flinkSink job runs on yarn ,the container is killed because physical memory use beyond physical memory limits [iceberg]

github-actions[bot] closed issue #3695: When flinkSink job runs on yarn ,the container is killed because physical memory use beyond physical memory limits URL: https://github.com/apache/iceberg/issues/3695 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Docs: we need a matrix to maintain the compatibility relationship between engines and iceberg version. [iceberg]

github-actions[bot] closed issue #3115: Docs: we need a matrix to maintain the compatibility relationship between engines and iceberg version. URL: https://github.com/apache/iceberg/issues/3115 -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Flink: Add operator to expire snapshots [iceberg]

github-actions[bot] commented on PR #3255: URL: https://github.com/apache/iceberg/pull/3255#issuecomment-2267959181 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Parquet: avoid premature closing of FSDataInputStream object [iceberg]

github-actions[bot] closed pull request #3276: Parquet: avoid premature closing of FSDataInputStream object URL: https://github.com/apache/iceberg/pull/3276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Fixes got unexpected number of data files after binpack rewrite when there are delete files [iceberg]

github-actions[bot] commented on PR #3301: URL: https://github.com/apache/iceberg/pull/3301#issuecomment-2267959236 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Fixes got unexpected number of data files after binpack rewrite when there are delete files [iceberg]

github-actions[bot] closed pull request #3301: Fixes got unexpected number of data files after binpack rewrite when there are delete files URL: https://github.com/apache/iceberg/pull/3301 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] Spark: Fix NPE if streaming read since an expired snapshot [iceberg]

github-actions[bot] commented on PR #3358: URL: https://github.com/apache/iceberg/pull/3358#issuecomment-2267959285 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If y

Re: [PR] Spark: Fix NPE if streaming read since an expired snapshot [iceberg]

github-actions[bot] closed pull request #3358: Spark: Fix NPE if streaming read since an expired snapshot URL: https://github.com/apache/iceberg/pull/3358 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [I] When flinkSink job runs on yarn ,the container is killed because physical memory use beyond physical memory limits [iceberg]

github-actions[bot] commented on issue #3695: URL: https://github.com/apache/iceberg/issues/3695#issuecomment-2267959415 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

RussellSpitzer commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703464805 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1015,6 +1015,11 @@ public Builder upgradeFormatVersion(int newFormatVersion) {

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

RussellSpitzer commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703466705 ## core/src/test/java/org/apache/iceberg/TestFormatVersions.java: ## @@ -28,51 +28,79 @@ public class TestFormatVersions extends TestBase { @Parameters(nam

Re: [PR] Simplify PrimitiveLiteral [iceberg-rust]

liurenjie1024 commented on code in PR #502: URL: https://github.com/apache/iceberg-rust/pull/502#discussion_r1703468335 ## crates/iceberg/src/spec/values.rs: ## @@ -65,24 +65,14 @@ pub enum PrimitiveLiteral { Float(OrderedFloat), /// Stored as 8-byte little-endian

Re: [PR] #10668 - Support case-insensitivity for column names in PartitionSpec [iceberg]

stevenzwu commented on code in PR #10678: URL: https://github.com/apache/iceberg/pull/10678#discussion_r1702369242 ## api/src/main/java/org/apache/iceberg/types/TypeUtil.java: ## @@ -182,11 +181,10 @@ public static Map indexQuotedNameById( } public static Map indexByLowe

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

RussellSpitzer commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703469879 ## core/src/test/java/org/apache/iceberg/TestTableMetadata.java: ## @@ -1452,52 +1452,148 @@ public void testCreateV2MetadataThroughTableProperty() { }

Re: [PR] Core: V3 Metadata Upgrade Validation and Testing [iceberg]

RussellSpitzer commented on code in PR #10861: URL: https://github.com/apache/iceberg/pull/10861#discussion_r1703471612 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1015,6 +1015,11 @@ public Builder upgradeFormatVersion(int newFormatVersion) {

Re: [PR] Core, API: Add addNonDefaultSpec to UpdatePartitionSpec to not set the new partition spec as default [iceberg]

RussellSpitzer commented on PR #10736: URL: https://github.com/apache/iceberg/pull/10736#issuecomment-2268067745 @shanielh In general we want to wait extra time on API module changes since we are committing the project to support this new API for ever. That said I think we've had this open

Re: [PR] Core, API: Add addNonDefaultSpec to UpdatePartitionSpec to not set the new partition spec as default [iceberg]

RussellSpitzer merged PR #10736: URL: https://github.com/apache/iceberg/pull/10736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Core, API: Add addNonDefaultSpec to UpdatePartitionSpec to not set the new partition spec as default [iceberg]

RussellSpitzer commented on PR #10736: URL: https://github.com/apache/iceberg/pull/10736#issuecomment-2268068867 Thanks @shanielh for the PR, and @amogh-jahagirdar for Reviewing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] The ColumnarToRow Spark optimization is not applied when using nested fields from an Iceberg table [iceberg]

RussellSpitzer commented on issue #10828: URL: https://github.com/apache/iceberg/issues/10828#issuecomment-2268070006 It isn't supported because we basically copied the Spark Vectorized read code and at the time we copied it nested structs weren't supported. -- This is an automated messag

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

shanielh commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703493428 ## core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java: ## @@ -66,20 +66,36 @@ class BaseUpdatePartitionSpec implements UpdatePartitionSpec { th

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703502870 ## api/src/main/java/org/apache/iceberg/UpdatePartitionSpec.java: ## @@ -29,6 +29,7 @@ * will not be resolved and will result in a {@link CommitFailedExcepti

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703504713 ## core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java: ## @@ -62,15 +62,20 @@ class BaseUpdatePartitionSpec implements UpdatePartitionSpec {

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703505303 ## core/src/test/java/org/apache/iceberg/TestTableUpdatePartitionSpec.java: ## @@ -90,6 +90,46 @@ public void testCommitUpdatedSpec() { assertThat(table.s

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703506059 ## core/src/test/java/org/apache/iceberg/TestTableUpdatePartitionSpec.java: ## @@ -90,6 +90,46 @@ public void testCommitUpdatedSpec() { assertThat(table.s

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703507104 ## core/src/test/java/org/apache/iceberg/TestTableUpdatePartitionSpec.java: ## @@ -90,6 +90,46 @@ public void testCommitUpdatedSpec() { assertThat(table.s

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703507372 ## core/src/test/java/org/apache/iceberg/TestTableUpdatePartitionSpec.java: ## @@ -90,6 +90,46 @@ public void testCommitUpdatedSpec() { assertThat(table.s

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

shanielh commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703549673 ## core/src/test/java/org/apache/iceberg/TestTableUpdatePartitionSpec.java: ## @@ -90,6 +90,46 @@ public void testCommitUpdatedSpec() { assertThat(table.spec().

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

shanielh commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703554438 ## core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java: ## @@ -62,15 +62,20 @@ class BaseUpdatePartitionSpec implements UpdatePartitionSpec { priv

Re: [PR] AddPartitionSpec: A new way to set new partition specs [iceberg]

RussellSpitzer commented on code in PR #10737: URL: https://github.com/apache/iceberg/pull/10737#discussion_r1703559470 ## core/src/main/java/org/apache/iceberg/BaseUpdatePartitionSpec.java: ## @@ -62,15 +62,20 @@ class BaseUpdatePartitionSpec implements UpdatePartitionSpec {

Re: [PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 5.58.0 to 5.61.0 [iceberg]

RussellSpitzer merged PR #10864: URL: https://github.com/apache/iceberg/pull/10864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] core: support support move a column with same name after rename column [iceberg]

RussellSpitzer commented on code in PR #10862: URL: https://github.com/apache/iceberg/pull/10862#discussion_r1703590516 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -185,6 +186,8 @@ private void internalAddColumn( @Override public UpdateSchema delete

Re: [PR] core: support support move a column with same name after rename column [iceberg]

RussellSpitzer commented on code in PR #10862: URL: https://github.com/apache/iceberg/pull/10862#discussion_r1703595539 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -201,6 +204,8 @@ public UpdateSchema renameColumn(String name, String newName) { Types.

Re: [PR] core: support support move a column with same name after rename column [iceberg]

RussellSpitzer commented on code in PR #10862: URL: https://github.com/apache/iceberg/pull/10862#discussion_r1703598014 ## core/src/main/java/org/apache/iceberg/SchemaUpdate.java: ## @@ -201,6 +204,8 @@ public UpdateSchema renameColumn(String name, String newName) { Types.

Re: [PR] Build: Bump nessie from 0.94.2 to 0.94.4 [iceberg]

Fokko merged PR #10869: URL: https://github.com/apache/iceberg/pull/10869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa