Re: [PR] Bump coverage from 7.5.1 to 7.5.2 [iceberg-python]

2024-05-27 Thread via GitHub
HonahX merged PR #772: URL: https://github.com/apache/iceberg-python/pull/772 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Bump moto from 5.0.7 to 5.0.8 [iceberg-python]

2024-05-27 Thread via GitHub
HonahX merged PR #771: URL: https://github.com/apache/iceberg-python/pull/771 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Add azure configuration variables [iceberg-python]

2024-05-27 Thread via GitHub
HonahX merged PR #745: URL: https://github.com/apache/iceberg-python/pull/745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Support getting a snapshot right before the given timestamp [iceberg-python]

2024-05-27 Thread via GitHub
HonahX commented on code in PR #748: URL: https://github.com/apache/iceberg-python/pull/748#discussion_r1616606353 ## pyiceberg/table/__init__.py: ## @@ -1290,6 +1291,19 @@ def snapshot_by_name(self, name: str) -> Optional[Snapshot]: return self.snapshot_by_id(ref.

Re: [PR] Docs: Remove link to Flink mailing list [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on PR #10388: URL: https://github.com/apache/iceberg/pull/10388#issuecomment-2134381110 Another option is to exclude this link from check. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Docs: Remove link to Flink mailing list [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on PR #10388: URL: https://github.com/apache/iceberg/pull/10388#issuecomment-2134380213 @pvary do you see other link check failure than this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Docs: Remove link to Flink mailing list [iceberg]

2024-05-27 Thread via GitHub
pvary commented on PR #10388: URL: https://github.com/apache/iceberg/pull/10388#issuecomment-2134372711 I'm not sure about the state of the hidden partitioning, but the link checker is very-very flaky. We should fix that first and then decide independently about changes in the doc -- Thi

Re: [PR] Bump mkdocs-material from 9.5.24 to 9.5.25 [iceberg-python]

2024-05-27 Thread via GitHub
Fokko merged PR #770: URL: https://github.com/apache/iceberg-python/pull/770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Feature: Schema into_builder method [iceberg-rust]

2024-05-27 Thread via GitHub
liurenjie1024 merged PR #381: URL: https://github.com/apache/iceberg-rust/pull/381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [I] Feature: Convert Schema into Builder [iceberg-rust]

2024-05-27 Thread via GitHub
liurenjie1024 closed issue #382: Feature: Convert Schema into Builder URL: https://github.com/apache/iceberg-rust/issues/382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Docs: Remove link to Flink mailing list [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on PR #10388: URL: https://github.com/apache/iceberg/pull/10388#issuecomment-2134301973 cc @stevenzwu @pvary to check if I've missed any recent progress on hidden partitioning support in Flink. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on PR #10252: URL: https://github.com/apache/iceberg/pull/10252#issuecomment-2134300377 @flyrain Thanks for review. Please take another look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on code in PR #10252: URL: https://github.com/apache/iceberg/pull/10252#discussion_r1616544922 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestChangelogTable.java: ## @@ -291,6 +291,59 @@ public void testMetadataColumns()

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on code in PR #10252: URL: https://github.com/apache/iceberg/pull/10252#discussion_r1616533549 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkScanBuilder.java: ## @@ -589,6 +589,21 @@ private Long getStartSnapshotId(Long startTimestamp

Re: [PR] feat: support lower_bound&&upper_bound for parquet writer [iceberg-rust]

2024-05-27 Thread via GitHub
ZENOTME commented on PR #383: URL: https://github.com/apache/iceberg-rust/pull/383#issuecomment-2134271741 cc @Fokko @liurenjie1024 @Xuanwo @sdd @marvinlanhenke -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Spark 3.5: Only traverse ancestors of current snapshot when building changelog scan [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on code in PR #10252: URL: https://github.com/apache/iceberg/pull/10252#discussion_r1616503562 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestChangelogTable.java: ## @@ -291,6 +291,59 @@ public void testMetadataColumns()

Re: [PR] Build: Bump mkdocs-material from 9.5.23 to 9.5.24 [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on PR #10380: URL: https://github.com/apache/iceberg/pull/10380#issuecomment-2134212368 I created https://github.com/apache/iceberg/pull/10388 to fix link check failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[PR] Docs: Remove link to Flink mailing list [iceberg]

2024-05-27 Thread via GitHub
manuzhang opened a new pull request, #10388: URL: https://github.com/apache/iceberg/pull/10388 The link check has been failing frequently, and I don't think there's ongoing discussion. ``` [✖] http://mail-archives.apache.org/mod_mbox/flink-dev/202008.mbox/%3ccabi+2jqco3msoa4+ywaxv

Re: [I] Flaky Test `TestIcebergSourceFailover > testBoundedWithSavepoint` [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on issue #10356: URL: https://github.com/apache/iceberg/issues/10356#issuecomment-2134207162 Another failure case ``` TestIcebergSourceFailover > testBoundedWithSavepoint FAILED java.util.concurrent.ExecutionException: org.apache.flink.runtime.checkpoint

Re: [I] Table scan using functional filters [iceberg-python]

2024-05-27 Thread via GitHub
github-actions[bot] commented on issue #170: URL: https://github.com/apache/iceberg-python/issues/170#issuecomment-2134161773 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity oc

Re: [I] iceberg-bundled-guava module shadowJar relocate use questions ask [iceberg]

2024-05-27 Thread via GitHub
github-actions[bot] commented on issue #1750: URL: https://github.com/apache/iceberg/issues/1750#issuecomment-2134160671 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [PR] Core: Throw CommitStateUnknownException if RuntimeException that is not marked as cleanable is thrown [iceberg]

2024-05-27 Thread via GitHub
zackurey commented on PR #10373: URL: https://github.com/apache/iceberg/pull/10373#issuecomment-2134141718 👍 to the conservative approach being taken here to _not_ assume a failure is `Cleanable` unless explicitly categorized as such. I worry about the burden it puts on catalog implementa

Re: [PR] Support special chars in S3URI [iceberg]

2024-05-27 Thread via GitHub
dimas-b commented on PR #10283: URL: https://github.com/apache/iceberg/pull/10283#issuecomment-2134126834 See #10329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [PR] Support special chars in S3URI [iceberg]

2024-05-27 Thread via GitHub
dimas-b closed pull request #10283: Support special chars in S3URI URL: https://github.com/apache/iceberg/pull/10283 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscr

[PR] Bump coverage from 7.5.1 to 7.5.2 [iceberg-python]

2024-05-27 Thread via GitHub
dependabot[bot] opened a new pull request, #772: URL: https://github.com/apache/iceberg-python/pull/772 Bumps [coverage](https://github.com/nedbat/coveragepy) from 7.5.1 to 7.5.2. Changelog Sourced from https://github.com/nedbat/coveragepy/blob/master/CHANGES.rst";>coverage's chang

[PR] Bump moto from 5.0.7 to 5.0.8 [iceberg-python]

2024-05-27 Thread via GitHub
dependabot[bot] opened a new pull request, #771: URL: https://github.com/apache/iceberg-python/pull/771 Bumps [moto](https://github.com/getmoto/moto) from 5.0.7 to 5.0.8. Changelog Sourced from https://github.com/getmoto/moto/blob/master/CHANGELOG.md";>moto's changelog. 5.0.

[PR] Bump mkdocs-material from 9.5.24 to 9.5.25 [iceberg-python]

2024-05-27 Thread via GitHub
dependabot[bot] opened a new pull request, #770: URL: https://github.com/apache/iceberg-python/pull/770 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.24 to 9.5.25. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk

Re: [I] Unpredictable behaviour with `S3FileIO` when column names contain `#` [iceberg]

2024-05-27 Thread via GitHub
danielcweeks closed issue #10279: Unpredictable behaviour with `S3FileIO` when column names contain `#` URL: https://github.com/apache/iceberg/issues/10279 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Url encode field names for partition paths [iceberg]

2024-05-27 Thread via GitHub
danielcweeks merged PR #10329: URL: https://github.com/apache/iceberg/pull/10329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceb

Re: [PR] Url encode field names for partition paths [iceberg]

2024-05-27 Thread via GitHub
danielcweeks commented on code in PR #10329: URL: https://github.com/apache/iceberg/pull/10329#discussion_r1616406486 ## core/src/test/java/org/apache/iceberg/TestLocationProvider.java: ## @@ -285,4 +286,22 @@ public void testObjectStorageWithinTableLocation() { assertThat(

Re: [I] Have a better error message in case of a schema mismatch [iceberg-python]

2024-05-27 Thread via GitHub
Fokko closed issue #328: Have a better error message in case of a schema mismatch URL: https://github.com/apache/iceberg-python/issues/328 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [I] Have a better error message in case of a schema mismatch [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on issue #328: URL: https://github.com/apache/iceberg-python/issues/328#issuecomment-2134067437 @MehulBatra Thanks, but I think this was already fixed by https://github.com/apache/iceberg-python/pull/352 -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Infra: Track subtasks from Iceberg improvement proposal [iceberg]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #10183: URL: https://github.com/apache/iceberg/pull/10183#discussion_r1616337365 ## .github/ISSUE_TEMPLATE/iceberg_proposal.yml: ## @@ -47,4 +47,10 @@ body: - label: REST - label: Puffin - label: Encryption -- labe

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-27 Thread via GitHub
sdd commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2133921862 One problem with doing it through an env var is that it applies to every table you hit in your service. I think it would be better if it was a config param so that you can configure it per

[I] SqlCatalog and other catalog works different on load_namespace_properties [iceberg-python]

2024-05-27 Thread via GitHub
giucris opened a new issue, #769: URL: https://github.com/apache/iceberg-python/issues/769 ### Apache Iceberg version 0.6.0 (latest release) ### Please describe the bug 🐞 During the development of a new feature I encountered a behaviour in the implementation of the SqlCa

Re: [I] pyiceberg causing mypy to raise Exception [iceberg-python]

2024-05-27 Thread via GitHub
Quixotical commented on issue #768: URL: https://github.com/apache/iceberg-python/issues/768#issuecomment-2133826721 actually we can close this out, I was using an older version of mypy and upgrading to the newest version seems to have resolved my issue. Thanks for inquiring about this! -

Re: [I] pyiceberg causing mypy to raise Exception [iceberg-python]

2024-05-27 Thread via GitHub
Quixotical closed issue #768: pyiceberg causing mypy to raise Exception URL: https://github.com/apache/iceberg-python/issues/768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Kafka Connect: Commit coordination [iceberg]

2024-05-27 Thread via GitHub
fqaiser94 commented on code in PR #10351: URL: https://github.com/apache/iceberg/pull/10351#discussion_r1616117443 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/IcebergSinkTask.java: ## @@ -0,0 +1,103 @@ +/* + * Licensed to the Apache Software Foundatio

Re: [I] PyIceberg Near-Term Roadmap [iceberg-python]

2024-05-27 Thread via GitHub
tusharchou commented on issue #736: URL: https://github.com/apache/iceberg-python/issues/736#issuecomment-2133660186 @Fokko can we add issues for creating tests and documentation for the new features of 0.7.0 as good first issues? -- This is an automated message from the Apache Git Servic

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-27 Thread via GitHub
liurenjie1024 commented on PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#issuecomment-2133627116 cc @Xuanwo @Fokko @sdd @marvinlanhenke What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Make current-snapshot-id optional while maintaining backwards compatibility [iceberg-rust]

2024-05-27 Thread via GitHub
liurenjie1024 commented on code in PR #374: URL: https://github.com/apache/iceberg-rust/pull/374#discussion_r1613216338 ## crates/iceberg/src/spec/table_metadata.rs: ## @@ -472,6 +474,19 @@ pub(super) mod _serde { pub default_sort_order_id: Option, } +// Skip

Re: [PR] Feature: Schema into_builder method [iceberg-rust]

2024-05-27 Thread via GitHub
liurenjie1024 commented on PR #381: URL: https://github.com/apache/iceberg-rust/pull/381#issuecomment-2133620343 Let's wait for a while to see what others think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Prevent deadlock in Jackson [iceberg]

2024-05-27 Thread via GitHub
Fokko commented on PR #10379: URL: https://github.com/apache/iceberg/pull/10379#issuecomment-2133619097 @snazy Good point. I've added a point for the sync this Wednesday: https://docs.google.com/document/d/1YuGhUdukLP5gGiqCbk0A5_Wifqe2CZWgOd3TbhY3UQg/ -- This is an automated message from

Re: [PR] Prevent deadlock in Jackson [iceberg]

2024-05-27 Thread via GitHub
snazy commented on PR #10379: URL: https://github.com/apache/iceberg/pull/10379#issuecomment-2133611386 In Iceberg maybe - but that doesn't prevent other code to initialize the classes concurrently -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] feat: make file scan task serializable [iceberg-rust]

2024-05-27 Thread via GitHub
liurenjie1024 merged PR #377: URL: https://github.com/apache/iceberg-rust/pull/377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Prevent deadlock in Jackson [iceberg]

2024-05-27 Thread via GitHub
Fokko commented on PR #10379: URL: https://github.com/apache/iceberg/pull/10379#issuecomment-2133609274 @snazy Thanks for working on this. Have you encountered a deadlock? It looks like we have some additional synchronization logic on the Iceberg side. -- This is an automated message from

Re: [PR] Prevent deadlock in Jackson [iceberg]

2024-05-27 Thread via GitHub
snazy commented on PR #10379: URL: https://github.com/apache/iceberg/pull/10379#issuecomment-2133603788 @Fokko thanks for merging. I suspect the deadlock is a big enough risk to include this change in 1.5.3? -- This is an automated message from the Apache Git Service. To respond to the me

Re: [PR] Build: Bump nessie from 0.82.0 to 0.83.2 [iceberg]

2024-05-27 Thread via GitHub
Fokko merged PR #10381: URL: https://github.com/apache/iceberg/pull/10381 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Prevent deadlock in Jackson [iceberg]

2024-05-27 Thread via GitHub
Fokko merged PR #10379: URL: https://github.com/apache/iceberg/pull/10379 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

[PR] Feature: Schema into_builder method [iceberg-rust]

2024-05-27 Thread via GitHub
c-thiel opened a new pull request, #381: URL: https://github.com/apache/iceberg-rust/pull/381 Convert a schema back to a builder -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

Re: [I] pyiceberg causing mypy to raise Exception [iceberg-python]

2024-05-27 Thread via GitHub
ndrluis commented on issue #768: URL: https://github.com/apache/iceberg-python/issues/768#issuecomment-2133428433 Hello @Quixotical, thank you for reporting this issue. Can you share a minimal setup that reproduces this error? -- This is an automated message from the Apache Git Service. T

Re: [PR] Spark Action to Analyze table [iceberg]

2024-05-27 Thread via GitHub
karuppayya commented on code in PR #10288: URL: https://github.com/apache/iceberg/pull/10288#discussion_r1615994308 ## spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/AnalyzeTableSparkAction.java: ## @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Found

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615942072 ## pyiceberg/table/__init__.py: ## @@ -2897,12 +2987,152 @@ def _commit(self) -> UpdatesAndRequirements: ), ( AssertTable

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615924919 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingSn

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615924919 ## pyiceberg/table/__init__.py: ## @@ -2931,14 +3161,52 @@ def _deleted_entries(self) -> List[ManifestEntry]: return [] -class OverwriteFiles(_MergingSn

Re: [PR] Support partial deletes [iceberg-python]

2024-05-27 Thread via GitHub
Fokko commented on code in PR #569: URL: https://github.com/apache/iceberg-python/pull/569#discussion_r1615912422 ## pyiceberg/table/__init__.py: ## @@ -443,6 +471,74 @@ def overwrite( for data_file in data_files: update_snapshot.append_data

Re: [PR] Build: Bump com.google.cloud:libraries-bom from 26.28.0 to 26.39.0 [iceberg]

2024-05-27 Thread via GitHub
Fokko commented on PR #10313: URL: https://github.com/apache/iceberg/pull/10313#issuecomment-2133187879 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. T

Re: [PR] Build: Bump org.springframework:spring-web from 5.3.35 to 5.3.36 [iceberg]

2024-05-27 Thread via GitHub
Fokko merged PR #10382: URL: https://github.com/apache/iceberg/pull/10382 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump io.airlift:aircompressor from 0.26 to 0.27 [iceberg]

2024-05-27 Thread via GitHub
Fokko merged PR #10383: URL: https://github.com/apache/iceberg/pull/10383 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [PR] Build: Bump io.netty:netty-buffer from 4.1.109.Final to 4.1.110.Final [iceberg]

2024-05-27 Thread via GitHub
Fokko merged PR #10384: URL: https://github.com/apache/iceberg/pull/10384 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa

Re: [I] Javadoc issues [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2133149811 I didn't start yet. Up to you. I can start on it tonight or you can take it. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [I] Have a better error message in case of a schema mismatch [iceberg-python]

2024-05-27 Thread via GitHub
MehulBatra commented on issue #328: URL: https://github.com/apache/iceberg-python/issues/328#issuecomment-2133147431 @Fokko If the situation allows I would love to take a shot at it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

Re: [PR] Docs: Add archive for documentations older than 1.4.0 [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on PR #10374: URL: https://github.com/apache/iceberg/pull/10374#issuecomment-2133145023 cc @nastra @Fokko -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

Re: [I] Javadoc issues [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2133146589 @jbonofre Are you working on all these issues? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Core: FileSystemTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

2024-05-27 Thread via GitHub
BsoBird commented on PR #9546: URL: https://github.com/apache/iceberg/pull/9546#issuecomment-2133100862 @nastra Hi.If you have time, can you check this? tks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] Using the Iceberg catalog in your file system [iceberg]

2024-05-27 Thread via GitHub
BsoBird commented on issue #10326: URL: https://github.com/apache/iceberg/issues/10326#issuecomment-2133087008 Hi, I've done some work on fixing hadoop_catalog before. In my experience, to use a filesystem-based catalog, you currently need to rely on the filesystem to provide atomic

Re: [I] iceberg TTL set [iceberg]

2024-05-27 Thread via GitHub
BsoBird commented on issue #10372: URL: https://github.com/apache/iceberg/issues/10372#issuecomment-2133063651 你是指iceberg可以支持将数据写入到多级存储的不同路径中么? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2024-05-27 Thread via GitHub
amit-cloudinary commented on PR #8797: URL: https://github.com/apache/iceberg/pull/8797#issuecomment-2132978069 > > @zinking - if i can assist with this please let me know (would love to contribute if possible) > > feel free to pick it up here. Ill give it a try :) ,thanks --

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1615673050 ## build.gradle: ## @@ -139,35 +137,6 @@ subprojects { apply plugin: 'java-library' - if (project.name in REVAPI_PROJECTS) { -apply plugin: 'com.palantir

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2024-05-27 Thread via GitHub
zinking commented on PR #8797: URL: https://github.com/apache/iceberg/pull/8797#issuecomment-2132920372 > @zinking - if i can assist with this please let me know (would love to contribute if possible) feel free to pick it up here. -- This is an automated message from the Apache Gi

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1615666783 ## build.gradle: ## @@ -139,35 +137,6 @@ subprojects { apply plugin: 'java-library' - if (project.name in REVAPI_PROJECTS) { -apply plugin: 'com.palantir

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1615666439 ## dev/revapi: ## @@ -0,0 +1,81 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

Re: [PR] Docs: add metrics-reporting back [iceberg]

2024-05-27 Thread via GitHub
manuzhang commented on code in PR #10377: URL: https://github.com/apache/iceberg/pull/10377#discussion_r1615665843 ## docs/mkdocs.yml: ## @@ -27,6 +27,7 @@ nav: - configuration.md - evolution.md - maintenance.md +- metrics-reporting.md Review Comment: I en

Re: [PR] Spark: support rewrite on specified target branch [iceberg]

2024-05-27 Thread via GitHub
amit-cloudinary commented on PR #8797: URL: https://github.com/apache/iceberg/pull/8797#issuecomment-2132899272 @zinking - if i can assist with this please let me know (would love to contribute if possible) -- This is an automated message from the Apache Git Service. To respond to the m

Re: [I] Support different JDBC backend in the `JdbcCatalog` [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on issue #9733: URL: https://github.com/apache/iceberg/issues/9733#issuecomment-2132884050 I'm resuming the branch here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Build: Bump software.amazon.awssdk:bom from 2.25.57 to 2.25.60 [iceberg]

2024-05-27 Thread via GitHub
nastra merged PR #10385: URL: https://github.com/apache/iceberg/pull/10385 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Docs: add metrics-reporting back [iceberg]

2024-05-27 Thread via GitHub
nastra merged PR #10377: URL: https://github.com/apache/iceberg/pull/10377 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap

Re: [PR] Docs: add metrics-reporting back [iceberg]

2024-05-27 Thread via GitHub
nastra commented on code in PR #10377: URL: https://github.com/apache/iceberg/pull/10377#discussion_r1615616956 ## docs/mkdocs.yml: ## @@ -27,6 +27,7 @@ nav: - configuration.md - evolution.md - maintenance.md +- metrics-reporting.md Review Comment: Can you

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-27 Thread via GitHub
nastra commented on code in PR #10386: URL: https://github.com/apache/iceberg/pull/10386#discussion_r1615606689 ## build.gradle: ## @@ -139,35 +137,6 @@ subprojects { apply plugin: 'java-library' - if (project.name in REVAPI_PROJECTS) { -apply plugin: 'com.palantir.r

Re: [I] iceberg TTL set [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on issue #10372: URL: https://github.com/apache/iceberg/issues/10372#issuecomment-2132811702 By TTL, you mean on a table snapshot ? Table snapshot has a timestamp, so, we could have a TTL based on snapshots timestamp and remove "expired" snapshot. Is it what you

Re: [I] Javadoc issues [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on issue #10378: URL: https://github.com/apache/iceberg/issues/10378#issuecomment-2132800947 Good catch @manuzhang ! Let me fix that on the website. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] feat: make file scan task serializable [iceberg-rust]

2024-05-27 Thread via GitHub
Fokko commented on PR #377: URL: https://github.com/apache/iceberg-rust/pull/377#issuecomment-2132797415 I'm in favor of this change since this will keep the API much cleaner. The ManifestEntry will change over time, maybe good to keep Iceberg-specific things out of there. The pattern of th

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.27.15 to 1.27.16 [iceberg-go]

2024-05-27 Thread via GitHub
Fokko commented on PR #90: URL: https://github.com/apache/iceberg-go/pull/90#issuecomment-2132793690 @dependabot rebase -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/service/glue from 1.80.3 to 1.81.1 [iceberg-go]

2024-05-27 Thread via GitHub
Fokko merged PR #88: URL: https://github.com/apache/iceberg-go/pull/88 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache

Re: [PR] Flink: refactor sink shuffling statistics collection [iceberg]

2024-05-27 Thread via GitHub
yegangy0718 commented on code in PR #10331: URL: https://github.com/apache/iceberg/pull/10331#discussion_r1615024325 ## flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/AggregatedStatisticsTracker.java: ## @@ -30,71 +42,99 @@ * {@link AggregatedStatistics}

Re: [PR] [Build] Add a script to execute revapi without gradle plugin [iceberg]

2024-05-27 Thread via GitHub
jbonofre commented on PR #10386: URL: https://github.com/apache/iceberg/pull/10386#issuecomment-2132784942 Some details: 1. `dev/revapi` script checks the different project. 2. I just create a simple gradle task to execute the script with `gradle revapi` 3. I propose to include `dev