Re: [PR] Flink: Bump minor versions [iceberg]

pvary merged PR #9875: URL: https://github.com/apache/iceberg/pull/9875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] feat: complete the basic file scan [iceberg-rust]

ZENOTME closed pull request #199: feat: complete the basic file scan URL: https://github.com/apache/iceberg-rust/pull/199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] feat: complete the basic file scan [iceberg-rust]

ZENOTME commented on PR #199: URL: https://github.com/apache/iceberg-rust/pull/199#issuecomment-1990916957 completed in #207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

liurenjie1024 commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1520950487 ## crates/catalog/rest/src/catalog.rs: ## @@ -113,6 +117,19 @@ impl RestCatalogConfig { ), ]); +if let Some(token) = self.props

Re: [PR] Spark SystemFunctions are not pushed down during JOIN [iceberg]

tmnd1991 commented on PR #9233: URL: https://github.com/apache/iceberg/pull/9233#issuecomment-1990929877 @aokolnychyi I see you fixed part of this in #9873 but Spark 3.4 looks stil bugged on main. Do you have any time to give me a feedback on this? -- This is an automated message from the

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

liurenjie1024 commented on PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#issuecomment-1990934283 BTW, is it possible to update [examples](https://github.com/apache/iceberg-rust/blob/be65c89001307d56e1c7f2ca000ef17e026e7c23/crates/examples/src/rest_catalog_namespace.rs) so th

Re: [I] Add metadata tables [iceberg-python]

Fokko commented on issue #511: URL: https://github.com/apache/iceberg-python/issues/511#issuecomment-1990943100 @Gowthami03B That would be great, any specific one that you have in mind? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [I] Support Maintenance Operations on PyIceberg [iceberg-python]

Fokko commented on issue #516: URL: https://github.com/apache/iceberg-python/issues/516#issuecomment-1990945756 Metadata rewrites are already discussed in https://github.com/apache/iceberg-python/issues/270 I think snapshot expiration is one that we should also focus on. I think http

Re: [PR] Spark: IN clause on system function is not pushed down [iceberg]

tmnd1991 commented on PR #9192: URL: https://github.com/apache/iceberg/pull/9192#issuecomment-1990954605 @aokolnychyi I see you fixed part of this in https://github.com/apache/iceberg/pull/9873 but Spark 3.4 looks stil bugged on main. Do you have any time to give me a feedback on this? --

Re: [PR] Docs: Enhance Spark pages [iceberg]

nastra merged PR #9920: URL: https://github.com/apache/iceberg/pull/9920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [I] OSSFileIO not compatible with aliyun-sdk-oss higher than 3.11.3 [iceberg]

nastra commented on issue #9934: URL: https://github.com/apache/iceberg/issues/9934#issuecomment-1990990294 Currently the version being used in Iceberg is https://github.com/apache/iceberg/blob/b714978583f5ba32ec34396cb82b0a8f100bf837/gradle/libs.versions.toml#L24. I'm not sure who actively

Re: [I] how to create iceberg hive catalog-type catalog in flink with kerberos [iceberg]

HuiHang-Yu commented on issue #8213: URL: https://github.com/apache/iceberg/issues/8213#issuecomment-1990998871 Same question. It is ok to use hive catalog like below CREATE CATALOG myhive WITH ( 'type' = 'hive', 'hive-conf-dir' = 'xxx' ); Exception (org.ap

Re: [I] Read Parquet data file with projection [iceberg-rust]

liurenjie1024 commented on issue #244: URL: https://github.com/apache/iceberg-rust/issues/244#issuecomment-1991000253 > But currently the select API cannot select deeply nested fields. In fact, current api is supposed to allow user to select deeply nested fields like `select("person.

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521025132 ## crates/catalog/rest/src/catalog.rs: ## @@ -113,6 +117,19 @@ impl RestCatalogConfig { ), ]); +if let Some(token) = self.props.g

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521027528 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

Re: [I] Loading table from metadata file directly. [iceberg-rust]

a-agmon commented on issue #246: URL: https://github.com/apache/iceberg-rust/issues/246#issuecomment-1991011883 Hi @liurenjie1024 , Thanks for adding this in response to my question on Slack :-) I would be happy to work on this as good-first-issue if you think its a good idea. I h

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#issuecomment-1991016398 > Hi, @TennyZhuang Thanks for this pr! It looks great, but there are something missing: > > 1. We should try to refresh token when error happens > 2. Please make the prope

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521025132 ## crates/catalog/rest/src/catalog.rs: ## @@ -113,6 +117,19 @@ impl RestCatalogConfig { ), ]); +if let Some(token) = self.props.g

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521040362 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521041665 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521046528 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

Re: [PR] [WIP] Migrate Metadata files in Core to JUnit5 [iceberg]

nastra commented on code in PR #9927: URL: https://github.com/apache/iceberg/pull/9927#discussion_r1521054509 ## core/src/test/java/org/apache/iceberg/TableMetadataParserTest.java: ## @@ -70,24 +67,30 @@ public void testCompressionProperty() throws IOException { String loca

Re: [PR] Add doc fixes for 1.5.0 [iceberg]

Fokko commented on code in PR #9935: URL: https://github.com/apache/iceberg/pull/9935#discussion_r1521056978 ## 1.5.0/docs/spark-getting-started.md: ## @@ -123,18 +123,78 @@ SELECT * FROM local.db.table.snapshots; +-++---+---

Re: [PR] [WIP] Migrate Metadata files in Core to JUnit5 [iceberg]

tomtongue commented on code in PR #9927: URL: https://github.com/apache/iceberg/pull/9927#discussion_r1521061734 ## core/src/test/java/org/apache/iceberg/TableMetadataParserTest.java: ## @@ -70,24 +67,30 @@ public void testCompressionProperty() throws IOException { String l

Re: [PR] Add doc fixes for 1.5.0 [iceberg]

Fokko merged PR #9935: URL: https://github.com/apache/iceberg/pull/9935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

[I] Manage root of URL in manifests in one place per entire table. [iceberg]

PigNatovsky opened a new issue, #9936: URL: https://github.com/apache/iceberg/issues/9936 ### Feature Request / Improvement Currently manifests files contains full URL to every manifest/data file in the table. If we need to change location (for example Allxuio cache migration, or S3

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

liurenjie1024 commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521174970 ## crates/catalog/rest/tests/rest_catalog_test.rs: ## Review Comment: Could you add some tests here to mock fetch token process? ## crates/cata

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

liurenjie1024 commented on PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#issuecomment-1991227015 > In that version you referred, token refresh is not implemented, I prefer to add it in later PR, what do you think? Sounds reasonable to me. -- This is an automated mes

Re: [I] I'm curious about the `hive.metastore.disallow.incompatible.col.type.changes` settings. [iceberg]

nastra closed issue #9897: I'm curious about the `hive.metastore.disallow.incompatible.col.type.changes` settings. URL: https://github.com/apache/iceberg/issues/9897 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] I'm curious about the `hive.metastore.disallow.incompatible.col.type.changes` settings. [iceberg]

nastra commented on issue #9897: URL: https://github.com/apache/iceberg/issues/9897#issuecomment-1991240287 I agree with @pvary, this seems like a question that should be asked in the Hive community at https://github.com/apache/hive (or the Hive JIRA) -- This is an automated message from

Re: [I] Loading table from metadata file directly. [iceberg-rust]

liurenjie1024 commented on issue #246: URL: https://github.com/apache/iceberg-rust/issues/246#issuecomment-1991256797 Hi, @a-agmon I think both method could be provided and the second version could be implemented as a simple wrapper as the first one. What I want to add is that

Re: [I] About /v1/config REST API endpoint [iceberg]

ajantha-bhat commented on issue #9880: URL: https://github.com/apache/iceberg/issues/9880#issuecomment-1991260095 Thanks for the clarification. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [I] About /v1/config REST API endpoint [iceberg]

ajantha-bhat closed issue #9880: About /v1/config REST API endpoint URL: https://github.com/apache/iceberg/issues/9880 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [I] Loading table from metadata file directly. [iceberg-rust]

a-agmon commented on issue #246: URL: https://github.com/apache/iceberg-rust/issues/246#issuecomment-1991294429 Thanks @liurenjie1024 Sounds good to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] Docs: update document release steps [iceberg]

ajantha-bhat commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521213670 ## site/docs/how-to-release.md: ## @@ -311,74 +311,7 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Release

Re: [PR] Docs: update document release steps [iceberg]

ajantha-bhat commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521220929 ## site/docs/how-to-release.md: ## @@ -311,74 +311,7 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Release

Re: [PR] Open-api: update prefix param description [iceberg]

ajantha-bhat commented on PR #9870: URL: https://github.com/apache/iceberg/pull/9870#issuecomment-1991316873 ping. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubs

Re: [PR] Docs: update document release steps [iceberg]

bitsondatadev commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521228633 ## site/docs/how-to-release.md: ## @@ -311,74 +311,7 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Releas

Re: [PR] Docs: update document release steps [iceberg]

bitsondatadev commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521228633 ## site/docs/how-to-release.md: ## @@ -311,74 +311,7 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Releas

Re: [PR] revert apache 9726 [iceberg]

artyrian commented on PR #9882: URL: https://github.com/apache/iceberg/pull/9882#issuecomment-1991483223 @nastra sure I apologize for the confusion; the revert in the upstream repository was opened by mistake, instead of in a fork, as it was intended for debugging purposes. Th

Re: [PR] Kafka Connect: Record converters [iceberg]

fqaiser94 commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521338858 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] fix minor version for striclty libs versions [iceberg]

artyrian commented on PR #9886: URL: https://github.com/apache/iceberg/pull/9886#issuecomment-1991496911 During the assembly of release 1.4.3, classes from JDK 21 started to inadvertently mix into the runtime classpath. The build began to fail with the error java.lang.IllegalArgumentExcepti

Re: [PR] Add `table_exists` method to Catalog [iceberg-python]

anupam-saini commented on code in PR #512: URL: https://github.com/apache/iceberg-python/pull/512#discussion_r1521352741 ## pyiceberg/catalog/__init__.py: ## @@ -646,6 +646,13 @@ def purge_table(self, identifier: Union[str, Identifier]) -> None: delete_files(io, prev_m

[I] Add session configuration for `adaptiveSplitSizeEnabled` [iceberg]

wForget opened a new issue, #9937: URL: https://github.com/apache/iceberg/issues/9937 ### Feature Request / Improvement When DRA is enabled and initialExecutors is set to 0, we read from a small iceberg table and write to other table (ETL scenario). We found that more parallelism lea

Re: [PR] Kafka Connect: Record converters [iceberg]

fqaiser94 commented on PR #9641: URL: https://github.com/apache/iceberg/pull/9641#issuecomment-1991551990 What's expected to be included in the next kafka-connect PR? Asking b/c I'm doing some work on implementing zombie-fencing in tabular/iceberg-kafka-connect currently and we may w

Re: [PR] Docs: update document release steps [iceberg]

ajantha-bhat commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521417655 ## site/docs/how-to-release.md: ## @@ -311,74 +311,7 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Release

Re: [PR] Update site to 1.5.0 docs [iceberg]

ajantha-bhat commented on code in PR #9931: URL: https://github.com/apache/iceberg/pull/9931#discussion_r1521419850 ## site/mkdocs.yml: ## @@ -78,7 +78,7 @@ markdown_extensions: permalink: 🔗 extra: - icebergVersion: '1.4.3' + icebergVersion: '1.5.0' Review Comment:

Re: [PR] fix minor version for striclty libs versions [iceberg]

nastra commented on PR #9886: URL: https://github.com/apache/iceberg/pull/9886#issuecomment-1991617035 Just merging this change to the 1.4.x branch won't make this immediately available. The community would have to do an actual 1.4.4 release in such a case. We just released 1.5.0, so as a f

Re: [PR] Core: add support to add custom schemes via properties in ResolvingFileIO [iceberg]

jbonofre commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1991621141 @danielcweeks stupid question: I can provide my custom `FileIO` impl via configuration, but I can have only one custom impl. If the user wants to mix `Impl1` with scheme `impl1:` and `Im

Re: [PR] Docs: update document release steps [iceberg]

nastra commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521442735 ## site/docs/how-to-release.md: ## @@ -311,74 +311,9 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Release Do

Re: [PR] Docs: update document release steps [iceberg]

ajantha-bhat commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521449612 ## site/docs/how-to-release.md: ## @@ -311,74 +311,9 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Release

Re: [PR] Core: Support appending files with different specs [iceberg]

fqaiser94 commented on code in PR #9860: URL: https://github.com/apache/iceberg/pull/9860#discussion_r1521454344 ## core/src/test/java/org/apache/iceberg/TestMergeAppend.java: ## @@ -92,6 +96,92 @@ public void testEmptyTableAppend() { statuses(Status.ADDED, Status.ADDED

Re: [PR] Core: Support appending files with different specs [iceberg]

fqaiser94 commented on code in PR #9860: URL: https://github.com/apache/iceberg/pull/9860#discussion_r1521453498 ## core/src/test/java/org/apache/iceberg/TestOverwrite.java: ## @@ -293,23 +332,178 @@ public void testValidatedOverwriteWithAppendOutsideOfDeleteMetrics() { }

Re: [PR] Core: Support appending files with different specs [iceberg]

nastra commented on code in PR #9860: URL: https://github.com/apache/iceberg/pull/9860#discussion_r1521469812 ## core/src/test/java/org/apache/iceberg/TestOverwrite.java: ## @@ -293,23 +332,178 @@ public void testValidatedOverwriteWithAppendOutsideOfDeleteMetrics() { }

[PR] Fix `TestStrictMetricsEvaluator` assertion message [iceberg]

Fokko opened a new pull request, #9938: URL: https://github.com/apache/iceberg/pull/9938 All values (`5` and `6`) are not between the upper and lower bound of `[30, 79]`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[PR] Add StrictMetricsEvaluator [iceberg-python]

Fokko opened a new pull request, #518: URL: https://github.com/apache/iceberg-python/pull/518 This will enable use to delete whole datafiles by evaluating the metrics, and not needing to open the Parquet files. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Library public api isolation and import decoupling [iceberg-python]

ndrluis commented on issue #499: URL: https://github.com/apache/iceberg-python/issues/499#issuecomment-1991736688 We have some news about the issue: mypy has implemented a rule that could protect us. You can find more details here: https://mypy.readthedocs.io/en/stable/config_file.html#conf

Re: [PR] Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo… [iceberg]

zhangbutao commented on PR #8202: URL: https://github.com/apache/iceberg/pull/8202#issuecomment-1991746964 Any update? hope we can continue to push this review forward. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to G

Re: [PR] Sql catalog [iceberg-rust]

JanKaul commented on code in PR #229: URL: https://github.com/apache/iceberg-rust/pull/229#discussion_r1521544992 ## crates/catalog/sql/src/catalog.rs: ## @@ -0,0 +1,397 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements.

Re: [PR] Docs: update document release steps [iceberg]

bitsondatadev commented on code in PR #9921: URL: https://github.com/apache/iceberg/pull/9921#discussion_r1521552842 ## site/docs/how-to-release.md: ## @@ -311,74 +311,9 @@ Create a PR in the `iceberg` repo to make revapi run on the new release. For an ### Documentation Releas

Re: [I] Read Parquet data file with projection [iceberg-rust]

viirya commented on issue #244: URL: https://github.com/apache/iceberg-rust/issues/244#issuecomment-1991782353 Ah, I see. Thanks for the point. I began looking at Java implementation of #251. -- This is an automated message from the Apache Git Service. To respond to the message, please l

Re: [PR] Core: Add KLL Datasketch and Hive ColumnStatisticsObj as standard blo… [iceberg]

tdcmeehan commented on code in PR #8202: URL: https://github.com/apache/iceberg/pull/8202#discussion_r1521568921 ## format/puffin-spec.md: ## @@ -126,6 +126,23 @@ The blob metadata for this blob may include following properties: - `ndv`: estimate of number of distinct values

Re: [I] Implement `PruneColumns` for `Schema`. [iceberg-rust]

Dysprosium0626 commented on issue #251: URL: https://github.com/apache/iceberg-rust/issues/251#issuecomment-1991812879 Hi @viirya @liurenjie1024 , Can I help with this issue and I have nearly completing implement `PruneColumn` for Schema. See https://github.com/Dysprosium0626/iceberg-rus

Re: [PR] Kafka Connect: Record converters [iceberg]

bryanck commented on PR #9641: URL: https://github.com/apache/iceberg/pull/9641#issuecomment-1991814651 > What's expected to be included in the next kafka-connect PR? > > Asking b/c I'm doing some work on implementing zombie-fencing in tabular/iceberg-kafka-connect currently and we ma

Re: [PR] Kafka Connect: Record converters [iceberg]

bryanck commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521621409 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

TennyZhuang commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521635258 ## crates/catalog/rest/src/catalog.rs: ## @@ -497,13 +516,56 @@ impl RestCatalog { client: config.try_create_rest_client()?, config,

[PR] [docs] Fix broken links to config reference [iceberg]

rmoff opened a new pull request, #9939: URL: https://github.com/apache/iceberg/pull/9939 Some of the Flink and Spark docs currently have a broken link to the Table Configuration page. This PR fixes that. -- This is an automated message from the Apache Git Service. To respond to th

Re: [I] Add hive metastore catalog support [iceberg-rust]

marvinlanhenke commented on issue #113: URL: https://github.com/apache/iceberg-rust/issues/113#issuecomment-1991881914 @liurenjie1024 just a quick update: I've implemented most of the stuff I can do without the `TableMetadataBuilder`. Also you can assign this issue to me, if you'd like.

Re: [PR] [WIP] Add Data Files from Parquet Files [iceberg-python]

syun64 commented on PR #506: URL: https://github.com/apache/iceberg-python/pull/506#issuecomment-1991884158 Updates from offline discussions: 1. The task of creating the correct Iceberg Table Schema with the desired Partition Spec, from an external table (like Hive) is out of scope of thi

[PR] Rename stat_table to table_exists in Catalog trait [iceberg-rust]

lewiszlw opened a new pull request, #257: URL: https://github.com/apache/iceberg-rust/pull/257 Fixes https://github.com/apache/iceberg-rust/issues/236. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Rename stat_table to table_exists in Catalog trait [iceberg-rust]

Xuanwo commented on code in PR #257: URL: https://github.com/apache/iceberg-rust/pull/257#discussion_r1521718679 ## crates/iceberg/src/catalog/mod.rs: ## @@ -83,7 +83,7 @@ pub trait Catalog: Debug + Sync + Send { async fn drop_table(&self, table: &TableIdent) -> Result<()>;

Re: [PR] Rename stat_table to table_exists in Catalog trait [iceberg-rust]

Xuanwo commented on code in PR #257: URL: https://github.com/apache/iceberg-rust/pull/257#discussion_r1521718679 ## crates/iceberg/src/catalog/mod.rs: ## @@ -83,7 +83,7 @@ pub trait Catalog: Debug + Sync + Send { async fn drop_table(&self, table: &TableIdent) -> Result<()>;

Re: [PR] Build: Align Jackson versions [iceberg]

nastra commented on code in PR #9925: URL: https://github.com/apache/iceberg/pull/9925#discussion_r1521727110 ## gradle/libs.versions.toml: ## @@ -193,8 +191,6 @@ flink118-runtime = { module = "org.apache.flink:flink-runtime", version.ref = "f flink118-test-utils = { module =

Re: [PR] Kafka Connect: Record converters [iceberg]

ajreid21 commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521739300 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Foundat

Re: [I] Implement `PruneColumns` for `Schema`. [iceberg-rust]

viirya commented on issue #251: URL: https://github.com/apache/iceberg-rust/issues/251#issuecomment-1992036933 Of course, yes. I plan to implement this but as you are close to finish it. I can help review once you submit the PR. 🙂 -- This is an automated message from the Apache Git Servi

Re: [I] Implement covnersion from `ArrowSchema` to iceberg `Schema`. [iceberg-rust]

viirya commented on issue #252: URL: https://github.com/apache/iceberg-rust/issues/252#issuecomment-1992039740 I will work on this for #244. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [PR] Kafka Connect: Record converters [iceberg]

bryanck commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521780274 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Foundati

[PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

nastra opened a new pull request, #9940: URL: https://github.com/apache/iceberg/pull/9940 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

Re: [PR] Kafka Connect: Record converters [iceberg]

bryanck commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521807811 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] Build: Align Jackson versions [iceberg]

nastra merged PR #9925: URL: https://github.com/apache/iceberg/pull/9925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Build: Align Jackson versions [iceberg]

nastra commented on PR #9925: URL: https://github.com/apache/iceberg/pull/9925#issuecomment-1992118120 thanks for reviewing @Fokko @amogh-jahagirdar -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Handle Arrow `large_string` data type [iceberg-python]

kevinjqliu commented on issue #520: URL: https://github.com/apache/iceberg-python/issues/520#issuecomment-1992122868 Looks like it was added in #382 for #226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521821213 ## aws/src/main/resources/s3-signer-open-api.yaml: ## @@ -56,13 +56,18 @@ servers: description: Optional prefix to be appended to all routes default: "

Re: [PR] Fix pagination description based on new feedback [iceberg]

rahil-c commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1521823394 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to ma

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521825337 ## open-api/rest-catalog-open-api.yaml: ## @@ -80,12 +88,14 @@ paths: " All REST clients should first call this route to get catalog configuration

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521825954 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,6 +61,14 @@ security: - OAuth2: [catalog] - BearerAuth: [] +tags: + - name: views +description: Requires s

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521826641 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,6 +61,14 @@ security: - OAuth2: [catalog] - BearerAuth: [] +tags: Review Comment: I think that we should ad

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521829334 ## open-api/rest-catalog-open-api.yaml: ## @@ -114,7 +124,8 @@ paths: }, "defaults": { "clients": "4" -

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521830908 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,6 +61,14 @@ security: - OAuth2: [catalog] - BearerAuth: [] +tags: Review Comment: I'd like to add to the ov

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521831389 ## open-api/rest-catalog-open-api.yaml: ## @@ -1559,6 +1578,22 @@ components: type: string description: Properties that should be us

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

rdblue commented on PR #9940: URL: https://github.com/apache/iceberg/pull/9940#issuecomment-1992146984 @nastra, this is a great start. I think we also want to add a section in the overall API description that covers how tags are used when there is a matching capability. That is, capability

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

flyrain commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521837108 ## crates/catalog/rest/src/catalog.rs: ## @@ -113,6 +117,19 @@ impl RestCatalogConfig { ), ]); +if let Some(token) = self.props.get("

Re: [I] Manifest builder length [iceberg-go]

thorfour closed issue #69: Manifest builder length URL: https://github.com/apache/iceberg-go/issues/69 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: i

Re: [I] Manifest builder length [iceberg-go]

thorfour commented on issue #69: URL: https://github.com/apache/iceberg-go/issues/69#issuecomment-1992166800 Ah I realize now that I was confused by the comment on the `ManifestFile interface` ```bash // Length is the length in bytes of the manifest file. ``` Wherea

Re: [I] Proxy support unavailable for iceberg spark client [iceberg]

syun64 commented on issue #9174: URL: https://github.com/apache/iceberg/issues/9174#issuecomment-1992179279 Maybe having specific proxy configurations for the REST Catalog would be better than relying on the SystemProperties? It *looks* like the connection manager is taking the config

Re: [PR] Kafka Connect: Record converters [iceberg]

fqaiser94 commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521874230 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]

flyrain commented on code in PR #9889: URL: https://github.com/apache/iceberg/pull/9889#discussion_r1521879235 ## docs/docs/spark-procedures.md: ## @@ -756,8 +756,8 @@ Creates a view that contains the changes from a given table. | `table` | ✔️ | string

Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]

flyrain commented on code in PR #9889: URL: https://github.com/apache/iceberg/pull/9889#discussion_r1521884083 ## docs/docs/spark-procedures.md: ## @@ -823,6 +823,8 @@ second snapshot deleted 1 record. |2 | Bob |INSERT |0 |5390529835796506035| |1 | Alice

[PR] Add partition stats in snapshot summary [iceberg-python]

jqin61 opened a new pull request, #521: URL: https://github.com/apache/iceberg-python/pull/521 Changes include: - add write.summary.partition-limit to table properties - add changed-partition-count to snapshot summary - add partition stats. these are key value pairs whose key is

Re: [PR] Add partition stats in snapshot summary [iceberg-python]

Fokko commented on PR #521: URL: https://github.com/apache/iceberg-python/pull/521#issuecomment-1992323876 > @Fokko As we were discussing last time, it looks like the trustPartitionMetrics flag is always set to true in spark, so I ignored the flag. Please let me know the concerns if there i

  1   2   >