Re: [I] how to create iceberg hive catalog-type catalog in flink with kerberos [iceberg]

2024-03-12 Thread via GitHub
pvary commented on issue #8213: URL: https://github.com/apache/iceberg/issues/8213#issuecomment-1993682192 @HuiHang-Yu: The correct Flonk catalog configuration is documented in the [doc](https://iceberg.apache.org/docs/1.5.0/flink/#catalog-configuration)s. Kerberos configuration should be d

Re: [PR] Hive: Add View support for HIVE catalog [iceberg]

2024-03-12 Thread via GitHub
nk1506 commented on code in PR #9852: URL: https://github.com/apache/iceberg/pull/9852#discussion_r1522568230 ## core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java: ## @@ -309,65 +304,20 @@ protected enum CommitStatus { * @return Commit Status of Success

Re: [PR] Build: Bump nessie from 0.77.1 to 0.78.0 [iceberg]

2024-03-12 Thread via GitHub
dependabot[bot] commented on PR #9909: URL: https://github.com/apache/iceberg/pull/9909#issuecomment-1993614406 OK, I won't notify you again about this release, but will get in touch when a new version is available. You can also ignore all major, minor, or patch releases for a dependency by

Re: [PR] Build: Bump nessie from 0.77.1 to 0.78.0 [iceberg]

2024-03-12 Thread via GitHub
ajantha-bhat commented on PR #9909: URL: https://github.com/apache/iceberg/pull/9909#issuecomment-1993614382 we can go with the newly released [Nessie 0.79.0](https://github.com/projectnessie/nessie/releases/tag/nessie-0.79.0) [Latest](https://github.com/projectnessie/nessie/releases/latest

Re: [PR] Build: Bump nessie from 0.77.1 to 0.78.0 [iceberg]

2024-03-12 Thread via GitHub
ajantha-bhat closed pull request #9909: Build: Bump nessie from 0.77.1 to 0.78.0 URL: https://github.com/apache/iceberg/pull/9909 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[PR] Fix hive mapreduce insert error in kerberos environmentfix hiveme insert error [iceberg]

2024-03-12 Thread via GitHub
ming95 opened a new pull request, #9944: URL: https://github.com/apache/iceberg/pull/9944 ### What changes were proposed in this pull request? When we are in kerberos environment, use hive mr to insert data into iceberg table, we will get Exception : ``` Caused by: M

Re: [I] Loading table from metadata file directly. [iceberg-rust]

2024-03-12 Thread via GitHub
a-agmon commented on issue #246: URL: https://github.com/apache/iceberg-rust/issues/246#issuecomment-1993467063 Hi @liurenjie1024 , Please take a look at the [PR](https://github.com/apache/iceberg-rust/pull/259) How does this look? -- This is an automated message from the Apache G

Re: [PR] Add local nightly build to test current docs changes [iceberg]

2024-03-12 Thread via GitHub
bitsondatadev commented on PR #9943: URL: https://github.com/apache/iceberg/pull/9943#issuecomment-1993461676 CC: @manuzhang I can't add you as a reviewer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[PR] feat (static table): implement a read-only table struct loaded from metadata [iceberg-rust]

2024-03-12 Thread via GitHub
a-agmon opened a new pull request, #259: URL: https://github.com/apache/iceberg-rust/pull/259 The PR adds a `StaticTable` struct that encapsulates `Table` and enables loading a table from meta-data, regardless of a catalog but provides only read-only access to meta-data and scan. This a

[PR] Add local nightly build to test current docs changes [iceberg]

2024-03-12 Thread via GitHub
bitsondatadev opened a new pull request, #9943: URL: https://github.com/apache/iceberg/pull/9943 Add local nightly build to test current docs changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522331514 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,28 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

[I] Table Change Detection in Iceberg Rest-API [iceberg]

2024-03-12 Thread via GitHub
mingnuj opened a new issue, #9942: URL: https://github.com/apache/iceberg/issues/9942 ### Feature Request / Improvement When using the Iceberg Rest-API Catalog, in order to detect changes in the table, it is necessary to retrieve all table metadata using loadTable and then check the

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522339857 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,104 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [I] Implement `PruneColumns` for `Schema`. [iceberg-rust]

2024-03-12 Thread via GitHub
liurenjie1024 commented on issue #251: URL: https://github.com/apache/iceberg-rust/issues/251#issuecomment-1992924747 @Dysprosium0626 Yes, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Rename stat_table to table_exists in Catalog trait [iceberg-rust]

2024-03-12 Thread via GitHub
liurenjie1024 commented on code in PR #257: URL: https://github.com/apache/iceberg-rust/pull/257#discussion_r1522332653 ## crates/iceberg/src/catalog/mod.rs: ## @@ -83,7 +83,7 @@ pub trait Catalog: Debug + Sync + Send { async fn drop_table(&self, table: &TableIdent) -> Resu

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522331514 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,28 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522322818 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,28 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [PR] Rename stat_table to table_exists in Catalog trait [iceberg-rust]

2024-03-12 Thread via GitHub
lewiszlw commented on code in PR #257: URL: https://github.com/apache/iceberg-rust/pull/257#discussion_r1522322828 ## crates/iceberg/src/catalog/mod.rs: ## @@ -83,7 +83,7 @@ pub trait Catalog: Debug + Sync + Send { async fn drop_table(&self, table: &TableIdent) -> Result<()

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522320972 ## open-api/rest-catalog-open-api.yaml: ## @@ -3804,6 +4016,21 @@ components: } } +UnprocessableEntityResponse: + description: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522311554 ## open-api/rest-catalog-open-api.yaml: ## @@ -2106,6 +2204,28 @@ components: items: $ref: '#/components/schemas/PartitionStatisticsFile' +

Re: [I] Manage root of URL in manifests in one place per entire table. [iceberg]

2024-03-12 Thread via GitHub
RussellSpitzer commented on issue #9936: URL: https://github.com/apache/iceberg/issues/9936#issuecomment-1992838051 We have had multiple requests for this or something like it , we have several design docs floating around for relative paths (basically this.) Since this will require a spec c

Re: [PR] Docs: Enhance Spark pages [iceberg]

2024-03-12 Thread via GitHub
manuzhang commented on PR #9920: URL: https://github.com/apache/iceberg/pull/9920#issuecomment-1992837667 @bitsondatadev @nastra There is already https://github.com/apache/iceberg/pull/9939 reverting some of the links. I can submit another PR fixing other link changes in the following

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522308416 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522302973 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522301796 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522299017 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522296831 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [I] Do we need to provide a more detailed document for HIVE ? [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1668: Do we need to provide a more detailed document for HIVE ? URL: https://github.com/apache/iceberg/issues/1668 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [I] Action: support spark3 and customer catalog [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1662: Action: support spark3 and customer catalog URL: https://github.com/apache/iceberg/issues/1662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [I] Outline approach for extending APIs to support cross table operations [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1647: Outline approach for extending APIs to support cross table operations URL: https://github.com/apache/iceberg/issues/1647 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [I] There is a vulnerability in spotless-plugin-gradle 3.14.0 ,upgrade recommended [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1642: There is a vulnerability in spotless-plugin-gradle 3.14.0 ,upgrade recommended URL: https://github.com/apache/iceberg/issues/1642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Iceberg table via Parquet files on S3 implementation [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1621: URL: https://github.com/apache/iceberg/issues/1621#issuecomment-1992784387 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] The AvroSchemaUtil will convert iceberg's optional map into confusing union. [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1578: URL: https://github.com/apache/iceberg/issues/1578#issuecomment-1992784322 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Consider adding extra validation for metrics config when creating a table [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1548: URL: https://github.com/apache/iceberg/issues/1548#issuecomment-1992784208 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Spark Predicate Pushdown for Metadata Table Scans [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1552: Spark Predicate Pushdown for Metadata Table Scans URL: https://github.com/apache/iceberg/issues/1552 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] Ignore rewrite commits while validating copy-on-write deletes in OverwriteFiles [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1547: Ignore rewrite commits while validating copy-on-write deletes in OverwriteFiles URL: https://github.com/apache/iceberg/issues/1547 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] numBuckets can cause division by zero [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1543: numBuckets can cause division by zero URL: https://github.com/apache/iceberg/issues/1543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Capture the template parameter in the tests for TestIcebergInputFormat#Factory [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1542: URL: https://github.com/apache/iceberg/issues/1542#issuecomment-1992784058 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Do we need to provide a more detailed document for HIVE ? [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1668: URL: https://github.com/apache/iceberg/issues/1668#issuecomment-1992784868 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Optimize generation of CombinedScanTask for RewriteDataFilesAction [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1667: Optimize generation of CombinedScanTask for RewriteDataFilesAction URL: https://github.com/apache/iceberg/issues/1667 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] Optimize generation of CombinedScanTask for RewriteDataFilesAction [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1667: URL: https://github.com/apache/iceberg/issues/1667#issuecomment-1992784817 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Action: support spark3 and customer catalog [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1662: URL: https://github.com/apache/iceberg/issues/1662#issuecomment-1992784772 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Extend HadoopTableOperations to also work with other FS guarantees [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1655: URL: https://github.com/apache/iceberg/issues/1655#issuecomment-1992784702 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Extend HadoopTableOperations to also work with other FS guarantees [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1655: Extend HadoopTableOperations to also work with other FS guarantees URL: https://github.com/apache/iceberg/issues/1655 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [I] Outline approach for extending APIs to support cross table operations [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1647: URL: https://github.com/apache/iceberg/issues/1647#issuecomment-1992784657 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] There is a vulnerability in spotless-plugin-gradle 3.14.0 ,upgrade recommended [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1642: URL: https://github.com/apache/iceberg/issues/1642#issuecomment-1992784588 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Implements the Flink source based on the new FLIP-27 interface [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1626: URL: https://github.com/apache/iceberg/issues/1626#issuecomment-1992784493 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Implement Initial Spark Structured Streaming Source [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1628: Implement Initial Spark Structured Streaming Source URL: https://github.com/apache/iceberg/issues/1628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] Implement Initial Spark Structured Streaming Source [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1628: URL: https://github.com/apache/iceberg/issues/1628#issuecomment-1992784539 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] class not found exception emr 6.1.0, iceberg-spark-0.9.1,iceberg-spark-runtime-0.9.1 java.lang.ClassNotFoundException: org.apache.spark.sql.sources.v2.DataSourceV2 [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1622: URL: https://github.com/apache/iceberg/issues/1622#issuecomment-1992784440 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Implements the Flink source based on the new FLIP-27 interface [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1626: Implements the Flink source based on the new FLIP-27 interface URL: https://github.com/apache/iceberg/issues/1626 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] class not found exception emr 6.1.0, iceberg-spark-0.9.1,iceberg-spark-runtime-0.9.1 java.lang.ClassNotFoundException: org.apache.spark.sql.sources.v2.DataSourceV2 [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1622: class not found exception emr 6.1.0, iceberg-spark-0.9.1,iceberg-spark-runtime-0.9.1 java.lang.ClassNotFoundException: org.apache.spark.sql.sources.v2.DataSourceV2 URL: https://github.com/apache/iceberg/issues/1622 -- This is an automated message from

Re: [I] Iceberg table via Parquet files on S3 implementation [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1621: Iceberg table via Parquet files on S3 implementation URL: https://github.com/apache/iceberg/issues/1621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

Re: [I] Spark Predicate Pushdown for Metadata Table Scans [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1552: URL: https://github.com/apache/iceberg/issues/1552#issuecomment-1992784264 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] The AvroSchemaUtil will convert iceberg's optional map into confusing union. [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1578: The AvroSchemaUtil will convert iceberg's optional map into confusing union. URL: https://github.com/apache/iceberg/issues/1578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Ignore rewrite commits while validating copy-on-write deletes in OverwriteFiles [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1547: URL: https://github.com/apache/iceberg/issues/1547#issuecomment-1992784155 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Consider adding extra validation for metrics config when creating a table [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1548: Consider adding extra validation for metrics config when creating a table URL: https://github.com/apache/iceberg/issues/1548 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] numBuckets can cause division by zero [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] commented on issue #1543: URL: https://github.com/apache/iceberg/issues/1543#issuecomment-1992784098 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Gi

Re: [I] Capture the template parameter in the tests for TestIcebergInputFormat#Factory [iceberg]

2024-03-12 Thread via GitHub
github-actions[bot] closed issue #1542: Capture the template parameter in the tests for TestIcebergInputFormat#Factory URL: https://github.com/apache/iceberg/issues/1542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522269323 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522268817 ## open-api/rest-catalog-open-api.yaml: ## @@ -2838,6 +2978,63 @@ components: additionalProperties: type: string +PreplanTableRequest: +

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522242401 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +3986,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesEmpt

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522236346 ## open-api/rest-catalog-open-api.yaml: ## @@ -3789,6 +3986,21 @@ components: EmptyResponse: $ref: '#/components/examples/ListNamespacesEmpt

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r150329 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,104 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1522215924 ## open-api/rest-catalog-open-api.yaml: ## @@ -537,6 +537,104 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1522201417 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to ma

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1522198390 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to ma

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1522198390 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to ma

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1522198390 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to ma

Re: [PR] build: Move back to the mmh3 [iceberg-python]

2024-03-12 Thread via GitHub
HonahX merged PR #460: URL: https://github.com/apache/iceberg-python/pull/460 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg

Re: [PR] Docs: Enhance Spark pages [iceberg]

2024-03-12 Thread via GitHub
bitsondatadev commented on PR #9920: URL: https://github.com/apache/iceberg/pull/9920#issuecomment-1992620793 ### Oops Apologies, as I asked this and accidentally made a review instead of posted a comment. I generally don't dislike GH's UI but that one has bitten me quite a few times

Re: [PR] docs: Remove roadmap [iceberg]

2024-03-12 Thread via GitHub
bitsondatadev commented on PR #9941: URL: https://github.com/apache/iceberg/pull/9941#issuecomment-1992600292 +1 That said, I think the better solution is to update that page to a list of projects in GitHub, and assign "owners" of different projects (ideally two people) who will regularly u

Re: [PR] Website: Add release schedule on the releases page [iceberg]

2024-03-12 Thread via GitHub
Fokko commented on code in PR #9666: URL: https://github.com/apache/iceberg/pull/9666#discussion_r1522089229 ## site/docs/releases.md: ## @@ -77,7 +77,11 @@ Apache Iceberg 1.4.3 was released on December 27, 2023. The main issue it solves - Core: Expired Snapshot files in a tra

[PR] docs: Remove roadmap [iceberg]

2024-03-12 Thread via GitHub
Fokko opened a new pull request, #9941: URL: https://github.com/apache/iceberg/pull/9941 For now. It is outdated and confuses users. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [I] Parquet bloom filter doesn't work with nested fields [iceberg]

2024-03-12 Thread via GitHub
hussein-awala commented on issue #9898: URL: https://github.com/apache/iceberg/issues/9898#issuecomment-1992533205 @amogh-jahagirdar Today I found out that I had this issue on a single table, I tried with nested and root fields, with single and multiple bloom filters, and none of them w

Re: [PR] Support of before and after actions in preorderschema traversal [iceberg-python]

2024-03-12 Thread via GitHub
Fokko closed pull request #42: Support of before and after actions in preorderschema traversal URL: https://github.com/apache/iceberg-python/pull/42 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] Support of before and after actions in preorderschema traversal [iceberg-python]

2024-03-12 Thread via GitHub
Fokko commented on PR #42: URL: https://github.com/apache/iceberg-python/pull/42#issuecomment-1992512626 Let's keep it as is, and close this for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Accessing Iceberg tables without catalog [iceberg]

2024-03-12 Thread via GitHub
Fokko commented on issue #5512: URL: https://github.com/apache/iceberg/issues/5512#issuecomment-1992494734 @jarias1 You can read a table directly from the metadata: https://iceberg.apache.org/javadoc/1.5.0/org/apache/iceberg/StaticTableOperations.html -- This is an automated message from

Re: [PR] Add issue template and docs for iceberg proposals [iceberg]

2024-03-12 Thread via GitHub
Fokko commented on code in PR #9932: URL: https://github.com/apache/iceberg/pull/9932#discussion_r1522025168 ## .github/ISSUE_TEMPLATE/iceberg_proposal.yml: ## @@ -0,0 +1,50 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

Re: [I] Support Maintenance Operations on PyIceberg [iceberg-python]

2024-03-12 Thread via GitHub
syun64 commented on issue #516: URL: https://github.com/apache/iceberg-python/issues/516#issuecomment-1992385342 Noted. Adjusting the Issue title as suggested 👍 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Implement `PruneColumns` for `Schema`. [iceberg-rust]

2024-03-12 Thread via GitHub
Fokko commented on issue #251: URL: https://github.com/apache/iceberg-rust/issues/251#issuecomment-1992383664 If you are looking for tests, we have them on PyIceberg: https://github.com/apache/iceberg-python/blob/0ee7f1233e062b17b03fccc1eac6449c72e575f0/tests/test_schema.py#L564-L819 -- T

Re: [PR] Add `table_exists` method to Catalog [iceberg-python]

2024-03-12 Thread via GitHub
Fokko commented on PR #512: URL: https://github.com/apache/iceberg-python/pull/512#issuecomment-1992363493 This looks good @anupam-saini, thanks for working on this. One more request, could you also update the docs? Otherwise people won't be able to find this :) -- This is an automated me

Re: [PR] Decouple imports reported by mypy linter [iceberg-python]

2024-03-12 Thread via GitHub
Fokko merged PR #519: URL: https://github.com/apache/iceberg-python/pull/519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.

Re: [PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-12 Thread via GitHub
Fokko commented on PR #521: URL: https://github.com/apache/iceberg-python/pull/521#issuecomment-1992323876 > @Fokko As we were discussing last time, it looks like the trustPartitionMetrics flag is always set to true in spark, so I ignored the flag. Please let me know the concerns if there i

[PR] Add partition stats in snapshot summary [iceberg-python]

2024-03-12 Thread via GitHub
jqin61 opened a new pull request, #521: URL: https://github.com/apache/iceberg-python/pull/521 Changes include: - add write.summary.partition-limit to table properties - add changed-partition-count to snapshot summary - add partition stats. these are key value pairs whose key is

Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]

2024-03-12 Thread via GitHub
flyrain commented on code in PR #9889: URL: https://github.com/apache/iceberg/pull/9889#discussion_r1521884083 ## docs/docs/spark-procedures.md: ## @@ -823,6 +823,8 @@ second snapshot deleted 1 record. |2 | Bob |INSERT |0 |5390529835796506035| |1 | Alice

Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]

2024-03-12 Thread via GitHub
flyrain commented on code in PR #9889: URL: https://github.com/apache/iceberg/pull/9889#discussion_r1521879235 ## docs/docs/spark-procedures.md: ## @@ -756,8 +756,8 @@ Creates a view that contains the changes from a given table. | `table` | ✔️ | string

Re: [PR] Kafka Connect: Record converters [iceberg]

2024-03-12 Thread via GitHub
fqaiser94 commented on code in PR #9641: URL: https://github.com/apache/iceberg/pull/9641#discussion_r1521874230 ## kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/RecordConverter.java: ## @@ -0,0 +1,508 @@ +/* + * Licensed to the Apache Software Founda

Re: [I] Proxy support unavailable for iceberg spark client [iceberg]

2024-03-12 Thread via GitHub
syun64 commented on issue #9174: URL: https://github.com/apache/iceberg/issues/9174#issuecomment-1992179279 Maybe having specific proxy configurations for the REST Catalog would be better than relying on the SystemProperties? It *looks* like the connection manager is taking the config

Re: [I] Manifest builder length [iceberg-go]

2024-03-12 Thread via GitHub
thorfour commented on issue #69: URL: https://github.com/apache/iceberg-go/issues/69#issuecomment-1992166800 Ah I realize now that I was confused by the comment on the `ManifestFile interface` ```bash // Length is the length in bytes of the manifest file. ``` Wherea

Re: [I] Manifest builder length [iceberg-go]

2024-03-12 Thread via GitHub
thorfour closed issue #69: Manifest builder length URL: https://github.com/apache/iceberg-go/issues/69 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: i

Re: [PR] feat: implement OAuth for catalog rest client [iceberg-rust]

2024-03-12 Thread via GitHub
flyrain commented on code in PR #254: URL: https://github.com/apache/iceberg-rust/pull/254#discussion_r1521837108 ## crates/catalog/rest/src/catalog.rs: ## @@ -113,6 +117,19 @@ impl RestCatalogConfig { ), ]); +if let Some(token) = self.props.get("

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on PR #9940: URL: https://github.com/apache/iceberg/pull/9940#issuecomment-1992146984 @nastra, this is a great start. I think we also want to add a section in the overall API description that covers how tags are used when there is a matching capability. That is, capability

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521831389 ## open-api/rest-catalog-open-api.yaml: ## @@ -1559,6 +1578,22 @@ components: type: string description: Properties that should be us

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521830908 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,6 +61,14 @@ security: - OAuth2: [catalog] - BearerAuth: [] +tags: Review Comment: I'd like to add to the ov

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521829334 ## open-api/rest-catalog-open-api.yaml: ## @@ -114,7 +124,8 @@ paths: }, "defaults": { "clients": "4" -

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521826641 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,6 +61,14 @@ security: - OAuth2: [catalog] - BearerAuth: [] +tags: Review Comment: I think that we should ad

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521825954 ## open-api/rest-catalog-open-api.yaml: ## @@ -61,6 +61,14 @@ security: - OAuth2: [catalog] - BearerAuth: [] +tags: + - name: views +description: Requires s

Re: [PR] OpenAPI: Express server capabilities via /config endpoint [iceberg]

2024-03-12 Thread via GitHub
rdblue commented on code in PR #9940: URL: https://github.com/apache/iceberg/pull/9940#discussion_r1521825337 ## open-api/rest-catalog-open-api.yaml: ## @@ -80,12 +88,14 @@ paths: " All REST clients should first call this route to get catalog configuration

Re: [PR] Fix pagination description based on new feedback [iceberg]

2024-03-12 Thread via GitHub
rahil-c commented on code in PR #9917: URL: https://github.com/apache/iceberg/pull/9917#discussion_r1521823394 ## open-api/rest-catalog-open-api.yaml: ## @@ -1610,13 +1610,27 @@ components: PageToken: description: -An opaque token which allows clients to ma

  1   2   >