Re: [PR] Support usage of Separate OIDC Authorization Server URI [iceberg]

danielcweeks commented on code in PR #8976: URL: https://github.com/apache/iceberg/pull/8976#discussion_r1493075652 ## core/src/test/java/org/apache/iceberg/rest/RESTCatalogAdapter.java: ## @@ -105,6 +105,8 @@ enum HTTPMethod { enum Route { TOKENS(HTTPMethod.POST, "v1/

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493074229 ## format/spec.md: ## @@ -1237,17 +1237,36 @@ Content file (data or delete) is serialized as a JSON object according to the fo | **`equality-ids`** |`JSON lis

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493072836 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -355,71 +475,90 @@ int findVersion() { * an attempt will be made to delete

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493071466 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -355,71 +475,90 @@ int findVersion() { * an attempt will be made to delete

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493071273 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jack

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493068494 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -149,26 +183,71 @@ public void commit(TableMetadata base, TableMetadata metada

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493067836 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -149,26 +183,71 @@ public void commit(TableMetadata base, TableMetadata metada

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493067372 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -129,6 +132,27 @@ public TableMetadata refresh() { @Override public voi

Re: [PR] Support usage of Separate OIDC Authorization Server URI [iceberg]

danielcweeks commented on code in PR #8976: URL: https://github.com/apache/iceberg/pull/8976#discussion_r1493066567 ## core/src/main/java/org/apache/iceberg/rest/HTTPClient.java: ## @@ -204,17 +204,28 @@ private static void throwFailure( throw new RESTException("Unhandled e

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493066170 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -59,7 +61,7 @@ */ public class HadoopTableOperations implements TableOperati

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493066170 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -59,7 +61,7 @@ */ public class HadoopTableOperations implements TableOperati

Re: [PR] Support for REPLACE TABLE operation [iceberg-python]

anupam-saini commented on code in PR #433: URL: https://github.com/apache/iceberg-python/pull/433#discussion_r1493065425 ## pyiceberg/schema.py: ## @@ -1213,34 +1213,50 @@ def build_position_accessors(schema_or_type: Union[Schema, IcebergType]) -> Dict return visit(schema_

Re: [PR] Support usage of Separate OIDC Authorization Server URI [iceberg]

danielcweeks commented on code in PR #8976: URL: https://github.com/apache/iceberg/pull/8976#discussion_r1493063230 ## core/src/main/java/org/apache/iceberg/rest/auth/OAuth2Util.java: ## @@ -361,6 +392,23 @@ public static class AuthSession { private final String credential;

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

emkornfield commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493059452 ## format/spec.md: ## @@ -1237,17 +1237,36 @@ Content file (data or delete) is serialized as a JSON object according to the fo | **`equality-ids`** |`JSON l

Re: [PR] Support for REPLACE TABLE operation [iceberg-python]

anupam-saini commented on PR #433: URL: https://github.com/apache/iceberg-python/pull/433#issuecomment-1949447704 Thank you @Fokko for taking time to explain in such great detail. Now it makes much more sense to have this part of the Catalog API. Made changes as suggested. -- This is an

Re: [PR] Support for REPLACE TABLE operation [iceberg-python]

syun64 commented on code in PR #433: URL: https://github.com/apache/iceberg-python/pull/433#discussion_r1493052840 ## pyiceberg/schema.py: ## @@ -1213,34 +1213,50 @@ def build_position_accessors(schema_or_type: Union[Schema, IcebergType]) -> Dict return visit(schema_or_typ

Re: [PR] Core: HadoopTable needs to skip file cleanup after task failure under some boundary conditions. [iceberg]

RussellSpitzer commented on code in PR #9546: URL: https://github.com/apache/iceberg/pull/9546#discussion_r1493050575 ## core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java: ## @@ -53,13 +54,15 @@ import org.slf4j.LoggerFactory; /** - * TableOperations im

Re: [PR] Support for REPLACE TABLE operation [iceberg-python]

anupam-saini commented on code in PR #433: URL: https://github.com/apache/iceberg-python/pull/433#discussion_r1493051291 ## pyiceberg/schema.py: ## @@ -1213,34 +1213,50 @@ def build_position_accessors(schema_or_type: Union[Schema, IcebergType]) -> Dict return visit(schema_

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493050495 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jack

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493034865 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jack

Re: [PR] Build: Bump pytest from 7.4.4 to 8.0.0 [iceberg-python]

dependabot[bot] closed pull request #319: Build: Bump pytest from 7.4.4 to 8.0.0 URL: https://github.com/apache/iceberg-python/pull/319 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] Build: Bump pytest from 7.4.4 to 8.0.0 [iceberg-python]

dependabot[bot] commented on PR #319: URL: https://github.com/apache/iceberg-python/pull/319#issuecomment-1949437791 Superseded by #439. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[PR] Build: Bump pytest from 7.4.4 to 8.0.1 [iceberg-python]

dependabot[bot] opened a new pull request, #439: URL: https://github.com/apache/iceberg-python/pull/439 Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.4 to 8.0.1. Release notes Sourced from https://github.com/pytest-dev/pytest/releases";>pytest's releases. 8.0

Re: [PR] Send X-Iceberg-Access-Delegation header to signal support for vended credentials/remote signing [iceberg-python]

danielcweeks commented on code in PR #436: URL: https://github.com/apache/iceberg-python/pull/436#discussion_r1493048345 ## pyiceberg/catalog/rest.py: ## @@ -237,6 +237,7 @@ def _create_session(self) -> Session: session.headers["Content-type"] = "application/json"

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493034865 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jack

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493034865 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jack

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493037575 ## format/spec.md: ## @@ -1237,17 +1237,36 @@ Content file (data or delete) is serialized as a JSON object according to the fo | **`equality-ids`** |`JSON lis

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493037575 ## format/spec.md: ## @@ -1237,17 +1237,36 @@ Content file (data or delete) is serialized as a JSON object according to the fo | **`equality-ids`** |`JSON lis

Re: [I] Getting the following error when using from spark thrift server [iceberg]

tkent commented on issue #3010: URL: https://github.com/apache/iceberg/issues/3010#issuecomment-1949418909 :wave: Recording one solution to this error. I ran into this using an Azure databricks cluster and the underlying error was what @RussellSpitzer noted. However, the runtime JAR for ice

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1493034865 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jack

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

geruh commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1493023248 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type: boo

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1493018795 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1493017978 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1493017691 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1493016912 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1493011625 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

[I] "CREATE TABLE (REPLACE TABLE) ... AS SELECT" Support [iceberg-python]

syun64 opened a new issue, #438: URL: https://github.com/apache/iceberg-python/issues/438 ### Feature Request / Improvement In Spark SQL, we have the ability to combine the table overwrite using AS SELECT statement, with create_table or replace_table, as an atomic operation. Do

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1492981360 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1492980261 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1492979601 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1492979077 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1492976199 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [PR] OpenAPI: Add AppendDataFileUpdate to TableUpdate for rest appends [iceberg]

jackye1995 commented on code in PR #9717: URL: https://github.com/apache/iceberg/pull/9717#discussion_r1492973284 ## open-api/rest-catalog-open-api.yaml: ## @@ -3324,6 +3348,211 @@ components: type: integer format: int64 +BooleanTypeValue: + type

Re: [I] Minimum Requirement of Data File Name in Apache Iceberg? [iceberg-python]

syun64 closed issue #429: Minimum Requirement of Data File Name in Apache Iceberg? URL: https://github.com/apache/iceberg-python/issues/429 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [I] Minimum Requirement of Data File Name in Apache Iceberg? [iceberg-python]

syun64 commented on issue #429: URL: https://github.com/apache/iceberg-python/issues/429#issuecomment-1949301152 I like the idea! This is definitely helpful, and I think this opens up a lot more options when looking to use other engines in parallelizing writes. -- This is an automated mes

[I] Spec is unclear w.r.t. optional fields in field_summary [iceberg]

JFinis opened a new issue, #9740: URL: https://github.com/apache/iceberg/issues/9740 ### Apache Iceberg version 1.4.3 (latest release) ### Query engine None ### Please describe the bug 🐞 I'm referring to the definition of `field_summary`, which is as follows

Re: [I] rewrite_data_files procedure fails with Premature end of Content-Length when using S3 client [iceberg]

paulpaul1076 commented on issue #9679: URL: https://github.com/apache/iceberg/issues/9679#issuecomment-1949240582 @nastra unfortunately this doesn't seem to be the only reason for the content-length exception. We now discovered that it still fails, even though I stopped using the direct str

Re: [I] InMemoryCatalog's FiloIO in memory map isn't persistent in RestCatalog [iceberg]

jackye1995 closed issue #9604: InMemoryCatalog's FiloIO in memory map isn't persistent in RestCatalog URL: https://github.com/apache/iceberg/issues/9604 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[I] Spec inconsistency: partition_spec_id column in ManifestList vs. partition_specs in metadata.json [iceberg]

JFinis opened a new issue, #9739: URL: https://github.com/apache/iceberg/issues/9739 ### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 The spec is inconsistent with respect to the `partition_spec_id` column. Here the spec n

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1492856785 ## open-api/rest-catalog-open-api.yaml: ## @@ -532,6 +532,100 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1492851000 ## open-api/rest-catalog-open-api.yaml: ## @@ -532,6 +532,100 @@ paths: 5XX: $ref: '#/components/responses/ServerErrorResponse' + /v1/{prefix}/na

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1492847929 ## open-api/rest-catalog-open-api.py: ## @@ -905,6 +973,28 @@ class CreateTableRequest(BaseModel): properties: Optional[Dict[str, str]] = None +class PlanConte

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

emkornfield commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1492846630 ## format/spec.md: ## @@ -1237,17 +1237,36 @@ Content file (data or delete) is serialized as a JSON object according to the fo | **`equality-ids`** |`JSON l

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1492846298 ## open-api/rest-catalog-open-api.py: ## @@ -905,6 +973,28 @@ class CreateTableRequest(BaseModel): properties: Optional[Dict[str, str]] = None +class PlanConte

Re: [PR] Add PrePlanTable and PlanTable Endpoints to open api spec [iceberg]

rahil-c commented on code in PR #9695: URL: https://github.com/apache/iceberg/pull/9695#discussion_r1492845781 ## open-api/rest-catalog-open-api.py: ## @@ -905,6 +973,28 @@ class CreateTableRequest(BaseModel): properties: Optional[Dict[str, str]] = None +class PlanConte

Re: [PR] Allow creating metadata tables based on SerializableTable instances [iceberg]

pvary commented on code in PR #9735: URL: https://github.com/apache/iceberg/pull/9735#discussion_r149288 ## core/src/main/java/org/apache/iceberg/MetadataTableUtils.java: ## @@ -32,9 +32,16 @@ public static boolean hasMetadataTableName(TableIdentifier identifier) { publi

Re: [PR] Allow creating metadata tables based on SerializableTable instances [iceberg]

pvary commented on code in PR #9735: URL: https://github.com/apache/iceberg/pull/9735#discussion_r1492832463 ## core/src/main/java/org/apache/iceberg/MetadataTableUtils.java: ## @@ -32,9 +32,16 @@ public static boolean hasMetadataTableName(TableIdentifier identifier) { publi

Re: [PR] Allow creating metadata tables based on SerializableTable instances [iceberg]

pvary commented on code in PR #9735: URL: https://github.com/apache/iceberg/pull/9735#discussion_r1492831999 ## core/src/test/java/org/apache/iceberg/hadoop/TestTableSerialization.java: ## @@ -153,6 +163,22 @@ public void testSerializableMetadataTablesPlanning() throws IOExcept

Re: [I] com.esotericsoftware.kryo.KryoException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.nio.ByteBuffer [iceberg]

adevore3 commented on issue #9738: URL: https://github.com/apache/iceberg/issues/9738#issuecomment-1949035999 I forgot to mention I did try to register KryoSerializer folowing this SOF: https://stackoverflow.com/questions/52562671/registering-classes-with-kryo-via-sparksession-in-spark-2 do

Re: [PR] WIP: don't reset snapshotLog when replacing table [iceberg]

rdblue commented on code in PR #9732: URL: https://github.com/apache/iceberg/pull/9732#discussion_r1492829614 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1246,7 +1246,6 @@ public Builder setRef(String name, SnapshotRef ref) { public Builder removeRef

[I] com.esotericsoftware.kryo.KryoException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.nio.ByteBuffer [iceberg]

adevore3 opened a new issue, #9738: URL: https://github.com/apache/iceberg/issues/9738 ### Apache Iceberg version None ### Query engine None ### Please describe the bug 🐞 We were upgrading to spark 3.4.1 when we ran into this issue. Currently running on spa

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

jackye1995 commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1492816128 ## core/src/main/java/org/apache/iceberg/FileScanTaskParser.java: ## @@ -21,116 +21,84 @@ import com.fasterxml.jackson.core.JsonGenerator; import com.fasterxml.jac

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1492800015 ## open-api/rest-catalog-open-api.yaml: ## @@ -1581,6 +1616,12 @@ components: type: string example: [ "accounting", "tax" ] +NextPageToken: +

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1492798196 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [PR] Allow creating metadata tables based on SerializableTable instances [iceberg]

szehon-ho commented on code in PR #9735: URL: https://github.com/apache/iceberg/pull/9735#discussion_r1492792745 ## core/src/main/java/org/apache/iceberg/SerializableTable.java: ## @@ -388,6 +388,12 @@ public Transaction newTransaction() { throw new UnsupportedOperationExce

Re: [PR] Core: Properly suppress historical snapshots when building TableMetadata with suppressHistoricalSnapshots() [iceberg]

rdblue merged PR #9234: URL: https://github.com/apache/iceberg/pull/9234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac

Re: [PR] Add pagination to open api spec for listing of namespaces, tables, views [iceberg]

rahil-c commented on code in PR #9660: URL: https://github.com/apache/iceberg/pull/9660#discussion_r1492792266 ## open-api/rest-catalog-open-api.yaml: ## @@ -1482,6 +1490,33 @@ components: explode: false example: "vended-credentials,remote-signing" +page-toke

Re: [PR] Core: Properly suppress historical snapshots when building TableMetadata with suppressHistoricalSnapshots() [iceberg]

rdblue commented on PR #9234: URL: https://github.com/apache/iceberg/pull/9234#issuecomment-1948966275 Thanks, @nastra! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] Core: Only write view history when currentVersionId changes [iceberg]

rdblue commented on code in PR #9725: URL: https://github.com/apache/iceberg/pull/9725#discussion_r1492788744 ## core/src/test/java/org/apache/iceberg/view/TestViewMetadata.java: ## @@ -63,8 +62,11 @@ public void testExpiration() { ViewVersion v2 = newViewVersion(2, "select

[I] branch schema affected by main table schema [iceberg]

namrathamyske opened a new issue, #9737: URL: https://github.com/apache/iceberg/issues/9737 ### Apache Iceberg version main (development) ### Query engine None ### Please describe the bug 🐞 regarding this PR: https://github.com/apache/iceberg/pull/9131 - th

Re: [I] Updating a property map in a iceberg table [iceberg]

namrathamyske closed issue #9659: Updating a property map in a iceberg table URL: https://github.com/apache/iceberg/issues/9659 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] WIP: don't reset snapshotLog when replacing table [iceberg]

amogh-jahagirdar commented on code in PR #9732: URL: https://github.com/apache/iceberg/pull/9732#discussion_r1492738591 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1246,7 +1246,6 @@ public Builder setRef(String name, SnapshotRef ref) { public Builder

Re: [PR] WIP: don't reset snapshotLog when replacing table [iceberg]

amogh-jahagirdar commented on code in PR #9732: URL: https://github.com/apache/iceberg/pull/9732#discussion_r1492738591 ## core/src/main/java/org/apache/iceberg/TableMetadata.java: ## @@ -1246,7 +1246,6 @@ public Builder setRef(String name, SnapshotRef ref) { public Builder

Re: [I] Null values in metadata_log_entries [iceberg]

amogh-jahagirdar commented on issue #9723: URL: https://github.com/apache/iceberg/issues/9723#issuecomment-1948850563 I did a little investigation as well, I'm not sure the original context on why the snapshot log was being cleared. But here are my thoughts: It does indeed some like w

Re: [PR] Spec, Core: add a new task-type field to task JSON serialization. add data task JSON serialization spec and imp. [iceberg]

stevenzwu commented on code in PR #9728: URL: https://github.com/apache/iceberg/pull/9728#discussion_r1492723658 ## core/src/main/java/org/apache/iceberg/BaseFileScanTaskParser.java: ## @@ -0,0 +1,122 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m

Re: [PR] Smooth out release steps [iceberg-rust]

nastra merged PR #197: URL: https://github.com/apache/iceberg-rust/pull/197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.a

Re: [PR] feat(catalog): Send X-Iceberg-Access-Delegation header to signal support for vended credentials [iceberg-go]

Fokko merged PR #67: URL: https://github.com/apache/iceberg-go/pull/67 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache

Re: [PR] Pass input file into iterators to get the file name [iceberg]

RussellSpitzer commented on code in PR #9691: URL: https://github.com/apache/iceberg/pull/9691#discussion_r1492708091 ## core/src/main/java/org/apache/iceberg/avro/AvroIterable.java: ## @@ -150,7 +153,7 @@ public D next(D reuse) { try { return reader.next(reuse);

[PR] Allow creating metadata tables based on SerializableTable instances [iceberg]

pvary opened a new pull request, #9735: URL: https://github.com/apache/iceberg/pull/9735 Currently it is not possible to create metadata tables above serialized tables. This could be useful as a general feature, and specifically it is needed if we want to create Flink jobs to run table main

Re: [PR] Pass input file into iterators to get the file name [iceberg]

RussellSpitzer commented on code in PR #9691: URL: https://github.com/apache/iceberg/pull/9691#discussion_r1492701634 ## core/src/main/java/org/apache/iceberg/avro/AvroIterable.java: ## @@ -150,7 +153,7 @@ public D next(D reuse) { try { return reader.next(reuse);

Re: [PR] Pass input file into iterators to get the file name [iceberg]

RussellSpitzer commented on code in PR #9691: URL: https://github.com/apache/iceberg/pull/9691#discussion_r1492700348 ## core/src/main/java/org/apache/iceberg/avro/AvroIterable.java: ## @@ -150,7 +153,7 @@ public D next(D reuse) { try { return reader.next(reuse);

Re: [PR] Spark 3.5: Fix testDeleteFileThenMetadataDelete failure due to table not refreshed [iceberg]

manuzhang commented on code in PR #9551: URL: https://github.com/apache/iceberg/pull/9551#discussion_r1492670868 ## spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/SparkRowLevelOperationsTestBase.java: ## @@ -166,6 +166,28 @@ public static Object[][

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

manuzhang commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1492660971 ## api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java: ## @@ -99,6 +99,20 @@ public interface RewriteDataFiles boolean USE_STARTING_SEQUENCE_NUMB

[PR] feat: complete the basic file scan [iceberg-rust]

ZENOTME opened a new pull request, #199: URL: https://github.com/apache/iceberg-rust/pull/199 This PR aims to complete #125. After #66, we prepared the plan file(metadata) for the scan. This PR will uses these plan file to construct the stream for read record batch. This PR also cha

Re: [PR] Core, Spark: Remove dangling deletes as part of RewriteDataFilesAction [iceberg]

manuzhang commented on code in PR #9724: URL: https://github.com/apache/iceberg/pull/9724#discussion_r1492660971 ## api/src/main/java/org/apache/iceberg/actions/RewriteDataFiles.java: ## @@ -99,6 +99,20 @@ public interface RewriteDataFiles boolean USE_STARTING_SEQUENCE_NUMB

Re: [PR] feat: complete the basic file scan [iceberg-rust]

ZENOTME commented on PR #199: URL: https://github.com/apache/iceberg-rust/pull/199#issuecomment-1948677398 For now, it just a draft for changed interface. I will complete them if this interface looks well. -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] feat: add parquet writer [iceberg-rust]

ZENOTME commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492447476 ## crates/iceberg/src/writer/file_writer/parquet_writer.rs: ## @@ -0,0 +1,385 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribut

Re: [PR] feat: add parquet writer [iceberg-rust]

ZENOTME commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492450927 ## crates/iceberg/src/writer/file_writer/track_writer.rs: ## @@ -0,0 +1,72 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor

Re: [PR] feat: add parquet writer [iceberg-rust]

ZENOTME commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492447476 ## crates/iceberg/src/writer/file_writer/parquet_writer.rs: ## @@ -0,0 +1,385 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribut

Re: [PR] feat: add parquet writer [iceberg-rust]

ZENOTME commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492446872 ## crates/iceberg/src/writer/file_writer/parquet_writer.rs: ## @@ -0,0 +1,385 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contribut

Re: [PR] Website: Add release schedule on the releases page [iceberg]

jbonofre commented on PR #9666: URL: https://github.com/apache/iceberg/pull/9666#issuecomment-1948376773 @bitsondatadev @Fokko thoughts about this PR ? Thanks ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

Re: [PR] feat: add parquet writer [iceberg-rust]

ZENOTME commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492445165 ## crates/iceberg/src/writer/file_writer/location_generator.rs: ## @@ -0,0 +1,99 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contri

Re: [PR] feat: add parquet writer [iceberg-rust]

Xuanwo commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492439815 ## crates/iceberg/src/io.rs: ## @@ -240,9 +241,9 @@ impl InputFile { } /// Trait for writing file. -pub trait FileWrite: AsyncWrite {} +pub trait FileWrite: Async

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

jbonofre commented on PR #9487: URL: https://github.com/apache/iceberg/pull/9487#issuecomment-1948364274 @nastra yes, actually, I plan to improve `V0` / `V1` and more support in a separate PR introducing `JdbcAdapter` allowing us to adapt to different backend (like I did here https://githu

Re: [PR] feat: add parquet writer [iceberg-rust]

ZENOTME commented on code in PR #176: URL: https://github.com/apache/iceberg-rust/pull/176#discussion_r1492434129 ## crates/iceberg/src/io.rs: ## @@ -240,9 +241,9 @@ impl InputFile { } /// Trait for writing file. -pub trait FileWrite: AsyncWrite {} +pub trait FileWrite: Asyn

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492427645 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -81,85 +114,242 @@ final class JdbcUtil { + TABLE_NAME + ")" + ")"; -

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492412706 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcUtil.java: ## @@ -25,31 +25,62 @@ import java.util.Map; import java.util.Properties; import java.util.Set; +import o

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492410874 ## core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java: ## @@ -158,6 +161,92 @@ public void testInitialize() { jdbcCatalog.initialize("test_jdbc_catalog

Re: [PR] Core: Add view support for JDBC catalog [iceberg]

nastra commented on code in PR #9487: URL: https://github.com/apache/iceberg/pull/9487#discussion_r1492409536 ## core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java: ## @@ -182,18 +177,23 @@ private void createTable(String newMetadataLocation) throws SQLExceptio

Re: [PR] Feat: Add fail_if_exists param to create_table [iceberg-python]

Fokko commented on PR #415: URL: https://github.com/apache/iceberg-python/pull/415#issuecomment-1948321931 @hussein-awala Thanks! You're right, and I think `create_table_if_not_exists` would be a cleaner solution, see #406. -- This is an automated message from the Apache Git Service. To

  1   2   >