Re: [I] PyIceberg Cookbook [iceberg-python]
shiv-io commented on issue #1201: URL: https://github.com/apache/iceberg-python/issues/1201#issuecomment-2424052259 @kevinjqliu are you accepting contributions for this cookbook yet? Happy to help if so! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Custom fileio docs [iceberg-python]
kevinjqliu commented on code in PR #1238:
URL: https://github.com/apache/iceberg-python/pull/1238#discussion_r1807441718
##
mkdocs/docs/configuration.md:
##
@@ -47,6 +55,8 @@ Iceberg tables support table properties to configure table
behavior.
| `commit.manifest.target-size-bytes` | Size in bytes | 8388608 (8MB) |
Target size when merging manifest files |
| `commit.manifest.min-count-to-merge` | Number of manifests | 100 |
Target size when merging manifest files |
| `commit.manifest-merge.enabled` | Boolean | False |
Controls whether to automatically merge manifests on writes |
+| `schema.name-mapping.default` | Name mapping strategy
| N/A | Default name mapping for schema evolution.
|
+| `format-version` | `{1, 2}`
| 2 | The version of the Iceberg table format to use.
|
Review Comment:
these changes are from the other PR.
Can you rebase this PR with main so that only changes for this PR will show?
##
mkdocs/docs/configuration.md:
##
@@ -151,6 +161,62 @@ For the FileIO there are several configuration options
available:
+### Custom FileIO Implementations
+
+
+
+The `pyIceberg` library allows you to use custom FileIO implementations,
enabling flexible file handling tailored to your specific needs. This feature
is particularly useful when working with different storage backends or file
formats.
+
+ Bringing Your Own FileIO with `PY_IO_IMPL`
Review Comment:
nit: merge with above section
##
mkdocs/docs/configuration.md:
##
@@ -151,6 +161,62 @@ For the FileIO there are several configuration options
available:
+### Custom FileIO Implementations
+
+
+
+The `pyIceberg` library allows you to use custom FileIO implementations,
enabling flexible file handling tailored to your specific needs. This feature
is particularly useful when working with different storage backends or file
formats.
Review Comment:
nit: `PyIceberg`
##
mkdocs/docs/configuration.md:
##
@@ -151,6 +161,62 @@ For the FileIO there are several configuration options
available:
+### Custom FileIO Implementations
+
+
Review Comment:
nit: this is not necessary unless there are links here that would not work
##
mkdocs/docs/configuration.md:
##
@@ -151,6 +161,62 @@ For the FileIO there are several configuration options
available:
+### Custom FileIO Implementations
+
+
+
+The `pyIceberg` library allows you to use custom FileIO implementations,
enabling flexible file handling tailored to your specific needs. This feature
is particularly useful when working with different storage backends or file
formats.
+
+ Bringing Your Own FileIO with `PY_IO_IMPL`
+
+To implement a custom FileIO, you can specify the `PY_IO_IMPL` property in
your configuration. The property should point to the custom FileIO class you
wish to use. Below is a brief guide on how to use this feature.
+
+```python
+PY_IO_IMPL = "py-io-impl"
+```
+
+ Implementation Details
+
+The following functions are key to inferring and loading custom FileIO
implementations:
+
+# `_infer_file_io_from_scheme`
+
+```python
+def _infer_file_io_from_scheme(path: str, properties: Properties) ->
Optional[FileIO]:
+```
+
+- **Purpose**: Infers the appropriate FileIO implementation based on the
scheme of the provided file path.
+- **Parameters**:
+- `path` (str): The file path from which to infer the scheme.
+- `properties` (Properties): Configuration properties to assist with
loading.
+- **Returns**: An instance of `FileIO` if a suitable implementation is found;
otherwise, `None`.
+
+# Usage Example
+
+```python
+file_io = _infer_file_io_from_scheme("s3://my-bucket/my-file.txt", properties)
+```
+
+# `load_file_io`
+
+```python
+def load_file_io(properties: Properties = EMPTY_DICT, location: Optional[str]
= None) -> FileIO:
+```
+
+- **Purpose**: Loads the custom FileIO implementation specified in the
`properties`.
+- **Parameters**:
+- `properties` (Properties): A dictionary of configuration properties,
which may include `PY_IO_IMPL`.
+- `location` (Optional[str]): An optional location to specify the file
path.
+- **Returns**: An instance of `FileIO`.
Review Comment:
these are not necessary as part of docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsub
Re: [PR] Feature: Write to branches [iceberg-python]
kevinjqliu commented on PR #941: URL: https://github.com/apache/iceberg-python/pull/941#issuecomment-2424130438 Thanks for the contribution! I'll take a look. I remember adding support for branch is complicated since we need to consider different edge cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] feat: Add support for YYYYMMDD date formats [iceberg-python]
omkenge commented on PR #1234: URL: https://github.com/apache/iceberg-python/pull/1234#issuecomment-2424103667 Hey @kevinjqliu sorry for the late reply ... So talking about use cases .. Many legacy systems and industries (e.g., government sectors) store dates in compact MMDD format. Supporting MMDD allows developers to work seamlessly with historical datasets without needing to pre-process them. As I am working in a Mechatronics company, the MMDD format is crucial for handling machine-generated data in areas such as maintenance schedules, log entries, and component tracking. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] feat: Add support for YYYYMMDD date formats [iceberg-python]
kevinjqliu commented on PR #1234: URL: https://github.com/apache/iceberg-python/pull/1234#issuecomment-2424128401 Thanks for the context! My opinion is that it's best not to add the `MMDD` format to the pyiceberg library. Here are my reasoning. - I think date parsing can be done outside of the pyiceberg library. For example, we can transform date strings from `MMDD` to the currently supported ISO-8601 format (`-MM-DD`). - Supporting yet another date string format adds more complexity. If we support `MMDD`, why not also support `MMDD`? I think its best to only support ISO-8601 (`-MM-DD`) in the pyiceberg library and have other date strings transform to this format. What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] docs/configuration.md: Documented table properties (#1231) [iceberg-python]
kevinjqliu commented on PR #1232: URL: https://github.com/apache/iceberg-python/pull/1232#issuecomment-2424118781 Hi @mths1, Thanks for the feedback. You're right, `write.target-file-size-bytes` does not represent the resulting file's size on disk. It's based on the size of the in-memory arrow buffers and since parquet can be compressed, the resulting file size can be smaller. This aligns with https://iceberg.apache.org/docs/latest/spark-writes/#controlling-file-sizes Perhaps we can mention this behavior in the table. For example, this is what the [java docs](https://iceberg.apache.org/docs/latest/configuration/#write-properties) mention ``` write.target-file-size-bytes | 536870912 (512 MB) | Controls the size of files generated to target about this many bytes ``` Maybe something like ``` Controls the target size of in-memory buffers for writing files. The actual file size may be smaller due to compression. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] feat: Add support for YYYYMMDD date formats [iceberg-python]
omkenge commented on PR #1234: URL: https://github.com/apache/iceberg-python/pull/1234#issuecomment-2424140033 Thank you for your feedback! I understand your concerns. Ok then lets keep it simple and we can closed this PR with your final comment ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] feat: Add support for YYYYMMDD date formats [iceberg-python]
omkenge commented on PR #1234: URL: https://github.com/apache/iceberg-python/pull/1234#issuecomment-2424141706 and also i have some suggestions on other parts also , i will create PR as soon as possible Plz also share your thoughts on that part also Thank you for your time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spec: Fix table of content generation [iceberg]
rdblue commented on code in PR #11067: URL: https://github.com/apache/iceberg/pull/11067#discussion_r1807491293 ## format/spec.md: ## @@ -158,27 +158,27 @@ Readers should be more permissive because v1 metadata files are allowed in v2 ta Readers may be more strict for metadata JSON files because the JSON files are not reused and will always match the table version. Required fields that were not present in or were optional in prior versions may be handled as required fields. For example, a v2 table that is missing `last-sequence-number` can throw an exception. -### Writing data files + Writing data files All columns must be written to data files even if they introduce redundancy with metadata stored in manifest files (e.g. columns with identity partition transforms). Writing all columns provides a backup in case of corruption or bugs in the metadata layer. Writers are not allowed to commit files with a partition spec that contains a field with an unknown transform. -## Schemas and Data Types +### Schemas and Data Types Review Comment: Okay, that sounds fine to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Docs: Fix incorrect wget command in Flink documentation [iceberg]
github-actions[bot] commented on PR #9483: URL: https://github.com/apache/iceberg/pull/9483#issuecomment-2424318735 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] API, Core: Add scan planning apis to REST Catalog [iceberg]
amogh-jahagirdar commented on code in PR #11180:
URL: https://github.com/apache/iceberg/pull/11180#discussion_r1807542082
##
core/src/test/java/org/apache/iceberg/TestBase.java:
##
@@ -63,7 +63,7 @@ public class TestBase {
public static final PartitionSpec SPEC =
PartitionSpec.builderFor(SCHEMA).bucket("data", BUCKETS_NUMBER).build();
- static final DataFile FILE_A =
+ public static final DataFile FILE_A =
Review Comment:
Why did these need to change?
##
core/src/test/java/org/apache/iceberg/TestBase.java:
##
@@ -63,7 +63,7 @@ public class TestBase {
public static final PartitionSpec SPEC =
PartitionSpec.builderFor(SCHEMA).bucket("data", BUCKETS_NUMBER).build();
- static final DataFile FILE_A =
+ public static final DataFile FILE_A =
Review Comment:
I see, this is used in the tests below. Could we just have the tests which
use this to extend `TestBase?
##
core/src/main/java/org/apache/iceberg/rest/responses/PlanTableScanResponse.java:
##
@@ -0,0 +1,122 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg.rest.responses;
+
+import java.util.List;
+import org.apache.iceberg.DeleteFile;
+import org.apache.iceberg.FileScanTask;
+import org.apache.iceberg.relocated.com.google.common.base.MoreObjects;
+import org.apache.iceberg.rest.PlanStatus;
+import org.apache.iceberg.rest.RESTResponse;
+
+public class PlanTableScanResponse implements RESTResponse {
+ private PlanStatus planStatus;
+ private String planId;
+ private List planTasks;
+ private List fileScanTasks;
+ private List deleteFiles;
+
+ private PlanTableScanResponse(
+ PlanStatus planStatus,
+ String planId,
+ List planTasks,
+ List fileScanTasks,
+ List deleteFiles) {
+this.planStatus = planStatus;
+this.planId = planId;
+this.planTasks = planTasks;
+this.fileScanTasks = fileScanTasks;
+this.deleteFiles = deleteFiles;
+ }
+
+ public PlanStatus planStatus() {
+return planStatus;
+ }
+
+ public String planId() {
+return planId;
+ }
+
+ public List planTasks() {
+return planTasks;
+ }
+
+ public List fileScanTasks() {
+return fileScanTasks;
+ }
+
+ public List deleteFiles() {
+return deleteFiles;
+ }
+
+ @Override
+ public String toString() {
+return MoreObjects.toStringHelper(this)
+.add("planStatus", planStatus)
+.add("planId", planId)
+.add("planTasks", planTasks)
+.add("fileScanTasks", fileScanTasks)
+.add("deleteFiles", deleteFiles)
Review Comment:
Let's double check that some of these `toString` make sense, I think we
shouldn't really be outputting in file scan tasks/delete files, it'll just be
references that a user won't really make sense of in logs. I think we should
just remove everything except status and ID here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] feat: Add support for YYYYMMDD date formats [iceberg-python]
omkenge closed pull request #1234: feat: Add support for MMDD date formats URL: https://github.com/apache/iceberg-python/pull/1234 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] feat: Add support for YYYYMMDD date formats [iceberg-python]
kevinjqliu commented on PR #1234: URL: https://github.com/apache/iceberg-python/pull/1234#issuecomment-2424162564 @omkenge do you mind creating an issue first? We can have a discussion and allow others to chime in before starting the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark 3.5: Fix testDeleteFileThenMetadataDelete failure due to table not refreshed [iceberg]
github-actions[bot] closed pull request #9551: Spark 3.5: Fix testDeleteFileThenMetadataDelete failure due to table not refreshed URL: https://github.com/apache/iceberg/pull/9551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Hive: Refactor hive-table commit operation to be used for other operations like view [iceberg]
github-actions[bot] commented on PR #9461: URL: https://github.com/apache/iceberg/pull/9461#issuecomment-2424318564 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [I] Concurrent writes failures [iceberg-python]
reinthal commented on issue #1084: URL: https://github.com/apache/iceberg-python/issues/1084#issuecomment-2423927862 Here's some code that worked for me for me ```python def append_to_table_with_retry(pa_df: pa.Table, table_name: str, catalog: Catalog) -> None: """Appends a pyarrow dataframe to the table in the catalog using tenacity exponential backoff.""" @retry( wait=wait_exponential(multiplier=1, min=4, max=32), stop=stop_after_attempt(20), reraise=True ) def append_with_retry(): table = catalog.load_table(table_name) # < If a process appends between this line ... table.append(pa_df) # <- and this line, then Tenacity will retry. append_with_retry() ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.6 [iceberg]
dependabot[bot] opened a new pull request, #11357: URL: https://github.com/apache/iceberg/pull/11357 Bumps [mkdocs-macros-plugin](https://github.com/fralau/mkdocs_macros_plugin) from 1.2.0 to 1.3.6. Changelog Sourced from https://github.com/fralau/mkdocs-macros-plugin/blob/master/CHANGELOG.md";>mkdocs-macros-plugin's changelog. 1.3.6, 2024-10-17 Added: complete test framework, using pytest and Mkdocs-Test (https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/244";>#244) A number of automated test cases are implemented. Changed: move from setup.py to pyproject.toml (https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/250";>#250) Commits https://github.com/fralau/mkdocs-macros-plugin/commit/3a73707abd48c9df214646d57204cff854bbe98b";>3a73707 Migrate to pyproject.toml https://github.com/fralau/mkdocs-macros-plugin/commit/99733bed5e62b9cc43cad1def816aefdf28651a1";>99733be Demote info messages as debug (https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/248";>#248) https://github.com/fralau/mkdocs-macros-plugin/commit/6a9a0e4f41e02a79f0e8f682f5329ed89f5dbb27";>6a9a0e4 Bump version number, for distribution (should solve https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/247";>#247) https://github.com/fralau/mkdocs-macros-plugin/commit/32cd528114ff8b4203c5bfe216f32ca613266bc9";>32cd528 Test the hooks for external registration (https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/237";>#237) https://github.com/fralau/mkdocs-macros-plugin/commit/16be58d9c64bf29ad489762fb30f6467eb192450";>16be58d Migrate all tests toward the mkdocs-test framework (https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/244";>#244) https://github.com/fralau/mkdocs-macros-plugin/commit/59f82db2a60eae28832b5a1d44fb475f8c9e309f";>59f82db Replacing local implementation of SuperDict by super-collections version https://github.com/fralau/mkdocs-macros-plugin/commit/38b9681833fd06ae049c10fded1d33514b6c8d8e";>38b9681 Merge branch 'master' of github.com:fralau/mkdocs_macros_plugin into master https://github.com/fralau/mkdocs-macros-plugin/commit/eebbbd1b6d9d587dbb36cb08d5aee35c5a8a34a7";>eebbbd1 Merge pull request https://redirect.github.com/fralau/mkdocs_macros_plugin/issues/246";>#246 from dwreeves/add-pathspec-as-dependency https://github.com/fralau/mkdocs-macros-plugin/commit/6d9982213eaca2fd44b9ddc101ed417371d829ed";>6d99822 Add pathspec as dependency https://github.com/fralau/mkdocs-macros-plugin/commit/69577e12bc5d4bc4f59f276d599f39263fa4ea82";>69577e1 Separate standard DocProject and MarkdownPage classes Additional commits viewable in https://github.com/fralau/mkdocs_macros_plugin/compare/v1.2.0...v1.3.6";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:
[PR] Build: Bump datamodel-code-generator from 0.26.1 to 0.26.2 [iceberg]
dependabot[bot] opened a new pull request, #11356: URL: https://github.com/apache/iceberg/pull/11356 Bumps [datamodel-code-generator](https://github.com/koxudaxi/datamodel-code-generator) from 0.26.1 to 0.26.2. Release notes Sourced from https://github.com/koxudaxi/datamodel-code-generator/releases";>datamodel-code-generator's releases. 0.26.2 What's Changed New CLI option --output-datetime-class https://redirect.github.com/koxudaxi/datamodel-code-generator/issues/1996";>#1996 by https://github.com/archetipo";>@archetipo in https://redirect.github.com/koxudaxi/datamodel-code-generator/pull/2100";>koxudaxi/datamodel-code-generator#2100 dataclass generator improvements by https://github.com/anis-campos";>@anis-campos in https://redirect.github.com/koxudaxi/datamodel-code-generator/pull/2102";>koxudaxi/datamodel-code-generator#2102 New Contributors https://github.com/archetipo";>@archetipo made their first contribution in https://redirect.github.com/koxudaxi/datamodel-code-generator/pull/2100";>koxudaxi/datamodel-code-generator#2100 https://github.com/anis-campos";>@anis-campos made their first contribution in https://redirect.github.com/koxudaxi/datamodel-code-generator/pull/2102";>koxudaxi/datamodel-code-generator#2102 Full Changelog: https://github.com/koxudaxi/datamodel-code-generator/compare/0.26.1...0.26.2";>https://github.com/koxudaxi/datamodel-code-generator/compare/0.26.1...0.26.2 Commits https://github.com/koxudaxi/datamodel-code-generator/commit/d0c0f16ca3bba782a9b664d56c649560c7109c0b";>d0c0f16 dataclass generator improvements (https://redirect.github.com/koxudaxi/datamodel-code-generator/issues/2102";>#2102) https://github.com/koxudaxi/datamodel-code-generator/commit/2df133c00949b6efce6f1493c6ce0e341872b955";>2df133c New CLI option --output-datetime-class https://redirect.github.com/koxudaxi/datamodel-code-generator/issues/1996";>#1996 (https://redirect.github.com/koxudaxi/datamodel-code-generator/issues/2100";>#2100) See full diff in https://github.com/koxudaxi/datamodel-code-generator/compare/0.26.1...0.26.2";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Build: Bump mkdocs-material from 9.5.39 to 9.5.40 [iceberg]
dependabot[bot] closed pull request #11309: Build: Bump mkdocs-material from 9.5.39 to 9.5.40 URL: https://github.com/apache/iceberg/pull/11309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] Build: Bump com.google.cloud:libraries-bom from 26.48.0 to 26.49.0 [iceberg]
dependabot[bot] opened a new pull request, #11363: URL: https://github.com/apache/iceberg/pull/11363 Bumps [com.google.cloud:libraries-bom](https://github.com/googleapis/java-cloud-bom) from 26.48.0 to 26.49.0. Release notes Sourced from https://github.com/googleapis/java-cloud-bom/releases";>com.google.cloud:libraries-bom's releases. v26.49.0 GCP Libraries BOM 26.49.0 Note about gRPC-Java v1.67.1 The Cloud Java SDK now pulls in v1.67.1 of gRPC-Java as part of the version of libraries-bom. This libraries-bom release changes gRPC-Java from https://github.com/grpc/grpc-java/releases/tag/v1.68.0";>v1.68.0 to https://github.com/grpc/grpc-java/releases/tag/v1.67.1";>v1.67.1. This is change is not a real downgrade in version, because the previous 1.67.0 was mistakenly released as 1.68.0. More context in https://github.com/googleapis/java-cloud-bom/releases/tag/v26.48.0";>v26.48.0 notes. Notes Here are the differences from the previous version (26.48.0) New Addition com.google.cloud:google-cloud-oracledatabase:0.1.0 The group ID of the following artifacts is com.google.cloud. Notable Changes google-cloud-bigtable 2.45.1 (prev: 2.44.1) Add support for Cloud Bigtable Node Scaling Factor for CBT Clusters (https://github.com/googleapis/java-bigtable/commit/caf879cb4086d74bd4571662510014b27e6113a7";>caf879c) Support override monitoring endpoint (https://redirect.github.com/googleapis/java-bigtable/issues/2364";>#2364) (https://github.com/googleapis/java-bigtable/commit/a341eb8530d959edabac0282c52c3e928abf733d";>a341eb8) google-cloud-datastore 2.23.0 (prev: 2.22.0) Support for field update operators in the Datastore API and resolution strategies when there is a conflict at write time (https://github.com/googleapis/java-datastore/commit/b299266e42037b731ee7bbba21dbded73a37323c";>b299266) google-cloud-firestore 3.27.2 (prev: 3.26.5) Add support for vector search with Query#findNearest (https://redirect.github.com/googleapis/java-firestore/issues/1827";>#1827) (https://github.com/googleapis/java-firestore/commit/4ad463a6932b90ca0cb3dd015b2543ffbf850eb9";>4ad463a) Update to Java 11, since runtime doesn't support 8. (https://redirect.github.com/googleapis/java-firestore/issues/1867";>#1867) (https://github.com/googleapis/java-firestore/commit/723c7ccc783b2f56ca72867cd741df197a9f68d7";>723c7cc) google-cloud-spanner 6.79.0 (prev: 6.77.0) Define ReplicaComputeCapacity and AsymmetricAutoscalingOption (https://github.com/googleapis/java-spanner/commit/f46a6b34383fe45d63b2db912389b26067f3a853";>f46a6b3) Support DML auto-batching in Connection API (https://redirect.github.com/googleapis/java-spanner/issues/3386";>#3386) (https://github.com/googleapis/java-spanner/commit/a1ce267cbd4d4c5c638ab7fe0dd5dba24bcfab86";>a1ce267) google-cloud-spanner-jdbc 2.23.0 (prev: 2.22.1) Auto_batch_dml connection property (https://redirect.github.com/googleapis/java-spanner-jdbc/issues/1787";>#1787) (https://github.com/googleapis/java-spanner-jdbc/commit/8aa0edbca8545348066cdcee511f2fc692987075";>8aa0edb) Fixed transaction tagging and reset datafaker version (https://redirect.github.com/googleapis/java-spanner-jdbc/issues/1777";>#1777) (https://github.com/googleapis/java-spanner-jdbc/commit/3b117c0723fb44d82170b5ab766afd6ff2c0c347";>3b117c0) Other libraries [admanager] Added support for Interactive Reporting (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add a dynamic retrieval API (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add a dynamic retrieval API (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add continuous sync option in feature_view.proto (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add enable_secure_private_service_connect in service attachment (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add new PscInterfaceConfig field to pipeline_job.proto (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add partner_model_tuning_spec to TuningJob (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [aiplatform] add psc_automation_configs to DeployIndex v1 (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa9343d7db83d7a1a736ffd7a2b7e6";>30455f7) [bigquerydatatransfer]Add scheduleOptionsV2 and Error fields for TransferConfig (https://github.com/googleapis/google-cloud-java/commit/30455f78e0aa934
Re: [PR] Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.5 [iceberg]
dependabot[bot] commented on PR #11310: URL: https://github.com/apache/iceberg/pull/11310#issuecomment-2424571107 Superseded by #11357. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.5 [iceberg]
dependabot[bot] closed pull request #11310: Build: Bump mkdocs-macros-plugin from 1.2.0 to 1.3.5 URL: https://github.com/apache/iceberg/pull/11310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] Build: Bump mkdocs-material from 9.5.39 to 9.5.41 [iceberg]
dependabot[bot] opened a new pull request, #11358: URL: https://github.com/apache/iceberg/pull/11358 Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from 9.5.39 to 9.5.41. Release notes Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mkdocs-material's releases. mkdocs-material-9.5.41 Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7619";>#7619: Improved tooltip on logo disappears after instant navigation Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7616";>#7616: Race condition in built-in privacy plugin when inlining assets Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7615";>#7615: Comments and "Was this page helpful?" visible when printing mkdocs-material-9.5.40 Updated Latvian translations Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7597";>#7597: Social cards not using site name on home page Thanks go to https://github.com/pekkaklarck";>@pekkaklarck and https://github.com/klejejs";>@klejejs for their contributions Changelog Sourced from https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG";>mkdocs-material's changelog. mkdocs-material-9.5.41 (2024-10-15) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7619";>#7619: Improved tooltip on logo disappears after instant navigation Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7616";>#7616: Race condition in built-in privacy plugin when inlining assets Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7615";>#7615: Comments and "Was this page helpful?" visible when printing mkdocs-material-9.5.40 (2024-10-10) Updated Latvian translations Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7597";>#7597: Social cards not using site name on home page mkdocs-material-9.5.39+insiders-4.53.14 (2024-09-29) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7567";>#7567: Empty headlines when using typeset plugin with anchorlinks mkdocs-material-9.5.39 (2024-09-29) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7226";>#7226: not staying on page when using mike's canonical versioning mkdocs-material-9.5.38 (2024-09-26) Added Albanian translations mkdocs-material-9.5.37 (2024-09-25) Added 4th and 5th level ordered list styles Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7548";>#7548: Tags have no spacing in search mkdocs-material-9.5.36 (2024-09-21) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7544";>#7544: Social cards incorrectly rendering HTML entities Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7542";>#7542: Improved support for setting custom list styles mkdocs-material-9.5.35 (2024-09-18) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7498";>#7498: Search not showing for Vietnamese language mkdocs-material-9.5.34+insiders-4.53.13 (2024-09-14) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7520";>#7520: Social plugin errors for generated files (MkDocs 1.6+) mkdocs-material-9.5.34 (2024-08-31) Updated Mermaid.js to version 11 (latest) mkdocs-material-9.5.33 (2024-08-23) Fixed https://redirect.github.com/squidfunk/mkdocs-material/issues/7453";>#7453: Incorrect position of tooltip when sorting table mkdocs-material-9.5.32 (2024-08-19) ... (truncated) Commits https://github.com/squidfunk/mkdocs-material/commit/5eef8158580d1adb6afee837c714a2ffa6de4fd0";>5eef815 Prepare 9.5.41 release https://github.com/squidfunk/mkdocs-material/commit/2d2e0415d612f7d9d4245bec0288137ee296a6d3";>2d2e041 Updated dependencies https://github.com/squidfunk/mkdocs-material/commit/790b0e80caf0b3af09df37e5129b89381b3f4de1";>790b0e8 JSON schema (https://redirect.github.com/squidfunk/mkdocs-material/issues/7622";>#7622) https://github.com/squidfunk/mkdocs-material/commit/649abd2ff8f3785bfc599c63dc610b8e3a392216";>649abd2 Fixed tooltip on logo lost on instant navigation https://github.com/squidfunk/mkdocs-material/commit/0418a22a5ad557377a7793956f972e4d2a5fc538";>0418a22 Updated README.md (https://redirect.github.com/squidfunk/mkdocs-material/issues/7618";>#7618) https://github.com/squidfunk/mkdocs-material/commit/e29e124e426e4ccd16d8937c13e9bdb7ef26d480";>e29e124 Fixed feedback visible on print https://github.com/squidfunk/mkdocs-material/commit/92089c67f1eeccb390302dbd59e849ecd82fd8b1";>92089c6 Fixed race condition in privacy plugin https://github.com/squidfunk/mkdocs-material/commit/39d14d5c190ca2f55732e10edf298bc07921f33d";>39d14d5 Fixed comment section appearing in print view https://github.com/squidfunk/mkdocs-ma
Re: [PR] Build: Bump mkdocs-material from 9.5.39 to 9.5.40 [iceberg]
dependabot[bot] commented on PR #11309: URL: https://github.com/apache/iceberg/pull/11309#issuecomment-2424571132 Superseded by #11358. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] Build: Bump com.palantir.baseline:gradle-baseline-java from 5.69.0 to 5.72.0 [iceberg]
dependabot[bot] opened a new pull request, #11362: URL: https://github.com/apache/iceberg/pull/11362 Bumps [com.palantir.baseline:gradle-baseline-java](https://github.com/palantir/gradle-baseline) from 5.69.0 to 5.72.0. Release notes Sourced from https://github.com/palantir/gradle-baseline/releases";>com.palantir.baseline:gradle-baseline-java's releases. 5.72.0 No documented user-facing changes 5.71.0 💡 Improvements Fix slow compilation for large pattern matching switch (https://redirect.github.com/palantir/gradle-baseline/pull/2851";>#2851) 5.70.0 🐛 Fixes migrate to newer plugin publish format (https://redirect.github.com/palantir/gradle-baseline/pull/2903";>#2903) 💡 Improvements Disable the InjectOnConstructorOfAbstractClass errorprone for gradle plugin projects (https://redirect.github.com/palantir/gradle-baseline/pull/2913";>#2913) Commits https://github.com/palantir/gradle-baseline/commit/5228d6603eb08b1a9b7a9f845aa8bfbfc8b5ce27";>5228d66 Release 5.72.0 https://github.com/palantir/gradle-baseline/commit/104f0a54f6cadb39e6c2423e90645ae475caafaa";>104f0a5 Excavator: Upgrade dependencies (https://redirect.github.com/palantir/gradle-baseline/issues/2923";>#2923) https://github.com/palantir/gradle-baseline/commit/f1237d21e34d639576ed9ec7daea5d123bfec131";>f1237d2 Excavator: Upgrades Baseline to the latest version (https://redirect.github.com/palantir/gradle-baseline/issues/2920";>#2920) https://github.com/palantir/gradle-baseline/commit/c05f82b489fa9da1dc35782b6cc2423d0770dd81";>c05f82b Release 5.71.0 https://github.com/palantir/gradle-baseline/commit/5a6d137fc6c0337b8ef8741fc661b85c7852d3d3";>5a6d137 Safety propagation avoids recursive analysis for pattern match variables (https://redirect.github.com/palantir/gradle-baseline/issues/2851";>#2851) https://github.com/palantir/gradle-baseline/commit/8562ed432b31511b29b8799b46e3f12e70543e87";>8562ed4 Excavator: Upgrade Jackson to the latest stable release (https://redirect.github.com/palantir/gradle-baseline/issues/2919";>#2919) https://github.com/palantir/gradle-baseline/commit/8c5e06a8f39ba81264b448ac351f543a7152d42e";>8c5e06a [High Priority] Excavator: Update gradle-jdks infrastructure plugins (https://redirect.github.com/palantir/gradle-baseline/issues/2918";>#2918) https://github.com/palantir/gradle-baseline/commit/a0b9144af1efeb36a77d0a23bec695fc42f2ecf6";>a0b9144 Excavator: Upgrade dependencies (https://redirect.github.com/palantir/gradle-baseline/issues/2917";>#2917) https://github.com/palantir/gradle-baseline/commit/cd48475bf14a3680dfd115ca8acd9afa483b31a9";>cd48475 Excavator: Upgrade dependencies (https://redirect.github.com/palantir/gradle-baseline/issues/2916";>#2916) https://github.com/palantir/gradle-baseline/commit/dce7d65104db755de16ee0e1f8b66de76ad478b9";>dce7d65 Excavator: Upgrades Baseline to the latest version (https://redirect.github.com/palantir/gradle-baseline/issues/2915";>#2915) Additional commits viewable in https://github.com/palantir/gradle-baseline/compare/5.69.0...5.72.0";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it your
[PR] Build: Bump software.amazon.awssdk:bom from 2.28.21 to 2.28.26 [iceberg]
dependabot[bot] opened a new pull request, #11359: URL: https://github.com/apache/iceberg/pull/11359 Bumps software.amazon.awssdk:bom from 2.28.21 to 2.28.26. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] Build: Bump com.google.errorprone:error_prone_annotations from 2.33.0 to 2.34.0 [iceberg]
dependabot[bot] opened a new pull request, #11360: URL: https://github.com/apache/iceberg/pull/11360 Bumps [com.google.errorprone:error_prone_annotations](https://github.com/google/error-prone) from 2.33.0 to 2.34.0. Release notes Sourced from https://github.com/google/error-prone/releases";>com.google.errorprone:error_prone_annotations's releases. Error Prone 2.34.0 Changes: Passing the javac flag --should-stop=ifError=FLOW is now required when running Error Prone (https://redirect.github.com/google/error-prone/issues/4595";>#4595) The MemberName check was renamed to https://errorprone.info/bugpattern/IdentifierName";>IdentifierName New checks: https://errorprone.info/bugpattern/FutureTransformAsync";>FutureTransformAsync: The usage of transformAsync is not necessary when all the return values of the transformation function are immediate futures. https://errorprone.info/bugpattern/MisformattedTestData";>MisformattedTestData: Reformats Java source code inside Error Prone test cases. https://errorprone.info/bugpattern/MisleadingEscapedSpace";>MisleadingEscapedSpace: Using \s anywhere except at the end of a line in a text block is potentially misleading. https://errorprone.info/bugpattern/ThrowIfUncheckedKnownUnchecked";>ThrowIfUncheckedKnownUnchecked: throwIfUnchecked(knownUnchecked) is equivalent to throw knownUnchecked. Closed issues: https://redirect.github.com/google/error-prone/issues/4595";>#4595, https://redirect.github.com/google/error-prone/issues/4598";>#4598, https://redirect.github.com/google/error-prone/issues/4620";>#4620 Full changelog: https://github.com/google/error-prone/compare/v2.33.0...v2.34.0";>https://github.com/google/error-prone/compare/v2.33.0...v2.34.0 Commits https://github.com/google/error-prone/commit/bb113af7cae2d2c4478aba33d70e96291e8a07ef";>bb113af Release Error Prone 2.34.0 https://github.com/google/error-prone/commit/82a216870b043e841cad14eddb4568b8edcb9eff";>82a2168 Recognize that Runtime.halt and exit never return. https://github.com/google/error-prone/commit/1d040949c37c1d5b5105f6b8fe2540101897b43c";>1d04094 A couple of fixes in MoreAnnotations https://github.com/google/error-prone/commit/6203a0eea4abaec9356fef6a2723a42057775055";>6203a0e Remove references to -XDshouldStopPolicyIfError now that `--should-stop=ifE... https://github.com/google/error-prone/commit/40bb976809f79046bf46e065ca2d2e01f6efbfe9";>40bb976 Tweak ThrowIfUncheckedKnownChecked implementation to match `ThrowIfUnchecke... https://github.com/google/error-prone/commit/6380cc27fbc25c7de55be3e5a891e5025ff8cca8";>6380cc2 Warn about throwIfUnchecked(unchecked), which could be just throw unchecked. https://github.com/google/error-prone/commit/7a736905260fcb8cc2097f62a565bd7c879c6ed4";>7a73690 Fix or suppress CheckReturnValue errors https://github.com/google/error-prone/commit/ca50d5c282bab7efb228434c75515135a0f97156";>ca50d5c Update --should-stop=ifError=FLOW flags https://github.com/google/error-prone/commit/c897d8f4cd5ab310af4dca63bf1acf2d10f6f49f";>c897d8f Open-source more of check_api/src/test/java/com/google/errorprone/util https://github.com/google/error-prone/commit/4f630fc668d107b7801281ce527e1a83a782dc1b";>4f630fc Rename MemberName to SymbolName, given it's handling classes too now. Additional commits viewable in https://github.com/google/error-prone/compare/v2.33.0...v2.34.0";>compare view [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@depen
[PR] Build: Bump calcite from 1.10.0 to 1.38.0 [iceberg]
dependabot[bot] opened a new pull request, #11361: URL: https://github.com/apache/iceberg/pull/11361 Bumps `calcite` from 1.10.0 to 1.38.0. Updates `org.apache.calcite:calcite-core` from 1.10.0 to 1.38.0 Commits https://github.com/apache/calcite/commit/e5e7faeff5985bc1b2342144b2bd31ca8ea84d3a";>e5e7fae [CALCITE-6582] Release Calcite 1.38.0 https://github.com/apache/calcite/commit/a7c3d166b3d6ef614f23320ecd00d855ced1f302";>a7c3d16 [CALCITE-6620] VALUES created by RelBuilder do not have a homogeneous type https://github.com/apache/calcite/commit/243c3adc0a2ace81fc6ce4c3724f034e0f16fb7d";>243c3ad [CALCITE-6617] TypeCoercion is not applied correctly to comparisons https://github.com/apache/calcite/commit/78e873d39c0364f9f36055b9cbe0600dfad49c71";>78e873d Revise fix for [CALCITE-6587] Support Java 23 and Guava 33.3.0 https://github.com/apache/calcite/commit/d59871d49cba28096b698efda80957ecc9e8c4ab";>d59871d [CALCITE-6598] In RelDataTypeSystem, deprecate methods getMaxNumericScale and... https://github.com/apache/calcite/commit/2ce9ba42da4bec48f4252eb178a5377b941fe41f";>2ce9ba4 [CALCITE-6603] Lattice SQL supports generation of specified dialects https://github.com/apache/calcite/commit/cb698c849c9c21ff9b2f8baf1ba3770711340232";>cb698c8 In linter that checks terminology, add terms 'Snowflake' and 'Redshift' https://github.com/apache/calcite/commit/547ab35460f370f2ae76216aa374323e9dc7e07e";>547ab35 [CALCITE-3592] Implement BITNOT scalar function https://github.com/apache/calcite/commit/5091b08936fef06fcf796cf97ffb9f9e7a87756a";>5091b08 [CALCITE-6609] Remove redundant warning suppression for Guava's Beta and Unst... https://github.com/apache/calcite/commit/1af49e4404bb75c9105f1157a1c3fbb7e68770fe";>1af49e4 [CALCITE-6599] RelMdPredicates should pull up more predicates from VALUES whe... Additional commits viewable in https://github.com/apache/calcite/compare/calcite-1.10.0...calcite-1.38.0";>compare view Updates `org.apache.calcite:calcite-druid` from 1.10.0 to 1.38.0 Commits https://github.com/apache/calcite/commit/e5e7faeff5985bc1b2342144b2bd31ca8ea84d3a";>e5e7fae [CALCITE-6582] Release Calcite 1.38.0 https://github.com/apache/calcite/commit/a7c3d166b3d6ef614f23320ecd00d855ced1f302";>a7c3d16 [CALCITE-6620] VALUES created by RelBuilder do not have a homogeneous type https://github.com/apache/calcite/commit/243c3adc0a2ace81fc6ce4c3724f034e0f16fb7d";>243c3ad [CALCITE-6617] TypeCoercion is not applied correctly to comparisons https://github.com/apache/calcite/commit/78e873d39c0364f9f36055b9cbe0600dfad49c71";>78e873d Revise fix for [CALCITE-6587] Support Java 23 and Guava 33.3.0 https://github.com/apache/calcite/commit/d59871d49cba28096b698efda80957ecc9e8c4ab";>d59871d [CALCITE-6598] In RelDataTypeSystem, deprecate methods getMaxNumericScale and... https://github.com/apache/calcite/commit/2ce9ba42da4bec48f4252eb178a5377b941fe41f";>2ce9ba4 [CALCITE-6603] Lattice SQL supports generation of specified dialects https://github.com/apache/calcite/commit/cb698c849c9c21ff9b2f8baf1ba3770711340232";>cb698c8 In linter that checks terminology, add terms 'Snowflake' and 'Redshift' https://github.com/apache/calcite/commit/547ab35460f370f2ae76216aa374323e9dc7e07e";>547ab35 [CALCITE-3592] Implement BITNOT scalar function https://github.com/apache/calcite/commit/5091b08936fef06fcf796cf97ffb9f9e7a87756a";>5091b08 [CALCITE-6609] Remove redundant warning suppression for Guava's Beta and Unst... https://github.com/apache/calcite/commit/1af49e4404bb75c9105f1157a1c3fbb7e68770fe";>1af49e4 [CALCITE-6599] RelMdPredicates should pull up more predicates from VALUES whe... Additional commits viewable in https://github.com/apache/calcite/compare/calcite-1.10.0...calcite-1.38.0";>compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore
Re: [I] Implement incremental update using commit stats (SnapshotSummary) [iceberg]
github-actions[bot] closed issue #8461: Implement incremental update using commit stats (SnapshotSummary) URL: https://github.com/apache/iceberg/issues/8461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [I] Implement incremental update using commit stats (SnapshotSummary) [iceberg]
github-actions[bot] commented on issue #8461: URL: https://github.com/apache/iceberg/issues/8461#issuecomment-2424317210 This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale' -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark 3.5: Support specifying filter in RewriteManifestsProcedure [iceberg]
github-actions[bot] commented on PR #9447: URL: https://github.com/apache/iceberg/pull/9447#issuecomment-2424318521 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] AWS: Add Option to don't write non current columns in glue schema closes #7584 [iceberg]
github-actions[bot] commented on PR #9420: URL: https://github.com/apache/iceberg/pull/9420#issuecomment-2424318504 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Build: Extract Gradle version from `gradle-wrapper.properties` [iceberg]
github-actions[bot] commented on PR #9448: URL: https://github.com/apache/iceberg/pull/9448#issuecomment-2424318529 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]
github-actions[bot] closed pull request #9414: API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs URL: https://github.com/apache/iceberg/pull/9414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark 3.5: Support specifying filter in RewriteManifestsProcedure [iceberg]
github-actions[bot] closed pull request #9447: Spark 3.5: Support specifying filter in RewriteManifestsProcedure URL: https://github.com/apache/iceberg/pull/9447 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Pushed filters to Parquet file on best effort basis in Vectorized Reader [iceberg]
github-actions[bot] commented on PR #9479: URL: https://github.com/apache/iceberg/pull/9479#issuecomment-2424318606 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark: Fix reading 2 level array issue [iceberg]
github-actions[bot] commented on PR #9515: URL: https://github.com/apache/iceberg/pull/9515#issuecomment-2424319453 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] API, Core: Add Schema#withUpdatedDoc and View#updateColumnDoc APIs [iceberg]
github-actions[bot] commented on PR #9414: URL: https://github.com/apache/iceberg/pull/9414#issuecomment-2424318493 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Hive: Refactor hive-table commit operation to be used for other operations like view [iceberg]
github-actions[bot] closed pull request #9461: Hive: Refactor hive-table commit operation to be used for other operations like view URL: https://github.com/apache/iceberg/pull/9461 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Feature: Write to branches [iceberg-python]
vinjai commented on PR #941: URL: https://github.com/apache/iceberg-python/pull/941#issuecomment-2424158737 I have mostly tried to cover all edge cases. The idea is that the branch is just another iceberg table where the snapshots append independently of the main branch. I also agree with your concern. If it helps, we can add more test cases if there are any missing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] feat: Implement list_views Method and __is_view Utility Function [iceberg-python]
omkenge opened a new pull request, #1239: URL: https://github.com/apache/iceberg-python/pull/1239 - Implemented the `list_views` method to retrieve a list of views from the specified Glue database. - Added a utility method `__is_view` to check if a table is a view based on its metadata. The new functionality enhances the usability of the Glue catalog by allowing users to easily retrieve views, improving data management and accessibility. Hello team ( @sungwy @kevinjqliu ) Could you plz check this PR Thank You -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spec: Support geo type [iceberg]
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1807496639 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate. NaNs are not permitted as lower or upper bounds. 3. If sort order ID is missing or unknown, then the order is assumed to be unsorted. Only data files and equality delete files should be written with a non-null order id. [Position deletes](#position-delete-files) are required to be sorted by file and position, not a table order, and should set sort order id to null. Readers must ignore sort order id for position delete files. 4. The following field ids are reserved on `data_file`: 141. +5. For `geometry`, this is a point. X = min value of all component points of all geometries in file when edges = PLANAR, westernmost bound of all geometries in file if edges = SPHERICAL. Y = max value of all component points of all geometries in file when edges = PLANAR, northernmost bound of all geometries if edges = SPHERICAL. Z is min value for all component points of all geometries in the file. M is min value of all component points of all geometries in the file. See Appendix D for encoding. Review Comment: Thanks for the detailed explanation, will take a further look. My earlier comment was just that I didnt want to ship something that starts with 'pseudo' as it would brings down the confidence. I am ok with replacing Planar => Linear and Spherical => Geodesic, if there is general agreement, cc @jiayuasu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spec: Support geo type [iceberg]
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1807496639 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate. NaNs are not permitted as lower or upper bounds. 3. If sort order ID is missing or unknown, then the order is assumed to be unsorted. Only data files and equality delete files should be written with a non-null order id. [Position deletes](#position-delete-files) are required to be sorted by file and position, not a table order, and should set sort order id to null. Readers must ignore sort order id for position delete files. 4. The following field ids are reserved on `data_file`: 141. +5. For `geometry`, this is a point. X = min value of all component points of all geometries in file when edges = PLANAR, westernmost bound of all geometries in file if edges = SPHERICAL. Y = max value of all component points of all geometries in file when edges = PLANAR, northernmost bound of all geometries if edges = SPHERICAL. Z is min value for all component points of all geometries in the file. M is min value of all component points of all geometries in the file. See Appendix D for encoding. Review Comment: Thanks for the detailed explanation, will take a further look. My earlier comment was just that I didnt want to ship something that starts with 'pseudo' as it would brings down the confidence. We could add another subtype of geodesic later. I am ok with replacing Planar => Linear and Spherical => Geodesic, if there is general agreement, cc @jiayuasu. It would be great to have the same terminology as in the Parquet spec. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spec: Support geo type [iceberg]
szehon-ho commented on code in PR #10981: URL: https://github.com/apache/iceberg/pull/10981#discussion_r1807496639 ## format/spec.md: ## @@ -483,6 +485,8 @@ Notes: 2. For `float` and `double`, the value `-0.0` must precede `+0.0`, as in the IEEE 754 `totalOrder` predicate. NaNs are not permitted as lower or upper bounds. 3. If sort order ID is missing or unknown, then the order is assumed to be unsorted. Only data files and equality delete files should be written with a non-null order id. [Position deletes](#position-delete-files) are required to be sorted by file and position, not a table order, and should set sort order id to null. Readers must ignore sort order id for position delete files. 4. The following field ids are reserved on `data_file`: 141. +5. For `geometry`, this is a point. X = min value of all component points of all geometries in file when edges = PLANAR, westernmost bound of all geometries in file if edges = SPHERICAL. Y = max value of all component points of all geometries in file when edges = PLANAR, northernmost bound of all geometries if edges = SPHERICAL. Z is min value for all component points of all geometries in the file. M is min value of all component points of all geometries in the file. See Appendix D for encoding. Review Comment: Thanks for the detailed explanation, will take a further look. My earlier comment was just that I didnt want to ship something that starts with 'pseudo' as it would brings down the confidence. We could add another subtype of geodesic later if necessary. I am ok with replacing Planar => Linear and Spherical => Geodesic, if there is general agreement, cc @jiayuasu. It would be great to have the same terminology as in the Parquet spec. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spec: Fix table of content generation [iceberg]
rdblue commented on PR #11067: URL: https://github.com/apache/iceberg/pull/11067#issuecomment-2424245333 I think this is fine. @danielcweeks is the organization what you want? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
[PR] Snapshot `summary` map must have `operation` key [iceberg]
kevinjqliu opened a new pull request, #11354: URL: https://github.com/apache/iceberg/pull/11354 This PR adds additional constraint in the `SnapshotParser` and also includes tests to verify `Snapshot`'s `summary` field is optional in V1 but required in V2. See https://iceberg.apache.org/spec/#snapshots When serializing `Snapshot` to `JSON`, if the `summary` field exists, the `operation` key must also exist ([code](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/SnapshotParser.java#L63-L66)). In other words, the `summary` field without the `operation` key is considered invalid. Devlist discussion https://lists.apache.org/thread/h9qmrmlgxh91ol0y2v8olt90b9q6p9xr Relates to [apache/iceberg-python/#1106](https://github.com/apache/iceberg-python/issues/1106) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark 3.5: Fix testDeleteFileThenMetadataDelete failure due to table not refreshed [iceberg]
github-actions[bot] commented on PR #9551: URL: https://github.com/apache/iceberg/pull/9551#issuecomment-2424319486 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark: Support min/max/count push down for Identity partition columns [iceberg]
github-actions[bot] commented on PR #9457: URL: https://github.com/apache/iceberg/pull/9457#issuecomment-2424318549 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark: Fix SparkTable to use name and effective snapshotID for comparing [iceberg]
github-actions[bot] commented on PR #9455: URL: https://github.com/apache/iceberg/pull/9455#issuecomment-2424318541 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark: Fix reading 2 level array issue [iceberg]
github-actions[bot] closed pull request #9515: Spark: Fix reading 2 level array issue URL: https://github.com/apache/iceberg/pull/9515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark 3.4: Cleanup the code branch for merge distribution mode conf which is no longer needed [iceberg]
github-actions[bot] commented on PR #9561: URL: https://github.com/apache/iceberg/pull/9561#issuecomment-2424319504 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Build: Extract Gradle version from `gradle-wrapper.properties` [iceberg]
github-actions[bot] closed pull request #9448: Build: Extract Gradle version from `gradle-wrapper.properties` URL: https://github.com/apache/iceberg/pull/9448 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Core: Fix setting updated parquet compression property [iceberg]
github-actions[bot] commented on PR #9503: URL: https://github.com/apache/iceberg/pull/9503#issuecomment-2424319115 This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Pushed filters to Parquet file on best effort basis in Vectorized Reader [iceberg]
github-actions[bot] closed pull request #9479: Pushed filters to Parquet file on best effort basis in Vectorized Reader URL: https://github.com/apache/iceberg/pull/9479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark 3.4: Cleanup the code branch for merge distribution mode conf which is no longer needed [iceberg]
github-actions[bot] closed pull request #9561: Spark 3.4: Cleanup the code branch for merge distribution mode conf which is no longer needed URL: https://github.com/apache/iceberg/pull/9561 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Docs: Fix incorrect wget command in Flink documentation [iceberg]
github-actions[bot] closed pull request #9483: Docs: Fix incorrect wget command in Flink documentation URL: https://github.com/apache/iceberg/pull/9483 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Core: Fix setting updated parquet compression property [iceberg]
github-actions[bot] closed pull request #9503: Core: Fix setting updated parquet compression property URL: https://github.com/apache/iceberg/pull/9503 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark: Fix SparkTable to use name and effective snapshotID for comparing [iceberg]
github-actions[bot] closed pull request #9455: Spark: Fix SparkTable to use name and effective snapshotID for comparing URL: https://github.com/apache/iceberg/pull/9455 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] Spark: Support min/max/count push down for Identity partition columns [iceberg]
github-actions[bot] closed pull request #9457: Spark: Support min/max/count push down for Identity partition columns URL: https://github.com/apache/iceberg/pull/9457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [PR] AWS: Add Option to don't write non current columns in glue schema closes #7584 [iceberg]
github-actions[bot] closed pull request #9420: AWS: Add Option to don't write non current columns in glue schema closes #7584 URL: https://github.com/apache/iceberg/pull/9420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
Re: [I] [Feature] Provide Nightly Build to PyPi [iceberg-python]
djouallah commented on issue #872: URL: https://github.com/apache/iceberg-python/issues/872#issuecomment-2424388455 any news on this, currently pyiceberg is broken with polaris and will like to use the latest update that fix it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
