Re: [I] Duplicate file name in Iceberg's metadata [iceberg]

2023-11-12 Thread via GitHub
github-raphael-douyere commented on issue #8953: URL: https://github.com/apache/iceberg/issues/8953#issuecomment-1807614244 Not sure it helps but I need to mention that our streaming app is restarted a lot. Maybe all work fine when everything is kept in memory but on restart some elements a

Re: [I] manifest exception [iceberg]

2023-11-12 Thread via GitHub
innocent123 commented on issue #8994: URL: https://github.com/apache/iceberg/issues/8994#issuecomment-1807611823 > > Thank you very much! my historical snapshot is lost ,You can only recover by deleting files.One more question.Is there a way to quickly know that the manifest file no lon

Re: [I] manifest exception [iceberg]

2023-11-12 Thread via GitHub
wangtaohz commented on issue #8994: URL: https://github.com/apache/iceberg/issues/8994#issuecomment-1807605797 > Thank you very much! my historical snapshot is lost ,You can only recover by deleting files.One more question.Is there a way to quickly know that the manifest file no longer exis

Re: [I] manifest exception [iceberg]

2023-11-12 Thread via GitHub
innocent123 commented on issue #8994: URL: https://github.com/apache/iceberg/issues/8994#issuecomment-1807569922 > > The problem is that the path in the manifest file does not exist in the hdfs. However, I cannot modify the manifest file directly. Is there an api that can modify the

Re: [I] manifest exception [iceberg]

2023-11-12 Thread via GitHub
wangtaohz commented on issue #8994: URL: https://github.com/apache/iceberg/issues/8994#issuecomment-1807532630 > The problem is that the path in the manifest file does not exist in the hdfs. However, I cannot modify the manifest file directly. Is there an api that can modify the manifest fi

Re: [PR] API: Optimize equals in CharSequenceWrapper [iceberg]

2023-11-12 Thread via GitHub
aokolnychyi commented on PR #9035: URL: https://github.com/apache/iceberg/pull/9035#issuecomment-1807481467 Thanks, @singhpk234 @amogh-jahagirdar! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] API: Optimize equals in CharSequenceWrapper [iceberg]

2023-11-12 Thread via GitHub
amogh-jahagirdar merged PR #9035: URL: https://github.com/apache/iceberg/pull/9035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] Replace i64 with DateTime [iceberg-rust]

2023-11-12 Thread via GitHub
liurenjie1024 commented on code in PR #94: URL: https://github.com/apache/iceberg-rust/pull/94#discussion_r1390573721 ## crates/iceberg/src/spec/snapshot.rs: ## @@ -72,7 +73,7 @@ pub struct Snapshot { sequence_number: i64, /// A timestamp when the snapshot was created,

Re: [I] manifest exception [iceberg]

2023-11-12 Thread via GitHub
innocent123 commented on issue #8994: URL: https://github.com/apache/iceberg/issues/8994#issuecomment-1807367945 The problem is that the path in the manifest file does not exist in the hdfs. However, I cannot modify the manifest file directly. Is there an api that can modify the manifest fi

Re: [I] manifest exception [iceberg]

2023-11-12 Thread via GitHub
innocent123 commented on issue #8994: URL: https://github.com/apache/iceberg/issues/8994#issuecomment-1807366431 > I guess the missing data file was caused by `deleteOrphanFiles set olderThan(System.currentTimeMillis())`, which mistakenly treated the uncommitted data files as orphan files a

Re: [I] Arrow reader fails to filter results [iceberg]

2023-11-12 Thread via GitHub
github-actions[bot] commented on issue #7556: URL: https://github.com/apache/iceberg/issues/7556#issuecomment-1807293720 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

Re: [I] Delete across all snapshots [iceberg]

2023-11-12 Thread via GitHub
github-actions[bot] commented on issue #7622: URL: https://github.com/apache/iceberg/issues/7622#issuecomment-1807293697 This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs.

[PR] API: Add CharSequenceMap [iceberg]

2023-11-12 Thread via GitHub
aokolnychyi opened a new pull request, #9047: URL: https://github.com/apache/iceberg/pull/9047 This PR adds `CharSequenceMap`, a map implementation similar to existing `CharSequenceSet`. This will be used in writers, delete file index, Spark executor-level content file cache. -- This is

Re: [PR] Docs: Remove UNIQUE keyword as it is not supported in Flink [iceberg]

2023-11-12 Thread via GitHub
PrabhuJoseph commented on PR #9046: URL: https://github.com/apache/iceberg/pull/9046#issuecomment-1807102358 @pvary Please review this when you have time. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Doc Bug: Iceberg Flink Example uses unsupported UNIQUE constraint [iceberg]

2023-11-12 Thread via GitHub
PrabhuJoseph commented on issue #8997: URL: https://github.com/apache/iceberg/issues/8997#issuecomment-1807102144 Have raised a PR (https://github.com/apache/iceberg/pull/9046) to fix this issue. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.22.0 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] closed pull request #30: build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.22.0 URL: https://github.com/apache/iceberg-go/pull/30 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.23.0 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] opened a new pull request, #35: URL: https://github.com/apache/iceberg-go/pull/35 Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.19.1 to 1.23.0. Commits https://github.com/aws/aws-sdk-go-v2/commit/66535243eae1c6b83035c4

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.19.1 to 1.22.0 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] commented on PR #30: URL: https://github.com/apache/iceberg-go/pull/30#issuecomment-1807057978 Superseded by #35. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.1 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] closed pull request #27: build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.1 URL: https://github.com/apache/iceberg-go/pull/27 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.1 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] commented on PR #27: URL: https://github.com/apache/iceberg-go/pull/27#issuecomment-1807057563 Superseded by #34. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[PR] build(deps): bump github.com/aws/aws-sdk-go-v2 from 1.21.2 to 1.22.2 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] opened a new pull request, #34: URL: https://github.com/apache/iceberg-go/pull/34 Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.21.2 to 1.22.2. Commits https://github.com/aws/aws-sdk-go-v2/commit/5aa6e2b081b5d1b038a0cc5d9198c7

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.1 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] commented on PR #28: URL: https://github.com/apache/iceberg-go/pull/28#issuecomment-1807056920 Superseded by #33. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

Re: [PR] build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.1 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] closed pull request #28: build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.1 URL: https://github.com/apache/iceberg-go/pull/28 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] build(deps): bump github.com/hamba/avro/v2 from 2.16.0 to 2.17.2 [iceberg-go]

2023-11-12 Thread via GitHub
dependabot[bot] opened a new pull request, #33: URL: https://github.com/apache/iceberg-go/pull/33 Bumps [github.com/hamba/avro/v2](https://github.com/hamba/avro) from 2.16.0 to 2.17.2. Release notes Sourced from https://github.com/hamba/avro/releases";>github.com/hamba/avro/v2's r

[PR] [WIP] Table metadata update, table requirement validation, and glue catalog commit [iceberg-python]

2023-11-12 Thread via GitHub
HonahX opened a new pull request, #140: URL: https://github.com/apache/iceberg-python/pull/140 Fixes #29 #22 This PR is still in progress. It contains a complete logic to commit a transaction via GlueCatalog. This PR can be divided into three parts: - Table metadata update: (which