nk1506 commented on code in PR #9852:
URL: https://github.com/apache/iceberg/pull/9852#discussion_r1578924531
##
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveViewOperations.java:
##
@@ -0,0 +1,395 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
HonahX merged PR #658:
URL: https://github.com/apache/iceberg-python/pull/658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg
HonahX merged PR #648:
URL: https://github.com/apache/iceberg-python/pull/648
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg
yeezychao commented on issue #10219:
URL: https://github.com/apache/iceberg/issues/10219#issuecomment-2076423855
@pvary Thank you for your reply. I understand what you mean and I have also
read the doc. However, we want to implement a function similar to this
[demo](https://nightlies.apache
nk1506 commented on code in PR #9852:
URL: https://github.com/apache/iceberg/pull/9852#discussion_r1578904316
##
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java:
##
@@ -152,6 +173,38 @@ public List listTables(Namespace
namespace) {
}
}
+ @Overri
liurenjie1024 commented on issue #348:
URL: https://github.com/apache/iceberg-rust/issues/348#issuecomment-2076420477
Also as we discussed in [this
doc](https://docs.google.com/document/d/1YncDX-qQ1T9jBGQmJNtRcPU1trRi00cB8eykv5diKw4/edit?usp=sharing),
do you mind to add datafusion integrati
sdd commented on code in PR #323:
URL: https://github.com/apache/iceberg-rust/pull/323#discussion_r1578902001
##
crates/iceberg/src/scan.rs:
##
@@ -186,6 +239,27 @@ impl TableScan {
.boxed())
}
+fn create_manifest_eval_factory(
+//&self,
+id:
HonahX merged PR #657:
URL: https://github.com/apache/iceberg-python/pull/657
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg
pvary commented on issue #10219:
URL: https://github.com/apache/iceberg/issues/10219#issuecomment-2076401034
Also, here is the documentation which describes what this feature can and
can't do:
https://iceberg.apache.org/docs/nightly/flink-queries/#emitting-watermarks
--
This is an automa
pvary commented on code in PR #10207:
URL: https://github.com/apache/iceberg/pull/10207#discussion_r1578865383
##
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestStreamScanSql.java:
##
@@ -48,7 +48,9 @@
import org.junit.jupiter.api.AfterEach;
import org.jun
pvary commented on code in PR #10207:
URL: https://github.com/apache/iceberg/pull/10207#discussion_r1578858474
##
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java:
##
@@ -130,7 +131,9 @@ private ScanContext(
this.watermarkColumn = watermarkCo
HonahX merged PR #655:
URL: https://github.com/apache/iceberg-python/pull/655
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg
pvary commented on code in PR #10207:
URL: https://github.com/apache/iceberg/pull/10207#discussion_r1578850573
##
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java:
##
@@ -155,6 +155,13 @@ private void validate() {
Preconditions.checkArgumen
amogh-jahagirdar commented on PR #10140:
URL: https://github.com/apache/iceberg/pull/10140#issuecomment-2076302600
I didn't forget about this (again :) ) I've added a test to verify the error
code configuration get parsed and considered correctly and am adding tests for
ClientPoolImpl which
ZENOTME commented on code in PR #349:
URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1578812779
##
crates/iceberg/src/transaction.rs:
##
@@ -121,6 +166,270 @@ impl<'a> Transaction<'a> {
}
}
+/// FastAppendAction is a transaction action for fast append d
ZENOTME commented on code in PR #349:
URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1578811481
##
crates/iceberg/src/io.rs:
##
@@ -368,6 +368,9 @@ impl Storage {
new_props.insert("root".to_string(), DEFAULT_ROOT_PATH.to_string());
match sch
ZENOTME commented on PR #349:
URL: https://github.com/apache/iceberg-rust/pull/349#issuecomment-2076295905
cc @liurenjie1024 @Fokko @Xuanwo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
liurenjie1024 commented on PR #229:
URL: https://github.com/apache/iceberg-rust/pull/229#issuecomment-2076286522
> @JanKaul WDYT? I think this PR is ready for review, I can add the update
and delete in a separate PR.
Cool, I'll take a look first.
--
This is an automated message fro
lurnagao-dahua commented on code in PR #10210:
URL: https://github.com/apache/iceberg/pull/10210#discussion_r1578796953
##
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java:
##
@@ -1206,6 +1208,55 @@ public void testRemoveAndAddBackColumnFr
liurenjie1024 commented on issue #348:
URL: https://github.com/apache/iceberg-rust/issues/348#issuecomment-2076190004
About the glue, hive, rest catalogs, I think we already have integrations:
https://github.com/apache/iceberg-rust/blob/2018ffc87625bdff939aac791784d8eabc4eda38/crates/cata
aokolnychyi commented on PR #10200:
URL: https://github.com/apache/iceberg/pull/10200#issuecomment-2076189673
After taking a closer look at `BaseTaskWriter`, I think we may have a
correctness issue when encoding changes if the table contains multiple specs.
Our current implementation of `Ba
liurenjie1024 commented on issue #348:
URL: https://github.com/apache/iceberg-rust/issues/348#issuecomment-2076184578
Hi, @Fokko About the read projection part, currently we can convert parquet
files into arrow streams, but there are some limitations: it only support
primitive types, and sc
ajantha-bhat commented on PR #10183:
URL: https://github.com/apache/iceberg/pull/10183#issuecomment-2076163538
Added in a template with default TODO for people to come back and edit it.
**Please suggest if there are any better way to track the status of inprogress
proposal.**
mailin
QuakeWang commented on issue #343:
URL: https://github.com/apache/iceberg-rust/issues/343#issuecomment-2076129123
> > Please assign it to me, I will do it later : )
>
> Sure. Thanks! But this issue was blocked by #341 #340 now.
OK, I get it. I will proceed after #340 and #341 ar
stevenzwu commented on code in PR #10207:
URL: https://github.com/apache/iceberg/pull/10207#discussion_r1578677929
##
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestStreamScanSql.java:
##
@@ -48,7 +48,9 @@
import org.junit.jupiter.api.AfterEach;
import org
stevenzwu commented on code in PR #10200:
URL: https://github.com/apache/iceberg/pull/10200#discussion_r1578676203
##
data/src/test/java/org/apache/iceberg/io/TestTaskEqualityDeltaWriter.java:
##
@@ -409,6 +421,55 @@ public void testUpsertDataWithFullRowSchema() throws
IOExcept
stevenzwu commented on code in PR #10207:
URL: https://github.com/apache/iceberg/pull/10207#discussion_r1578672004
##
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java:
##
@@ -155,6 +155,13 @@ private void validate() {
Preconditions.checkArg
github-actions[bot] commented on issue #2690:
URL: https://github.com/apache/iceberg/issues/2690#issuecomment-2076076863
This issue has been automatically marked as stale because it has been open
for 180 days with no activity. It will be closed in next 14 days if no further
activity occurs.
github-actions[bot] commented on issue #2696:
URL: https://github.com/apache/iceberg/issues/2696#issuecomment-2076076877
This issue has been automatically marked as stale because it has been open
for 180 days with no activity. It will be closed in next 14 days if no further
activity occurs.
sullis commented on PR #10217:
URL: https://github.com/apache/iceberg/pull/10217#issuecomment-2075967011
Ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
dependabot[bot] opened a new pull request, #658:
URL: https://github.com/apache/iceberg-python/pull/658
Bumps [pytest-mock](https://github.com/pytest-dev/pytest-mock) from 3.12.0
to 3.14.0.
Release notes
Sourced from https://github.com/pytest-dev/pytest-mock/releases";>pytest-mock'
dependabot[bot] opened a new pull request, #657:
URL: https://github.com/apache/iceberg-python/pull/657
Bumps [mypy-boto3-glue](https://github.com/youtype/mypy_boto3_builder) from
1.34.35 to 1.34.88.
Commits
See full diff in https://github.com/youtype/mypy_boto3_builder/commits
dependabot[bot] opened a new pull request, #656:
URL: https://github.com/apache/iceberg-python/pull/656
Bumps [pytest-checkdocs](https://github.com/jaraco/pytest-checkdocs) from
2.10.1 to 2.12.0.
Changelog
Sourced from https://github.com/jaraco/pytest-checkdocs/blob/main/NEWS.rst";
dependabot[bot] opened a new pull request, #655:
URL: https://github.com/apache/iceberg-python/pull/655
Bumps [thrift](https://github.com/apache/thrift) from 0.16.0 to 0.20.0.
Release notes
Sourced from https://github.com/apache/thrift/releases";>thrift's releases.
Tag to ret
singhpk234 commented on code in PR #9830:
URL: https://github.com/apache/iceberg/pull/9830#discussion_r1578549234
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java:
##
@@ -557,6 +568,88 @@ public View loadView(Identifier ident) throws
NoSuchViewExcept
s-akhtar-baig commented on issue #351:
URL: https://github.com/apache/iceberg-rust/issues/351#issuecomment-2075818521
@Fokko, can you please assign this to me? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use th
s-akhtar-baig commented on issue #350:
URL: https://github.com/apache/iceberg-rust/issues/350#issuecomment-2075812800
@Fokko, can you please assign this to me? @sdd, would it be okay if I work
on this?
--
This is an automated message from the Apache Git Service.
To respond to the message
fqaiser94 commented on code in PR #10007:
URL: https://github.com/apache/iceberg/pull/10007#discussion_r1578482898
##
core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java:
##
@@ -243,9 +247,12 @@ private void add(DeleteFileHolder fileHolder) {
PartitionSpec fi
hamzaezzi commented on issue #515:
URL: https://github.com/apache/iceberg-python/issues/515#issuecomment-2075729610
> Interesting can you run `aws sts get-caller-identity` in the terminal to
ensure the right identity is being used?
>
> you can also, explicitly set the S3FileIO by pass
amogh-jahagirdar commented on code in PR #10150:
URL: https://github.com/apache/iceberg/pull/10150#discussion_r1578433465
##
core/src/test/java/org/apache/iceberg/TestOverwrite.java:
##
@@ -295,5 +343,6 @@ public void testValidatedOverwriteWithAppendSuccess() {
.hasMess
Fokko merged PR #10218:
URL: https://github.com/apache/iceberg/pull/10218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa
Fokko merged PR #10214:
URL: https://github.com/apache/iceberg/pull/10214
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa
Fokko merged PR #10215:
URL: https://github.com/apache/iceberg/pull/10215
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa
Fokko merged PR #10216:
URL: https://github.com/apache/iceberg/pull/10216
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa
amogh-jahagirdar opened a new pull request, #10218:
URL: https://github.com/apache/iceberg/pull/10218
To reviewers: Don't merge until #10215 and #10216 are in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
bitsondatadev commented on PR #10215:
URL: https://github.com/apache/iceberg/pull/10215#issuecomment-2075515412
Yeah, so three main things I checked:
* The view file was added so files add up
* Search index excludes were properly added to avoid duplicates
* The version in `mkdocs.yml
amogh-jahagirdar commented on PR #10215:
URL: https://github.com/apache/iceberg/pull/10215#issuecomment-2075467694
Yeah I think we're good, there were 2 commits to the 1.5.0 docs after #9929,
and the changes add up.
--
This is an automated message from the Apache Git Service.
To respond t
amogh-jahagirdar commented on PR #10215:
URL: https://github.com/apache/iceberg/pull/10215#issuecomment-2075457959
Hm this should have the same line number delta as #9929 since I just copied
the folder and edited 2 lines in that copy, not sure why this is adding 8700
lines instead of 8538.
ZENOTME opened a new pull request, #349:
URL: https://github.com/apache/iceberg-rust/pull/349
This PR is complete https://github.com/apache/iceberg-rust/issues/345.
1. It adds the MergeSnapshotAction to commit the data file
2. It init the e2e test for write and read
--
This is an a
amogh-jahagirdar merged PR #9926:
URL: https://github.com/apache/iceberg/pull/9926
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@ic
Fokko merged PR #10212:
URL: https://github.com/apache/iceberg/pull/10212
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa
jurossiar opened a new issue, #10213:
URL: https://github.com/apache/iceberg/issues/10213
### Apache Iceberg version
1.4.2
### Query engine
None
### Please describe the bug 🐞
We are using version 1.4.2 but we are planning to upgrade to 1.5.0 and I
confirmed
nastra opened a new pull request, #10212:
URL: https://github.com/apache/iceberg/pull/10212
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-m
lkokhreidze opened a new issue, #10211:
URL: https://github.com/apache/iceberg/issues/10211
### Query engine
Flink
### Question
Hello,
I searched the documentation as much as possible but couldn't find the
decisive answer.
We have a Flink Streaming pipeline tha
nastra commented on code in PR #10210:
URL: https://github.com/apache/iceberg/pull/10210#discussion_r1577928873
##
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java:
##
@@ -1206,6 +1208,55 @@ public void testRemoveAndAddBackColumnFromIceber
nastra commented on code in PR #10210:
URL: https://github.com/apache/iceberg/pull/10210#discussion_r1577919902
##
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java:
##
@@ -1206,6 +1208,55 @@ public void testRemoveAndAddBackColumnFromIceber
nastra commented on code in PR #10210:
URL: https://github.com/apache/iceberg/pull/10210#discussion_r1577918910
##
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java:
##
@@ -1206,6 +1208,55 @@ public void testRemoveAndAddBackColumnFromIceber
Fokko commented on issue #348:
URL: https://github.com/apache/iceberg-rust/issues/348#issuecomment-2074960443
@marvinlanhenke No problem, thank you for all the work on the project. While
compiling this I realized how much work has been done 🚀
--
This is an automated message from the Apac
manuzhang commented on PR #10134:
URL: https://github.com/apache/iceberg/pull/10134#issuecomment-2074889892
Sorry, I missed Fokko's comment. I'm ok with either way to clean up backlog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
Fokko merged PR #647:
URL: https://github.com/apache/iceberg-python/pull/647
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.
lurnagao-dahua commented on PR #10210:
URL: https://github.com/apache/iceberg/pull/10210#issuecomment-2074824386
Hi, Can you take a look for me? @nastra
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
lurnagao-dahua closed pull request #7359: Hive: Fix using Date type as
partition field
URL: https://github.com/apache/iceberg/pull/7359
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific c
pvary commented on code in PR #10188:
URL: https://github.com/apache/iceberg/pull/10188#discussion_r1577765690
##
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergStorageHandler.java:
##
@@ -111,8 +111,15 @@ public void configureTableJobProperties(TableDesc
tableDesc, Map
pvary commented on code in PR #10200:
URL: https://github.com/apache/iceberg/pull/10200#discussion_r1577755913
##
core/src/main/java/org/apache/iceberg/deletes/ContinuousFileScopedPositionDeleteWriter.java:
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (
pvary commented on code in PR #10200:
URL: https://github.com/apache/iceberg/pull/10200#discussion_r1577753275
##
flink/v1.18/flink/src/main/java/org/apache/iceberg/flink/sink/BaseDeltaTaskWriter.java:
##
@@ -56,7 +58,8 @@ abstract class BaseDeltaTaskWriter extends
BaseTaskWrit
pvary commented on code in PR #10200:
URL: https://github.com/apache/iceberg/pull/10200#discussion_r1577749819
##
core/src/main/java/org/apache/iceberg/deletes/ContinuousFileScopedPositionDeleteWriter.java:
##
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (
pvary commented on code in PR #10208:
URL: https://github.com/apache/iceberg/pull/10208#discussion_r1577734223
##
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java:
##
@@ -88,16 +88,11 @@ protected void assertRec
pvary commented on code in PR #10208:
URL: https://github.com/apache/iceberg/pull/10208#discussion_r1577733915
##
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/SimpleDataUtil.java:
##
@@ -284,11 +284,7 @@ public static void assertRecordsEqual(List
expected, List actu
marvinlanhenke commented on issue #348:
URL: https://github.com/apache/iceberg-rust/issues/348#issuecomment-2074712249
@Fokko thanks for your effort here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
pvary commented on code in PR #10207:
URL: https://github.com/apache/iceberg/pull/10207#discussion_r1577703711
##
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/source/ScanContext.java:
##
@@ -155,6 +155,13 @@ private void validate() {
Preconditions.checkArgumen
pvary commented on code in PR #10208:
URL: https://github.com/apache/iceberg/pull/10208#discussion_r1577692634
##
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java:
##
@@ -40,24 +44,27 @@
import org.apache.iceberg.FileFormat;
import
pvary commented on code in PR #10208:
URL: https://github.com/apache/iceberg/pull/10208#discussion_r1577691189
##
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java:
##
@@ -149,6 +163,15 @@ private void testBoundedIcebergSource(Failove
pvary commented on code in PR #10208:
URL: https://github.com/apache/iceberg/pull/10208#discussion_r1577690466
##
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java:
##
@@ -201,8 +201,11 @@ private SplitEnumerator createEnumer
return new Co
jbonofre commented on PR #10134:
URL: https://github.com/apache/iceberg/pull/10134#issuecomment-2074521876
Yup. I reverted my change due to Fokko's comment. But I can change it back
and adapt once merged.
--
This is an automated message from the Apache Git Service.
To respond to the mess
Fokko commented on code in PR #322:
URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1577458027
##
crates/iceberg/src/expr/visitors/manifest_evaluator.rs:
##
@@ -0,0 +1,466 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
sdd commented on code in PR #323:
URL: https://github.com/apache/iceberg-rust/pull/323#discussion_r1577443557
##
crates/iceberg/src/scan.rs:
##
@@ -186,6 +239,27 @@ impl TableScan {
.boxed())
}
+fn create_manifest_eval_factory(
Review Comment:
Aah yes, th
sdd commented on code in PR #322:
URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1577440808
##
crates/iceberg/src/expr/visitors/manifest_evaluator.rs:
##
@@ -0,0 +1,459 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor li
sdd commented on code in PR #322:
URL: https://github.com/apache/iceberg-rust/pull/322#discussion_r1577440808
##
crates/iceberg/src/expr/visitors/manifest_evaluator.rs:
##
@@ -0,0 +1,459 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor li
manuzhang commented on PR #10134:
URL: https://github.com/apache/iceberg/pull/10134#issuecomment-2074275369
> Does the same apply when you push to a branch but don't comment on it?
Yes, as per
[doc](https://github.com/actions/stale?tab=readme-ov-file#days-before-stale)
> The issues
ZENOTME commented on issue #343:
URL: https://github.com/apache/iceberg-rust/issues/343#issuecomment-2074252906
> Please assign it to me, I will do it later : )
Sure. Thanks! But this issue was blocked by #341 #340 now.
--
This is an automated message from the Apache Git Service.
To
martin-g commented on issue #338:
URL: https://github.com/apache/iceberg-rust/issues/338#issuecomment-2074220391
There were some discussions about doing separate SDK releases -
https://lists.apache.org/thread/2rfnszd4dk36jxynpj382b1717gbyv1y but nothing
happened mainly due to the lack of in
81 matches
Mail list logo
