nastra commented on issue #9142:
URL: https://github.com/apache/iceberg/issues/9142#issuecomment-1825262409
that's a good idea @gabrywu, could you please open a PR to fix this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
irshadcc commented on issue #9118:
URL: https://github.com/apache/iceberg/issues/9118#issuecomment-1825260470
After the optimisation, the time taken to evaluate 25,000 manifest files
with schema containing 35,000 columns improved from 6 minutes 22 seconds to 13
seconds.
--
This is an au
tomtongue commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404014878
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestSparkFileRewriter.java:
##
@@ -190,9 +189,9 @@ private void
checkDataFileGroupWithTooMuchData(
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404012356
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestSparkFileRewriter.java:
##
@@ -190,9 +189,9 @@ private void
checkDataFileGroupWithTooMuchData(Siz
nk1506 opened a new pull request, #9143:
URL: https://github.com/apache/iceberg/pull/9143
[Throwables.propagate](https://github.com/google/guava/blob/e5cc39c9e0bcca1347a937cb090321fb96de0e5c/guava/src/com/google/common/base/Throwables.java#L229)
always translate to `RuntimeException`
Add
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404011983
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestSparkFileRewriter.java:
##
@@ -110,9 +109,9 @@ private void
checkDataFileSizeFiltering(SizeBasedD
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404011633
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestSparkFileRewriter.java:
##
@@ -110,9 +109,9 @@ private void
checkDataFileSizeFiltering(SizeBasedD
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404011518
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/actions/TestSparkFileRewriter.java:
##
@@ -129,9 +128,9 @@ private void
checkDataFilesDeleteThreshold(SizeBas
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404010724
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/SparkTestHelperBase.java:
##
@@ -69,19 +72,25 @@ protected void assertEquals(
}
protected void assertE
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404010177
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/SparkTestHelperBase.java:
##
@@ -55,12 +55,15 @@ private Object[] toJava(Row row) {
protected void assert
nastra commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1404009911
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/SparkTestHelperBase.java:
##
@@ -55,12 +55,15 @@ private Object[] toJava(Row row) {
protected void assert
nastra merged PR #9120:
URL: https://github.com/apache/iceberg/pull/9120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac
nastra closed issue #9078: Create JUnit5-version of FlinkTestBase
URL: https://github.com/apache/iceberg/issues/9078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscr
jia-zhengwei commented on issue #8508:
URL: https://github.com/apache/iceberg/issues/8508#issuecomment-1825179396
> You should use `MERGE INTO` if you want to do upsert. `INSERT INTO` will
append data instead of upsert data.
Got it, Thanks.
--
This is an automated message from th
pvary commented on PR #9011:
URL: https://github.com/apache/iceberg/pull/9011#issuecomment-1825177267
Thanks for all the work and patience @nk1506 !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
pvary merged PR #9011:
URL: https://github.com/apache/iceberg/pull/9011
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach
pvary commented on code in PR #9011:
URL: https://github.com/apache/iceberg/pull/9011#discussion_r1403960661
##
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:
##
@@ -207,11 +195,15 @@ protected void doCommit(TableMetadata base, TableMetadata
meta
pvary commented on issue #9138:
URL: https://github.com/apache/iceberg/issues/9138#issuecomment-1825173708
Once #9139 is merged, you can use it to compile your own version of
iceberg-flink-runtime, but officially it will not be supported by the
community, because 1.15 support of this featur
gabrywu opened a new issue, #9142:
URL: https://github.com/apache/iceberg/issues/9142
### Feature Request / Improvement
BaseLockManager use a static field scheduler, whose type is
ScheduledExecutorService, however, it doesn't shutdown when closing.
Can we close it since its corres
Xuanwo commented on PR #97:
URL: https://github.com/apache/iceberg-rust/pull/97#issuecomment-1825065124
> cc @Xuanwo Any other comments?
LGTM, let's go!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
liurenjie1024 commented on PR #97:
URL: https://github.com/apache/iceberg-rust/pull/97#issuecomment-1825064345
cc @Xuanwo Any other comments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the s
ConeyLiu commented on issue #8508:
URL: https://github.com/apache/iceberg/issues/8508#issuecomment-1825058664
You should use `MERGE INTO` if you want to do upsert. `INSERT INTO` will
append data instead of upsert data.
--
This is an automated message from the Apache Git Service.
To respon
github-actions[bot] closed issue #7446: Move iceberg table data from one bucket
to another using spark
URL: https://github.com/apache/iceberg/issues/7446
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
github-actions[bot] commented on issue #7418:
URL: https://github.com/apache/iceberg/issues/7418#issuecomment-1825012241
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
github-actions[bot] closed issue #7418: Hive: Add check for Hive version when
HIVE-26882 is released
URL: https://github.com/apache/iceberg/issues/7418
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t
github-actions[bot] commented on issue #7446:
URL: https://github.com/apache/iceberg/issues/7446#issuecomment-1825012223
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
github-actions[bot] commented on issue #7570:
URL: https://github.com/apache/iceberg/issues/7570#issuecomment-1825012209
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
github-actions[bot] closed issue #7570: Iceberg / Spark writing to s3 warehouse
: Unable to load region from any of the providers in the chain software
URL: https://github.com/apache/iceberg/issues/7570
--
This is an automated message from the Apache Git Service.
To respond to the message, pl
dependabot[bot] opened a new pull request, #162:
URL: https://github.com/apache/iceberg-python/pull/162
Bumps [mkdocs-material](https://github.com/squidfunk/mkdocs-material) from
9.4.10 to 9.4.11.
Release notes
Sourced from https://github.com/squidfunk/mkdocs-material/releases";>mk
stevenzwu commented on PR #9139:
URL: https://github.com/apache/iceberg/pull/9139#issuecomment-1824802554
@pvary can you separate this out as a separate PR?
Fix backport differences for https://github.com/apache/iceberg/pull/8852
(https://github.com/apache/iceberg/pull/8877,
https://
amogh-jahagirdar merged PR #8854:
URL: https://github.com/apache/iceberg/pull/8854
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@ic
amogh-jahagirdar commented on PR #8854:
URL: https://github.com/apache/iceberg/pull/8854#issuecomment-1824793066
Thanks for the reviews @nastra @rakesh-das08 @ajantha-bhat @aokolnychyi !
I'm going to merge this to keep the fix focused on the procedure output for
non-main branches. There are
nk1506 commented on code in PR #9011:
URL: https://github.com/apache/iceberg/pull/9011#discussion_r1403606978
##
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:
##
@@ -207,11 +195,15 @@ protected void doCommit(TableMetadata base, TableMetadata
met
andreacfm opened a new pull request, #9141:
URL: https://github.com/apache/iceberg/pull/9141
Added test about CDC creation failure after expiring all the snapshots.
Test case:
* update or delete at least a record in a table
* expire all the snapshots (keep just the last one a
tomtongue commented on PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#issuecomment-1824673027
Thanks for the review, @nastra! I add the following changes based on the
reviews:
* Change `SparkTestHelperBase` with the AssertJ style (removed
`TestHelperBase`)
* Switch `TestSp
tomtongue commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1403559874
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestHelperBase.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
tomtongue commented on code in PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#discussion_r1403559874
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/TestHelperBase.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
pvary commented on issue #9138:
URL: https://github.com/apache/iceberg/issues/9138#issuecomment-1824568318
> Our watermark column are of type timestamp inhereted from kafka through
flink before outputed to iceberg. Can I assume this is is interpreted correctly?
Yes, that's the plan
lpy148145 commented on issue #9140:
URL: https://github.com/apache/iceberg/issues/9140#issuecomment-1824563029
2023-11-23 20:39:52,422 INFO
org.apache.iceberg.flink.sink.IcebergFilesCommitter [] - Checkpoint
128740 completed. Attempting commit.
2023-11-23 20:39:52,652 INFO org
lpy148145 commented on issue #9140:
URL: https://github.com/apache/iceberg/issues/9140#issuecomment-1824547529
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:3181) ~[?:1.8.0_292]
at
org.apache.iceberg.relocated.com.google.c
nastra commented on issue #9140:
URL: https://github.com/apache/iceberg/issues/9140#issuecomment-1824524158
@lpy148145 can you please provide the full stack trace and any additional
details that can be helpful?
--
This is an automated message from the Apache Git Service.
To respond to the
jonathf commented on issue #9138:
URL: https://github.com/apache/iceberg/issues/9138#issuecomment-1824520586
Okay, understood.
I misunderstood the column metrics. Looks like metrics are off on our end,
but seems like they can easily be added through AWS Glue. We will definetly
give i
lpy148145 commented on issue #9140:
URL: https://github.com/apache/iceberg/issues/9140#issuecomment-1824515573
The snapshot with ID 8030770763716459131 exists in the metadata.json file
but cannot be found in the table.
--
This is an automated message from the Apache Git Service.
To respon
lpy148145 opened a new issue, #9140:
URL: https://github.com/apache/iceberg/issues/9140
### Apache Iceberg version
1.2.1
### Query engine
Flink
### Please describe the bug 🐞
The snapshots_id recorded in """*.metadata.json""" cannot be found in the
table aft
pvary commented on PR #9139:
URL: https://github.com/apache/iceberg/pull/9139#issuecomment-1824501764
CC: @nastra, @nk1506 - When backporting changes, please make sure that all
of the changes as similar as possible.
After fixing the backport differences caused by previous backports I
nastra commented on code in PR #9131:
URL: https://github.com/apache/iceberg/pull/9131#discussion_r1403420355
##
spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java:
##
@@ -171,7 +171,7 @@ public Table loadTable(Identifier ident, String version)
throws NoS
cccs-jc commented on PR #8980:
URL: https://github.com/apache/iceberg/pull/8980#issuecomment-1824466349
@singhpk234 As you recommended I removed the 3.4 implementation and only
kept one version 3.5.
However, now the test cases for 3.4 are failing. Any idea how to fix this.
Should I
snazy commented on code in PR #8857:
URL: https://github.com/apache/iceberg/pull/8857#discussion_r1403377308
##
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##
@@ -181,133 +186,206 @@ public IcebergTable table(TableIdentifier
tableIdentifier) {
}
shidayang closed pull request #4459: Core: HadoopTables support dynamic FileIO
URL: https://github.com/apache/iceberg/pull/4459
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
shidayang closed pull request #5264: Cache PositionIndex
URL: https://github.com/apache/iceberg/pull/5264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail
pvary commented on issue #9138:
URL: https://github.com/apache/iceberg/issues/9138#issuecomment-1824412420
#9139 is the backport PR.
> We are on AWS managed Flink application which is currently limited upward
to 1.15. Do you know anything about if or when AWS will update their support
jonathf commented on issue #9138:
URL: https://github.com/apache/iceberg/issues/9138#issuecomment-1824362361
> @jonathf: Always good to know if somebody is interested in a feature, so
feel free to ask your questions!
I am glad to hear that! And thank you for the swift and in-depth ans
liurenjie1024 commented on code in PR #103:
URL: https://github.com/apache/iceberg-rust/pull/103#discussion_r1403271641
##
crates/iceberg/src/spec/values.rs:
##
@@ -554,13 +554,14 @@ impl From<&Literal> for JsonValue {
PrimitiveLiteral::Decimal(_) => todo!(),
pvary commented on issue #9138:
URL: https://github.com/apache/iceberg/issues/9138#issuecomment-1824274267
@jonathf: Always good to know if somebody is interested in a feature, so
feel free to ask your questions!
> 1. Which versions of Flink will this support? Or is it agnostic to th
pvary commented on code in PR #9011:
URL: https://github.com/apache/iceberg/pull/9011#discussion_r1403245669
##
hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java:
##
@@ -207,11 +195,15 @@ protected void doCommit(TableMetadata base, TableMetadata
meta
pvary commented on code in PR #9120:
URL: https://github.com/apache/iceberg/pull/9120#discussion_r1403236720
##
flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/MiniFlinkClusterResource.java:
##
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) un
pvary commented on PR #9046:
URL: https://github.com/apache/iceberg/pull/9046#issuecomment-1824234049
Thanks for the fix @PrabhuJoseph!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specifi
pvary merged PR #9046:
URL: https://github.com/apache/iceberg/pull/9046
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach
jonathf opened a new issue, #9138:
URL: https://github.com/apache/iceberg/issues/9138
### Query engine
Flink 1.15.2
### Question
@pvary, congratulations on getting #8553 merged! It has benn interesting to
follow the progress to the PR.
I know that it is still goin
nastra commented on code in PR #8854:
URL: https://github.com/apache/iceberg/pull/8854#discussion_r1403186632
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/FastForwardBranchProcedure.java:
##
@@ -77,12 +79,13 @@ public InternalRow[] call(InternalRow args)
nastra commented on code in PR #8854:
URL: https://github.com/apache/iceberg/pull/8854#discussion_r1403184858
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/FastForwardBranchProcedure.java:
##
@@ -18,6 +18,8 @@
*/
package org.apache.iceberg.spark.proced
nastra commented on code in PR #9131:
URL: https://github.com/apache/iceberg/pull/9131#discussion_r1403181705
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSnapshotSelection.java:
##
@@ -425,16 +426,35 @@ public void
testSnapshotSelectionByBranchWithSche
nastra commented on code in PR #9131:
URL: https://github.com/apache/iceberg/pull/9131#discussion_r1403161057
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSnapshotSelection.java:
##
@@ -425,16 +426,35 @@ public void
testSnapshotSelectionByBranchWithSche
nastra commented on code in PR #9131:
URL: https://github.com/apache/iceberg/pull/9131#discussion_r1403161057
##
spark/v3.5/spark/src/test/java/org/apache/iceberg/spark/source/TestSnapshotSelection.java:
##
@@ -425,16 +426,35 @@ public void
testSnapshotSelectionByBranchWithSche
zhongyujiang commented on issue #9130:
URL: https://github.com/apache/iceberg/issues/9130#issuecomment-1824103082
Agg push down has the same issue:
--
This is an automated message from
nastra commented on code in PR #8909:
URL: https://github.com/apache/iceberg/pull/8909#discussion_r1403134265
##
core/src/test/java/org/apache/iceberg/view/ViewCatalogTests.java:
##
@@ -400,8 +400,15 @@ public void
replaceTableViaTransactionThatAlreadyExistsAsView() {
pvary commented on PR #8553:
URL: https://github.com/apache/iceberg/pull/8553#issuecomment-1824077702
Merged to main.
Thanks @stevenzwu, @dchristle, @sundargates, @gyfora and @jerqi for the
reviews!
--
This is an automated message from the Apache Git Service.
To respond to the message
pvary merged PR #8553:
URL: https://github.com/apache/iceberg/pull/8553
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach
snazy commented on code in PR #8909:
URL: https://github.com/apache/iceberg/pull/8909#discussion_r1402432505
##
nessie/src/main/java/org/apache/iceberg/nessie/NessieIcebergClient.java:
##
@@ -540,4 +634,35 @@ public void close() {
api.close();
}
}
+
+ public void
othmanefc opened a new issue, #161:
URL: https://github.com/apache/iceberg-python/issues/161
### Apache Iceberg version
0.4.0
### Please describe the bug 🐞
Create Table in SQL Catalog fails if location is not mentioned but
write.data.path and write.metadata.path.
bluzy closed issue #6750: Failed to get table info from metastore using
impersonation
URL: https://github.com/apache/iceberg/issues/6750
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
pvary commented on code in PR #8553:
URL: https://github.com/apache/iceberg/pull/8553#discussion_r1403082807
##
flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java:
##
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software
mobley-trent commented on code in PR #103:
URL: https://github.com/apache/iceberg-rust/pull/103#discussion_r1403082258
##
crates/iceberg/src/spec/values.rs:
##
@@ -554,13 +554,14 @@ impl From<&Literal> for JsonValue {
PrimitiveLiteral::Decimal(_) => todo!(),
pvary commented on code in PR #8553:
URL: https://github.com/apache/iceberg/pull/8553#discussion_r1403079553
##
flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java:
##
@@ -453,6 +492,18 @@ public IcebergSource build() {
contextBuilder.proj
amogh-jahagirdar commented on code in PR #8854:
URL: https://github.com/apache/iceberg/pull/8854#discussion_r1403040251
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/procedures/FastForwardBranchProcedure.java:
##
@@ -77,9 +77,9 @@ public InternalRow[] call(InternalR
pvary commented on code in PR #8553:
URL: https://github.com/apache/iceberg/pull/8553#discussion_r1403031058
##
flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java:
##
@@ -0,0 +1,451 @@
+/*
+ * Licensed to the Apache Softw
nastra commented on PR #9129:
URL: https://github.com/apache/iceberg/pull/9129#issuecomment-1823947363
> Add the following classes that basically cover JUnit5 and AssertJ styles.
Each class has a lot of inheritants so that new classes are created for now.
And, can I update existing Spark te
77 matches
Mail list logo
