nastra commented on issue #10245:
URL: https://github.com/apache/iceberg/issues/10245#issuecomment-2082002342
This will be fixed by https://github.com/apache/iceberg/pull/10045
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
sdd commented on PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#issuecomment-2082001999
@marvinlanhenke Thanks for taking the time for the great review. Have
addressed your raised points. I'm adding tests based on
https://github.com/apache/iceberg/blob/main/api/src/test/java/
sdd commented on code in PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1582607827
##
crates/iceberg/src/scan.rs:
##
@@ -262,6 +288,16 @@ impl TableScan {
)
}
+fn create_data_file_evaluator(
Review Comment:
Done!
--
This is
sdd commented on code in PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1582607631
##
crates/iceberg/src/expr/visitors/inclusive_metrics_evaluator.rs:
##
@@ -0,0 +1,657 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contr
sdd commented on code in PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1582607453
##
crates/iceberg/src/expr/visitors/inclusive_metrics_evaluator.rs:
##
@@ -0,0 +1,657 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contr
nastra merged PR #10236:
URL: https://github.com/apache/iceberg/pull/10236
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap
nastra merged PR #10239:
URL: https://github.com/apache/iceberg/pull/10239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap
nastra merged PR #10243:
URL: https://github.com/apache/iceberg/pull/10243
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap
nastra merged PR #10240:
URL: https://github.com/apache/iceberg/pull/10240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap
Fokko merged PR #664:
URL: https://github.com/apache/iceberg-python/pull/664
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582572379
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
sdd commented on code in PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1582568159
##
crates/iceberg/src/expr/visitors/inclusive_metrics_evaluator.rs:
##
@@ -0,0 +1,657 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contr
sdd commented on code in PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1582562229
##
crates/iceberg/src/scan.rs:
##
@@ -218,6 +225,25 @@ impl TableScan {
let mut manifest_entries =
iter(manifest.entries().iter().filter(|e| e.is_ali
shivjha30 commented on code in PR #1495:
URL: https://github.com/apache/iceberg/pull/1495#discussion_r1582544488
##
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergMetaHook.java:
##
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * o
manuzhang commented on issue #10234:
URL: https://github.com/apache/iceberg/issues/10234#issuecomment-2081756823
Which Spark version are you using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
tomtongue commented on PR #10232:
URL: https://github.com/apache/iceberg/pull/10232#issuecomment-2081755863
@nastra when you have a chance, could you review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
github-actions[bot] commented on issue #2725:
URL: https://github.com/apache/iceberg/issues/2725#issuecomment-2081711270
This issue has been automatically marked as stale because it has been open
for 180 days with no activity. It will be closed in next 14 days if no further
activity occurs.
github-actions[bot] commented on issue #2493:
URL: https://github.com/apache/iceberg/issues/2493#issuecomment-2081711197
This issue has been closed because it has not received any activity in the
last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Gi
github-actions[bot] closed issue #2493: expireSnapshots excepts with V2 when
delete-after-commit.enabled=true and previous-versions-max=1
URL: https://github.com/apache/iceberg/issues/2493
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on t
RahmanQureshi commented on code in PR #4479:
URL: https://github.com/apache/iceberg/pull/4479#discussion_r1582397928
##
spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreamingRead3.java:
##
@@ -138,6 +143,57 @@ public void
testReadStreamOnIcebergT
RahmanQureshi commented on code in PR #4479:
URL: https://github.com/apache/iceberg/pull/4479#discussion_r1582396495
##
spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/source/TestStructuredStreamingRead3.java:
##
@@ -138,6 +143,57 @@ public void
testReadStreamOnIcebergT
YannickLecroart21 commented on issue #7570:
URL: https://github.com/apache/iceberg/issues/7570#issuecomment-2081657309
> @dramaticlly
>
> Thank you for your suggestion. I wasn't able to properly set the
environment variables using that specific approach. But it got me thinking
about
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582370249
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582337529
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582338872
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582301433
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582301676
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582300860
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +216,399 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
ms opened a new issue, #10245:
URL: https://github.com/apache/iceberg/issues/10245
### Apache Iceberg version
1.5.1 (latest release)
### Query engine
Flink
### Please describe the bug 🐞
ADLSFileIO has an AzureProperties object. When ADLS_SHARED_KEY_ACCOU
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582283795
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582282103
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582281653
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582281598
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582280246
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582266898
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
viirya commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582266774
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
}
kevinjqliu opened a new pull request, #667:
URL: https://github.com/apache/iceberg-python/pull/667
Resolves #594 (and part of #511)
TODO:
* Add doc in `api.md`
This PR creates a metadata table for "Metadata Log Entries", similar to [its
spark
equivalent](https://iceberg.ap
amogh-jahagirdar commented on code in PR #10244:
URL: https://github.com/apache/iceberg/pull/10244#discussion_r1582207142
##
api/src/main/java/org/apache/iceberg/Snapshot.java:
##
@@ -171,4 +171,29 @@ default Iterable removedDeleteFiles(FileIO io)
{
default Integer schemaId(
liurenjie1024 commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081512923
I think a `FileIO` trait would still be necessary for two reasons:
1. It's more familiar to iceberg community, which makes this easier to
maintain and align with other imp
marvinlanhenke commented on issue #359:
URL: https://github.com/apache/iceberg-rust/issues/359#issuecomment-2081512633
I also like to work on this one; since it kind of blocks #358.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to Git
marvinlanhenke opened a new issue, #359:
URL: https://github.com/apache/iceberg-rust/issues/359
## Problem:
1. The `partition_filter` is also required by the ExpressionEvaluator; thus
should be moved outside ManifestEvaluator; cached by spec_id in scan.rs and
then used by ExpressionEvalu
liurenjie1024 commented on code in PR #295:
URL: https://github.com/apache/iceberg-rust/pull/295#discussion_r1582163907
##
crates/iceberg/src/arrow/reader.rs:
##
@@ -186,4 +221,637 @@ impl ArrowReader {
Ok(ProjectionMask::leaves(parquet_schema, indices))
}
tustvold commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081496562
> if FileIO becomes a trait and we can use an existing implementation
I guess I was suggesting to just not have a FileIO trait, and just use
`ObjectStore` which is already
marvinlanhenke commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081495685
> I'm not seeing a concrete proposal on which to comment, but the broad
theme of moving closer to the actual object store APIs is IMO a good idea.
>
> That being said
tustvold commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081488506
I'm not seeing a concrete proposal on which to comment, but the broad theme
of moving closer to the actual object store APIs is IMO a good idea.
That being said, I feel I s
marvinlanhenke commented on PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2081488204
> Let's wait a moment to see what others think about the catalog snapshot
problem. cc @Xuanwo @Fokko @viirya @sdd PTAL
If it turns out, that the PR is okay (for now) - let
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582159023
##
crates/integrations/datafusion/src/physical_plan/scan.rs:
##
@@ -0,0 +1,123 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more c
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582158406
##
crates/integrations/datafusion/src/catalog.rs:
##
@@ -0,0 +1,94 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor l
liurenjie1024 commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081483673
> > I agree that we should refactor the Input/Output API for best
performance, but we should be careful not to expose opendal's api to end user.
>
> Yep, I have a plan
liurenjie1024 commented on PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2081482903
Let's wait a moment to see what others think about the catalog snapshot
problem. cc @Xuanwo @Fokko @viirya @sdd PTAL
--
This is an automated message from the Apache Git Service
liurenjie1024 commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582152848
##
crates/integrations/datafusion/src/physical_plan/scan.rs:
##
@@ -0,0 +1,123 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
marvinlanhenke commented on code in PR #347:
URL: https://github.com/apache/iceberg-rust/pull/347#discussion_r1582148013
##
crates/iceberg/src/scan.rs:
##
@@ -218,6 +225,25 @@ impl TableScan {
let mut manifest_entries =
iter(manifest.entries().iter().filter(|
liurenjie1024 commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582152303
##
crates/integrations/datafusion/src/physical_plan/scan.rs:
##
@@ -0,0 +1,123 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
liurenjie1024 commented on PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2081477394
> @liurenjie1024 @ZENOTME @viirya @simonvandel ...I just went ahead and
pushed the recent updates; PTAL
>
> ## unresolved / todo:
> * [ ] proper integration tests for t
liurenjie1024 commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582151715
##
crates/integrations/datafusion/src/catalog.rs:
##
@@ -0,0 +1,94 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor li
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582080221
##
crates/integrations/datafusion/src/catalog.rs:
##
@@ -0,0 +1,94 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor l
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582083814
##
crates/integrations/datafusion/src/physical_plan/scan.rs:
##
@@ -0,0 +1,123 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more c
marvinlanhenke commented on PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#issuecomment-2081431552
@liurenjie1024 @ZENOTME @viirya @simonvandel
...I just went ahead and pushed the recent updates; PTAL
## unresolved / todo:
- [ ] proper integration tests for table
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582080221
##
crates/integrations/datafusion/src/catalog.rs:
##
@@ -0,0 +1,94 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor l
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582080221
##
crates/integrations/datafusion/src/catalog.rs:
##
@@ -0,0 +1,94 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor l
marvinlanhenke commented on code in PR #349:
URL: https://github.com/apache/iceberg-rust/pull/349#discussion_r1582058099
##
crates/e2e_test/tests/append_data_file_test.rs:
##
@@ -0,0 +1,212 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
Xuanwo commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081390290
> I agree that we should refactor the Input/Output API for best performance,
but we should be careful not to expose opendal's api to end user.
Yep, I have a plan to refactor
marvinlanhenke commented on code in PR #324:
URL: https://github.com/apache/iceberg-rust/pull/324#discussion_r1582055651
##
crates/integrations/datafusion/src/catalog.rs:
##
@@ -0,0 +1,94 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor l
liurenjie1024 commented on issue #356:
URL: https://github.com/apache/iceberg-rust/issues/356#issuecomment-2081378252
Also cc @tustvold Since we had a related discussion here:
https://github.com/apache/iceberg-rust/issues/172
--
This is an automated message from the Apache Git Service.
T
liurenjie1024 commented on code in PR #355:
URL: https://github.com/apache/iceberg-rust/pull/355#discussion_r1582047316
##
website/src/install.md:
##
@@ -23,10 +23,11 @@
Cargo 1.75.0 or later is required to build.
-Add `iceberg` into `Cargo.toml` dependencies:
+Add `iceberg
65 matches
Mail list logo
