lk-1984 commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2705764904
So, if the user of the pyiceberg CLI would be able to give the catalog name
`hive`, then it would work like when using the pyiceberg as a library -
`load_catalog(name="hive")
melin opened a new issue, #12474:
URL: https://github.com/apache/iceberg/issues/12474
### Feature Request / Improvement
Many cloud customers expect spark sql to disable full table scanning by
default, prompting users to add partition conditions.
If forget to bring the table partiti
ZENOTME opened a new pull request, #1049:
URL: https://github.com/apache/iceberg-rust/pull/1049
## Which issue does this PR close?
- Closes #.
## What changes are included in this PR?
## Are these changes tested?
--
This is an automated message
Xuanwo commented on issue #1047:
URL: https://github.com/apache/iceberg-rust/issues/1047#issuecomment-2705672170
Hi, I believe that's related to
https://github.com/apache/iceberg-rust/issues/1036
--
This is an automated message from the Apache Git Service.
To respond to the messag
kevinjqliu opened a new pull request, #1773:
URL: https://github.com/apache/iceberg-python/pull/1773
https://github.com/apache/iceberg-python/blob/main/mkdocs/docs/how-to-release.md#update-the-github-template
1. Update the Github template
2. Update the integration tests
-
ZENOTME commented on PR #1048:
URL: https://github.com/apache/iceberg-rust/pull/1048#issuecomment-2705727508
cc @Xuanwo @liurenjie1024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specifi
ZENOTME opened a new pull request, #1048:
URL: https://github.com/apache/iceberg-rust/pull/1048
## Which issue does this PR close?
- Closes
## What changes are included in this PR?
change the input param type of commit to make it more general. I think the
ajantha-bhat commented on code in PR #12450:
URL: https://github.com/apache/iceberg/pull/12450#discussion_r1984563373
##
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/ComputePartitionStatsSparkAction.java:
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Soft
lk-1984 commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2705666108
https://github.com/apache/iceberg-python/blob/main/pyiceberg/utils/config.py#L152
This is is the reason why an empty config is returned.
```
def get_catal
nastra merged PR #12462:
URL: https://github.com/apache/iceberg/pull/12462
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap
ZENOTME commented on issue #1047:
URL: https://github.com/apache/iceberg-rust/issues/1047#issuecomment-2705657473
How do you think? cc @liurenjie1024 @Xuanwo @Fokko @sdd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
ZENOTME opened a new issue, #1047:
URL: https://github.com/apache/iceberg-rust/issues/1047
### Is your feature request related to a problem or challenge?
For now, our arrow reader accepts the FileScanTask and returns the
RecordBatchStream to the user. After #630, the reader can proces
akhilputhiry commented on code in PR #12406:
URL: https://github.com/apache/iceberg/pull/12406#discussion_r1984523655
##
core/src/main/java/org/apache/iceberg/rest/RESTCatalog.java:
##
@@ -55,7 +56,22 @@ public class RESTCatalog
public RESTCatalog() {
this(
Sess
Xuanwo commented on issue #1046:
URL: https://github.com/apache/iceberg-rust/issues/1046#issuecomment-2705558999
Not affected. Downstream users can choose whether to enable it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub a
aihuaxu opened a new issue, #12472:
URL: https://github.com/apache/iceberg/issues/12472
### Feature Request / Improvement
#12139 implements the Variant readers and #12323 implements the Variant
writers. The arrays currently are not supported yet. We need to add reading and
writing ar
sfc-gh-mrojas opened a new issue, #12469:
URL: https://github.com/apache/iceberg/issues/12469
### Apache Iceberg version
1.8.1 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
When using polaris catalog to write to Iceberg from Spark, it
szehon-ho commented on code in PR #12346:
URL: https://github.com/apache/iceberg/pull/12346#discussion_r1984255343
##
api/src/main/java/org/apache/iceberg/types/EdgeInterpolationAlgorithm.java:
##
@@ -0,0 +1,64 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under on
szehon-ho commented on PR #11583:
URL: https://github.com/apache/iceberg/pull/11583#issuecomment-2705116805
Thanks a lot for working on it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spe
zeroshade commented on code in PR #319:
URL: https://github.com/apache/iceberg-go/pull/319#discussion_r1984232026
##
table/locations.go:
##
@@ -0,0 +1,118 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTIC
github-actions[bot] commented on PR #12164:
URL: https://github.com/apache/iceberg/pull/12164#issuecomment-2705224684
This pull request has been marked as stale due to 30 days of inactivity. It
will be closed in 1 week if no further activity occurs. If you think that’s
incorrect or this pul
github-actions[bot] commented on PR #11513:
URL: https://github.com/apache/iceberg/pull/11513#issuecomment-2705224536
This pull request has been marked as stale due to 30 days of inactivity. It
will be closed in 1 week if no further activity occurs. If you think that’s
incorrect or this pul
Xuanwo commented on PR #966:
URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2704518519
cc @manuzhang @liurenjie1024 @Fokko @sdd, what do you think about merging
this PR and enabling the community to build on it?
--
This is an automated message from the Apache Git Servic
rdblue commented on code in PR #12457:
URL: https://github.com/apache/iceberg/pull/12457#discussion_r1983693589
##
core/src/test/java/org/apache/iceberg/RandomVariants.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contr
Xuanwo commented on PR #1010:
URL: https://github.com/apache/iceberg-rust/pull/1010#issuecomment-2704530737
Waiting for @liurenjie1024 for double check, I will merge this PR in three
days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
Fokko merged PR #320:
URL: https://github.com/apache/iceberg-go/pull/320
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac
adutra commented on code in PR #12406:
URL: https://github.com/apache/iceberg/pull/12406#discussion_r1983799256
##
core/src/main/java/org/apache/iceberg/rest/RESTCatalog.java:
##
@@ -55,7 +56,22 @@ public class RESTCatalog
public RESTCatalog() {
this(
SessionCat
Fokko commented on code in PR #319:
URL: https://github.com/apache/iceberg-go/pull/319#discussion_r1984140659
##
table/locations.go:
##
@@ -0,0 +1,118 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE fi
Fokko merged PR #322:
URL: https://github.com/apache/iceberg-go/pull/322
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apac
szehon-ho commented on code in PR #12430:
URL: https://github.com/apache/iceberg/pull/12430#discussion_r1984132225
##
docs/docs/hive.md:
##
@@ -91,20 +91,9 @@ Hive supports the following additional features with Hive
version 4.0.0 and abov
Hive 4 comes with `hive-iceberg` th
lk-1984 commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2704503736
Ok, there is something that overrides new_config:
```
(.venv) dev@foobar~ % pyiceberg list --catalog hive
/Users/dev/Projects/external/iceberg-python
{'catalog'
Fokko merged PR #1045:
URL: https://github.com/apache/iceberg-rust/pull/1045
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.
kevinjqliu commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2704448846
Just to clarify
1. You set `PYICEBERG_HOME` to `/home/pyiceberg` in the environment variables
I see above
```
PYICEBERG_HOME=/home/pyiceberg
```
2. `/h
huaxingao commented on PR #11583:
URL: https://github.com/apache/iceberg/pull/11583#issuecomment-2705029804
@szehon-ho I am working on my local. There are still a few test failures. I
will push the changes to upstream once I fix all the test failures.
--
This is an automated message from
sdd opened a new issue, #1044:
URL: https://github.com/apache/iceberg-rust/issues/1044
### Apache Iceberg Rust version
None
### Describe the bug
Consider the following single-column table of nullable strings:
```
a
-
null
"foo"
"bar"
```
Fokko merged PR #12335:
URL: https://github.com/apache/iceberg/pull/12335
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apa
demshar23 opened a new issue, #12471:
URL: https://github.com/apache/iceberg/issues/12471
### Query engine
Pyspark in AWS glue
### Question
I am trying to use the rewrite_table_path procedure in an AWS Glue Version 5
pyspark job or notebook, where I am setting the spark
Xuanwo commented on PR #890:
URL: https://github.com/apache/iceberg-rust/pull/890#issuecomment-2704526092
> Thanks @hackintoshrao for this pr. I'll convert it to draft first as it's
only used for showing the expected path, but there is no guarantee in iceberg
spec that it should generate su
rdblue commented on code in PR #12470:
URL: https://github.com/apache/iceberg/pull/12470#discussion_r1984040406
##
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetWriters.java:
##
@@ -134,43 +145,15 @@ private ParquetValueWriter newOption(Type fieldType
rdblue commented on code in PR #12470:
URL: https://github.com/apache/iceberg/pull/12470#discussion_r1984039899
##
flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/data/FlinkParquetReaders.java:
##
@@ -274,17 +272,11 @@ public Optional> visit(
public Optional> vis
rdblue commented on PR #12470:
URL: https://github.com/apache/iceberg/pull/12470#issuecomment-2704918354
#12463 was merged and the changes for Parquet were small, so I included them
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
rdblue commented on code in PR #12457:
URL: https://github.com/apache/iceberg/pull/12457#discussion_r1983699001
##
core/src/main/java/org/apache/iceberg/avro/ValueReaders.java:
##
@@ -141,6 +144,13 @@ public static ValueReader
decimalBytesReader(Schema schema) {
}
}
+
adutra commented on code in PR #12197:
URL: https://github.com/apache/iceberg/pull/12197#discussion_r1983959694
##
aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3V4RestSignerClient.java:
##
@@ -0,0 +1,130 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) unde
rdblue commented on PR #12470:
URL: https://github.com/apache/iceberg/pull/12470#issuecomment-2704775229
I'll also follow up with a PR for Parquet readers, but that depends on
changes in #12463.
--
This is an automated message from the Apache Git Service.
To respond to the message, please
szehon-ho commented on PR #11583:
URL: https://github.com/apache/iceberg/pull/11583#issuecomment-2704765194
While we wait for Spark 4.0 final release, I think we can just merge this
against latest RC if you manage to rebase before that, to unblock other work,
and can change once the final R
Fokko commented on code in PR #946:
URL: https://github.com/apache/iceberg-rust/pull/946#discussion_r1983923593
##
crates/iceberg/src/expr/visitors/strict_projection.rs:
##
@@ -0,0 +1,3231 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor
lk-1984 commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2704478405
Same:
```
(.venv) dev@foobar~ % pyiceberg list --catalog hive
URI missing, please provide using --uri, the config or environment variable
PYICEBERG_CATALOG__DEFAU
RussellSpitzer commented on issue #12467:
URL: https://github.com/apache/iceberg/issues/12467#issuecomment-2704525496
Figured it out.
The equality delete files are eliminated from scan planning because we
assume that the filters exclude any matching rows from being evaluated. This is
Xuanwo commented on PR #1010:
URL: https://github.com/apache/iceberg-rust/pull/1010#issuecomment-2704527530
I believe this PR should wait for
https://github.com/apache/iceberg-rust/pull/965.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
nastra merged PR #12464:
URL: https://github.com/apache/iceberg/pull/12464
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.ap
nastra closed pull request #12452: Core: Remove catalog name during rename
operation
URL: https://github.com/apache/iceberg/pull/12452
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
lk-1984 commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2704489427
With some debug print:
```
(.venv) dev@M-GM0J7P2WM1 ~ % pyiceberg list --catalog hive
/Users/dev/Projects/external/iceberg-python
{'catalog': {'hive': {'uri': '
rdblue commented on code in PR #12457:
URL: https://github.com/apache/iceberg/pull/12457#discussion_r1983693589
##
core/src/test/java/org/apache/iceberg/RandomVariants.java:
##
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contr
kevinjqliu commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2704471071
could you try `pyiceberg list --catalog hive`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
lk-1984 commented on issue #1771:
URL:
https://github.com/apache/iceberg-python/issues/1771#issuecomment-2704465656
The command was "list".
Btw, I pulled your repo, and built the thing in venv that has Python 3.11..
```
(.venv) dev@M-GM0J7P2WM1 iceberg-python % pyiceberg --h
kevinjqliu commented on issue #1763:
URL:
https://github.com/apache/iceberg-python/issues/1763#issuecomment-2704466782
I think there are a couple of different issues here.
1. `Error: Task timed out after 3.00 seconds` this seems to be related to
lambda configuration. see
https://stacko
lk-1984 opened a new issue, #1771:
URL: https://github.com/apache/iceberg-python/issues/1771
### Apache Iceberg version
None
### Please describe the bug 🐞
I have set `PYICEBERG_HOME="$(pwd)" as told in the docs.
https://py.iceberg.apache.org/api/
```
This
rcsmith27 commented on issue #1222:
URL:
https://github.com/apache/iceberg-python/issues/1222#issuecomment-2704083114
@akshayah3 Have you had a chance to look at this? I may be able to work on
it if not
--
This is an automated message from the Apache Git Service.
To respond to the messag
and124578963 opened a new issue, #12467:
URL: https://github.com/apache/iceberg/issues/12467
### Apache Iceberg version
1.8.1 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
## Description
When executing a sequence of deletes (posi
varpa89 opened a new issue, #12466:
URL: https://github.com/apache/iceberg/issues/12466
### Apache Iceberg version
1.7.1
### Query engine
Spark
### Please describe the bug 🐞
When we use SparkCatalog with Iceberg Rest Catalog and try to select from
metadata
nastra commented on code in PR #12464:
URL: https://github.com/apache/iceberg/pull/12464#discussion_r1983223453
##
data/src/main/java/org/apache/iceberg/data/PartitionStatsHandler.java:
##
@@ -121,37 +119,35 @@ public static Schema schema(StructType
unifiedPartitionType) {
Fokko opened a new pull request, #1770:
URL: https://github.com/apache/iceberg-python/pull/1770
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe,
ajantha-bhat commented on PR #12464:
URL: https://github.com/apache/iceberg/pull/12464#issuecomment-2703384149
@Fokko, @nastra: Can you please help in review and merge of this minor
change?
I can rebase https://github.com/apache/iceberg/pull/12451 after this PR is
merged.
--
This is
ajantha-bhat commented on code in PR #12451:
URL: https://github.com/apache/iceberg/pull/12451#discussion_r1983056061
##
api/src/main/java/org/apache/iceberg/actions/ComputePartitionStats.java:
##
@@ -0,0 +1,40 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under on
ajantha-bhat opened a new pull request, #12464:
URL: https://github.com/apache/iceberg/pull/12464
While adding the spark procedure, observed that puffin stats exposes
snapshot-id and partition stats exposes branch name.
https://github.com/apache/iceberg/pull/12451#discussion_r1980150585
Xuanwo commented on PR #966:
URL: https://github.com/apache/iceberg-rust/pull/966#issuecomment-2703268633
> Ah, and the bindings are fairly minimal I see? I was going to try to
experiment with them + cuDF but it seems more work is necessary. In any case
it's neat that we can get a C/potenti
lidavidm commented on code in PR #966:
URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1982977723
##
bindings/c/CMakeLists.txt:
##
@@ -0,0 +1,61 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the
waruto210 commented on code in PR #966:
URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1982951213
##
bindings/c/src/specs.rs:
##
@@ -0,0 +1,153 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See th
Xuanwo commented on code in PR #966:
URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1982941064
##
bindings/c/CMakeLists.txt:
##
@@ -0,0 +1,61 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NO
waruto210 commented on code in PR #966:
URL: https://github.com/apache/iceberg-rust/pull/966#discussion_r1982908977
##
bindings/c/src/specs.rs:
##
@@ -0,0 +1,153 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See th
69 matches
Mail list logo
