amogh-jahagirdar closed pull request #10755: API: Add RemoveUnusedSpecs in Table
URL: https://github.com/apache/iceberg/pull/10755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment
amogh-jahagirdar commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2450573869
@advancedxy I updated the PR to your branch
https://github.com/advancedxy/iceberg/pull/1 in case there was still agreement
on adding all of this metadata cleanup as part of sna
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2367497991
@amogh-jahagirdar Updated, It would be great if you can take another look at
this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please lo
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2362587553
> yes I think we should send the new update type as part of this PR.that
would require some parser changes as well for the update type. Let me know if
any help is needed, I can try t
amogh-jahagirdar commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2362521114
Sorry for missing following up on this @advancedxy yes I think we should
send the new update type as part of this PR.that would require some parser
changes as well for the upda
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2357370672
Gently ping @amogh-jahagirdar @rdblue @RussellSpitzer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2316575214
@amogh-jahagirdar @RussellSpitzer and @rdblue since #10846 was already
merged, let's restart this PR.
Do you think we should add the `RemovePartitionSpec` support in the REST
ca
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2264359958
> @advancedxy Just FYI, I've raised #10846 for the rest spec changes, I'll
also start a mailing list discussion on the new update type.
Thank you for taking the effort, that's
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1701095912
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1102,42 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700782191
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1102,42 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
amogh-jahagirdar commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2263817132
@advancedxy Just FYI, I've raised #10846 for the rest spec changes, I'll
also start a mailing list discussion on the new update type.
--
This is an automated message from th
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700397090
##
core/src/main/java/org/apache/iceberg/MetadataUpdate.java:
##
@@ -165,6 +165,25 @@ public void applyTo(TableMetadata.Builder metadataBuilder)
{
}
}
+
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700394019
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1102,42 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700345811
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1102,42 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700315717
##
core/src/main/java/org/apache/iceberg/MetadataUpdate.java:
##
@@ -165,6 +165,25 @@ public void applyTo(TableMetadata.Builder metadataBuilder)
{
}
}
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700307772
##
core/src/main/java/org/apache/iceberg/BaseMetadataMaintenance.java:
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * o
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700302549
##
core/src/main/java/org/apache/iceberg/MetadataUpdate.java:
##
@@ -165,6 +165,25 @@ public void applyTo(TableMetadata.Builder metadataBuilder)
{
}
}
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700301085
##
core/src/main/java/org/apache/iceberg/BaseTransaction.java:
##
@@ -688,9 +688,9 @@ public UpdateProperties updateProperties() {
}
@Override
-
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700293560
##
core/src/main/java/org/apache/iceberg/BaseMetadataMaintenance.java:
##
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+
advancedxy closed pull request #10755: API: Add RemoveUnusedSpecs in Table
URL: https://github.com/apache/iceberg/pull/10755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1700268241
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1102,42 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697812946
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1446,6 +1480,12 @@ public TableMetadata build() {
"Cannot set metadata location with
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697811542
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697807013
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1425,6 +1460,7 @@ private boolean hasChanges() {
|| (discardChanges && !changes.isEm
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697706041
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697706041
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697317660
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1425,6 +1460,7 @@ private boolean hasChanges() {
|| (discardChanges && !change
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697317660
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1425,6 +1460,7 @@ private boolean hasChanges() {
|| (discardChanges && !change
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697297712
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697294897
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more cont
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697281086
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1446,6 +1480,12 @@ public TableMetadata build() {
"Cannot set metadata location with cha
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697279627
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1425,6 +1460,7 @@ private boolean hasChanges() {
|| (discardChanges && !changes.isEmpty(
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697269413
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,20 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.build()
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697266984
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697257962
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,20 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.build();
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697255880
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or mor
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697254875
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or mor
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697255336
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or mor
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697200471
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1425,6 +1460,7 @@ private boolean hasChanges() {
|| (discardChanges && !changes.isEm
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1697161958
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more cont
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696950601
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696945706
##
core/src/test/java/org/apache/iceberg/TestRemoveUnusedSpecs.java:
##
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696223028
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696133057
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696102245
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1425,6 +1460,7 @@ private boolean hasChanges() {
|| (discardChanges && !changes.isEmpty(
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696101726
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
}
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696100886
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
}
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696099907
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
}
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696099533
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,24 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.build();
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696097946
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or mor
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696095743
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or mor
rdblue commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696094932
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more cont
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1696070676
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1695462421
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,24 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.b
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693879617
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693525020
##
core/src/test/java/org/apache/iceberg/TestRemoveUnusedSpecs.java:
##
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693522333
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693521937
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693521273
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,24 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.bui
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693509902
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693504833
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693505221
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693351417
##
core/src/main/java/org/apache/iceberg/BaseRemoveUnusedSpecs.java:
##
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ *
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693349501
##
core/src/main/java/org/apache/iceberg/BaseMetadataTable.java:
##
@@ -192,6 +192,11 @@ public List statisticsFiles() {
return ImmutableList.of();
}
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693343653
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or m
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1693343653
##
api/src/main/java/org/apache/iceberg/RemoveUnusedSpecs.java:
##
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or m
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2252785064
Close and re-open to trigger the CI.
Also gently ping @amogh-jahagirdar @RussellSpitzer to take another look.
--
This is an automated message from the Apache Git Service.
To
advancedxy closed pull request #10755: API: Add RemoveUnusedSpecs in Table
URL: https://github.com/apache/iceberg/pull/10755
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1690700326
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1690698977
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1690697748
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1690422091
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
RussellSpitzer commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1690420193
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -1102,6 +1121,22 @@ public Builder setDefaultPartitionSpec(int specId) {
return this;
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1689817577
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,24 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.build()
advancedxy commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1689069772
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,12 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.build()
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2246811037
> I don't think I'd go with a general SetPartitionSpecs update, I think I'd
have a RemovePartitionSpec, and the TableMetadata builder APIs to remove a
given spec (which will have va
amogh-jahagirdar commented on code in PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#discussion_r1688408490
##
core/src/main/java/org/apache/iceberg/TableMetadata.java:
##
@@ -597,6 +597,12 @@ public TableMetadata replaceProperties(Map
rawProperties) {
.b
advancedxy commented on PR #10755:
URL: https://github.com/apache/iceberg/pull/10755#issuecomment-2245286738
@amogh-jahagirdar @RussellSpitzer @aokolnychyi @szehon-ho it would be great
if you guys could take a look at this.
This PR is raised based the previous discussion in
https://
78 matches
Mail list logo