wypoon commented on code in PR #12117:
URL: https://github.com/apache/iceberg/pull/12117#discussion_r1940285154
##########
spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkV2Filters.java:
##########
@@ -70,7 +70,8 @@
public class SparkV2Filters {
public static final Set<String> SUPPORTED_FUNCTIONS =
- ImmutableSet.of("years", "months", "days", "hours", "bucket",
"truncate");
+ ImmutableSet.of(
+ "year", "years", "month", "months", "day", "days", "hour", "hours",
"bucket", "truncate");
Review Comment:
@Fokko thanks for looking at this, and the explanation for why the plural
form was used.
There is no interference between the Iceberg functions and the Spark
built-in functions, because the Iceberg functions use the "system" namespace. I
added tests for the built-in functions to demonstrate this.
There is one thing that gave me pause, which is the comment in
[BaseCatalog::isFunctionNamespace](https://github.com/apache/iceberg/blob/ce2af527a8712fdf23aa9f5094b40c9e8b4056ce/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/BaseCatalog.java#L65-L71):
```
// Allow for empty namespace, as Spark's storage partitioned joins look
up
// the corresponding functions to generate transforms for partitioning
// with an empty namespace, such as `bucket`.
// Otherwise, use `system` namespace.
```
For this reason, I added some variants to `TestStoragePartitionJoins` to use
the singular forms of the transforms. Those tests pass too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
