huaxingao commented on code in PR #15994: URL: https://github.com/apache/iceberg/pull/15994#discussion_r3941739338
########## api/src/main/java/org/apache/iceberg/udf/UdfTypes.java: ########## @@ -0,0 +1,326 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.iceberg.udf; + +import java.util.Arrays; +import java.util.List; +import java.util.Objects; +import java.util.stream.Collectors; +import org.apache.iceberg.relocated.com.google.common.base.Preconditions; +import org.apache.iceberg.relocated.com.google.common.collect.ImmutableList; +import org.apache.iceberg.types.Types; + +/** + * Concrete implementations of {@link UdfType}: {@link PrimitiveType} for primitive and + * semi-structured types and {@link ListType}, {@link MapType}, {@link StructType} for nested types. + * {@link NestedField} represents a named field inside a {@link StructType}. + */ +public class UdfTypes { + + private UdfTypes() {} + + /** + * A UDF primitive or semi-structured type, encoded as a type string (e.g., {@code int}, {@code + * string}, {@code decimal(9, 2)}, {@code variant}). + * + * <p>The type string must be a recognized Iceberg primitive or semi-structured type as understood + * by {@link Types#fromTypeName(String)}. The input is canonicalized to Iceberg's standard form + * (lowercase, normalized whitespace), so {@code PrimitiveType.of("INT")} and {@code + * PrimitiveType.of("Decimal( 9 , 2 )")} produce {@code int} and {@code decimal(9, 2)} + * respectively. + */ + public static final class PrimitiveType implements UdfType { + + private final String typeString; + + public static PrimitiveType of(String typeString) { + Preconditions.checkArgument(typeString != null, "Invalid primitive type: null"); + // Validate against Iceberg's primitive/semi-structured type vocabulary and use the parsed + // type's canonical toString() so callers don't have to worry about casing or whitespace. + String canonical = Types.fromTypeName(typeString).toString(); Review Comment: @szehon-ho Good catch. It is not just geography. `DecimalType.toString()` is `decimal(%d, %d)`, so `PrimitiveType.of("decimal(9,2)")` stores `decimal(9, 2)`. The spec's own example type violates the rule that forbids spaces. I think the spec needs to change here rather than the code. The spec says type strings follow Iceberg's type JSON, but `SchemaParser` writes primitives as `type.toString()`, which has a space for decimal. Spaces can also be part of the value itself. `TestTypes` covers `geography(srid: 4269)` and `geography(projjson: TestIdentifier, karney)`, where the space belongs to the CRS, so stripping it would change the CRS. The rule is not satisfiable for types Iceberg already supports. My proposal: remove the no spaces rule for type strings, so a type string is just Iceberg's type string. PrimitiveType.of then stays as it is. The same rule also applies to `definition-id`, and `definition-id` is built from the parameter types. So a parameter of type `geography(srid: 4269)` would put a space in the definition-id as well. We can either drop the rule there too, or say that a geography with a spaced CRS cannot be a UDF parameter. I lean toward dropping it, since `definition-id` only appears inside JSON bodies and never in a URL, so a space does not affect matching. Nothing in this PR builds definition ids, so I would rather decide that in the next PR, the one that adds `UdfDefinition`. Does that work for you? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
