morrySnow commented on code in PR #30151:
URL: https://github.com/apache/doris/pull/30151#discussion_r1497010916
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionSet.java:
##########
@@ -1929,6 +1930,15 @@ private void initTableFunction() {
addTableFunctionWithCombinator(EXPLODE, Type.WILDCARD_DECIMAL,
Function.NullableMode.ALWAYS_NULLABLE,
Lists.newArrayList(new ArrayType(Type.WILDCARD_DECIMAL)),
false,
"_ZN5doris19DummyTableFunctions7explodeEPN9doris_udf15FunctionContextERKNS1_13CollectionValE");
+
+ initTableFunctionListWithCombinator(EXPLODE_MAP);
+ for (Type keyType : Type.getMapSubTypes()) {
+ for (Type valueType : Type.getMapSubTypes()) {
+ addTableFunctionWithCombinator(EXPLODE_MAP, new
StructType(keyType, valueType), Function.NullableMode.ALWAYS_NULLABLE,
+ Lists.newArrayList(new MapType(keyType, valueType)),
false,
+
"_ZN5doris19DummyTableFunctions7explodeEPN9doris_udf15FunctionContextERKNS1_13CollectionValE");
Review Comment:
why still need function symbol here?
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalGenerate.java:
##########
@@ -47,17 +47,25 @@ public class LogicalGenerate<CHILD_TYPE extends Plan>
extends LogicalUnary<CHILD
private final List<Function> generators;
private final List<Slot> generatorOutput;
+ // mapping with function.
+ private final List<List<String>> expandColumnAlias;
public LogicalGenerate(List<Function> generators, List<Slot>
generatorOutput, CHILD_TYPE child) {
- this(generators, generatorOutput, Optional.empty(), Optional.empty(),
child);
+ this(generators, generatorOutput, Lists.newArrayList(),
Optional.empty(), Optional.empty(), child);
Review Comment:
```suggestion
this(generators, generatorOutput, ImmutableList.of(),
Optional.empty(), Optional.empty(), child);
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java:
##########
@@ -624,8 +631,34 @@ protected boolean condition(Rule rule, Plan plan) {
Slot boundSlot = new
SlotReference(slot.getNameParts().get(1), generator.getDataType(),
generator.nullable(),
ImmutableList.of(slot.getNameParts().get(0)));
slotBuilder.add(boundSlot);
+ // the boundSlot may has two situation:
+ // 1. the expandColumnsAlias is not empty, we should
use make boundSlot expand to multi alias
+ // 2. the expandColumnsAlias is empty, we should use
origin boundSlot
+ if (generate.getExpandColumnAlias() != null
+ &&
!CollectionUtils.isEmpty(generate.getExpandColumnAlias().get(i))) {
+ // if the alias is not empty, we should bind it
with struct_element as child expr with alias
+ // struct_element(#expand_col#k, #k) as #k
+ // struct_element(#expand_col#v, #v) as #v
+ List<StructField> fields = ((StructType)
boundSlot.getDataType()).getFields();
+ for (int idx = 0; idx < fields.size(); ++idx) {
+ expandAlias.add(new Alias(new StructElement(
+ boundSlot, new
StringLiteral(fields.get(idx).getName())),
+
generate.getExpandColumnAlias().get(i).get(idx)));
+ }
+ }
+ }
+ LogicalGenerate ret = new LogicalGenerate<>(
+ boundFunctionGenerators, slotBuilder.build(),
generate.child());
+ if (expandAlias.size() > 0) {
+ // we need a project to deal with explode(map) to
struct with field alias
+ // project should contains: generator.child slot +
expandAlias
+ List<NamedExpression> allProjectSlots =
generate.child().getOutput().stream()
+ .map(NamedExpression.class::cast)
+ .collect(Collectors.toList());
+ allProjectSlots.addAll(expandAlias);
+ return new LogicalProject<>(allProjectSlots, ret);
Review Comment:
```suggestion
ret = new LogicalProject<>(allProjectSlots, ret);
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java:
##########
@@ -587,7 +593,7 @@ protected boolean condition(Rule rule, Plan plan) {
// we need to do cast before set operation, because we
maybe use these slot to do shuffle
// so, we must cast it before shuffle to get correct hash
code.
List<List<NamedExpression>> childrenProjections =
setOperation.collectChildrenProjections();
- ImmutableList.Builder<List<SlotReference>> childrenOutputs
= ImmutableList.builder();
+ Builder<List<SlotReference>> childrenOutputs =
ImmutableList.builder();
Review Comment:
do not refactor other code in this PR
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java:
##########
@@ -1021,14 +1021,21 @@ protected LogicalPlan withGenerate(LogicalPlan plan,
LateralViewContext ctx) {
return plan;
}
String generateName = ctx.tableName.getText();
- String columnName = ctx.columnName.getText();
+ // if later view explode map type, we need to add a project to convert
map to struct
+ String columnName = ctx.columnNames.get(0).getText();
+ List<String> expandColumnNames = Lists.newArrayList();
+ if (ctx.columnNames.size() > 1) {
+ columnName = "expand_cols";
Review Comment:
use column name generator to generate column name when connect context not
null.
```java
ConnectContext.get().getStatementContext().generateColumnName()
```
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/generator/ExplodeMapOuter.java:
##########
@@ -0,0 +1,80 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.nereids.trees.expressions.functions.generator;
+
+import org.apache.doris.catalog.FunctionSignature;
+import org.apache.doris.nereids.exceptions.AnalysisException;
+import org.apache.doris.nereids.trees.expressions.Expression;
+import org.apache.doris.nereids.trees.expressions.functions.AlwaysNullable;
+import org.apache.doris.nereids.trees.expressions.shape.UnaryExpression;
+import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor;
+import org.apache.doris.nereids.types.MapType;
+import org.apache.doris.nereids.types.StructField;
+import org.apache.doris.nereids.types.StructType;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
+
+import java.util.List;
+
+/**
+ * explode({"amory":1, "doris": 2}) generate two column and two lines with:
+ * key column: amory, doris
+ * value column: 1, 2
+ */
+public class ExplodeMapOuter extends TableGeneratingFunction implements
UnaryExpression, AlwaysNullable {
+
+ /**
+ * constructor with 1 argument.
+ */
+ public ExplodeMapOuter(Expression arg) {
+ super("explode_map_outer", arg);
+ }
+
+ /**
+ * withChildren.
+ */
+ @Override
+ public ExplodeMapOuter withChildren(List<Expression> children) {
+ Preconditions.checkArgument(children.size() == 1);
+ return new ExplodeMapOuter(children.get(0));
+ }
+
+ @Override
+ public void checkLegalityBeforeTypeCoercion() {
+ if (!(child().getDataType() instanceof MapType)) {
+ throw new AnalysisException("only support map type for explode_map
function but got "
+ + child().getDataType());
+ }
+ }
+
+ @Override
+ public List<FunctionSignature> getSignatures() {
+ return ImmutableList.of(
+ FunctionSignature.ret(new StructType(ImmutableList.of(
+ new StructField("col1", ((MapType)
child().getDataType()).getKeyType(), true, ""),
Review Comment:
create a function in structType like: `StructType
constructStructType(List<DataType> fieldTypes)` to generate default column name
struct type. and call this function here and in
`org.apache.doris.nereids.trees.expressions.literal.StructLiteral#computeDataType`
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/BindExpression.java:
##########
@@ -624,8 +631,34 @@ protected boolean condition(Rule rule, Plan plan) {
Slot boundSlot = new
SlotReference(slot.getNameParts().get(1), generator.getDataType(),
generator.nullable(),
ImmutableList.of(slot.getNameParts().get(0)));
slotBuilder.add(boundSlot);
+ // the boundSlot may has two situation:
+ // 1. the expandColumnsAlias is not empty, we should
use make boundSlot expand to multi alias
+ // 2. the expandColumnsAlias is empty, we should use
origin boundSlot
+ if (generate.getExpandColumnAlias() != null
+ &&
!CollectionUtils.isEmpty(generate.getExpandColumnAlias().get(i))) {
Review Comment:
`generate.getExpandColumnAlias().get(i)` could generate OutOfBound
exception, should check first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
