kangkaisen opened a new issue #2932: TupleIsNullPredicate lead BE core URL: https://github.com/apache/incubator-doris/issues/2932 **Describe the bug** ``` *** Aborted at 1581922536 (unix time) try "date -d @1581922536" if you are using GNU date *** PC: @ 0xf522c9 doris::RowDescriptor::tuple_is_nullable() *** SIGSEGV (@0x0) received by PID 82036 (TID 0x7f1b83496700) from PID 0; stack trace: *** @ 0x7f1c39a4d5d0 (unknown) @ 0xf522c9 doris::RowDescriptor::tuple_is_nullable() @ 0xd9e4fe doris::TupleIsNullPredicate::prepare() @ 0xd50c22 doris::Expr::prepare() @ 0xd50c22 doris::Expr::prepare() @ 0xd721b4 doris::ScalarFnCall::prepare() @ 0xd50c22 doris::Expr::prepare() @ 0xd721b4 doris::ScalarFnCall::prepare() @ 0xd5a37c doris::ExprContext::prepare() @ 0xd5141d doris::Expr::prepare() @ 0x158919c doris::DataStreamSender::prepare() @ 0x1019316 doris::PlanFragmentExecutor::prepare() @ 0xfa4ba4 doris::FragmentExecState::prepare() @ 0xfa7f85 doris::FragmentMgr::exec_plan_fragment() @ 0xfa8c72 doris::FragmentMgr::exec_plan_fragment() @ 0x1058e86 doris::PInternalServiceImpl<>::_exec_plan_fragment() @ 0x1058fa2 doris::PInternalServiceImpl<>::exec_plan_fragment() @ 0x12eb22c doris::PBackendService::CallMethod() ``` ``` #3 doris::RowDescriptor::tuple_is_nullable (this=this@entry=0xa9ae77678, tuple_idx=-1) at /home/kangkaisen/palo/be/src/runtime/descriptors.cpp:371 ``` **To Reproduce** Steps to reproduce the behavior: 1. ``` CREATE TABLE `afo_app_gpu_resource` ( `collect_day` date NOT NULL COMMENT "", `collect_min` char(20) NOT NULL COMMENT "", `app_id` varchar(50) NOT NULL COMMENT "", `group_name` varchar(100) NOT NULL COMMENT "", `queue_name` varchar(200) NOT NULL COMMENT "", `app_name` varchar(300) NULL COMMENT "", `app_type` varchar(50) NULL COMMENT "", `app_mode` varchar(20) NULL COMMENT "", `res_type` varchar(20) NULL , `res_schedule` double SUM NULL COMMENT "", `res_used` double SUM NULL COMMENT "", `res_util` double SUM NULL COMMENT "" ) ENGINE=OLAP AGGREGATE KEY(`collect_day`, `collect_min`, `app_id`, `group_name`, `queue_name`, `app_name`, `app_type`, `app_mode`, `res_type`) COMMENT "OLAP" PARTITION BY RANGE(`collect_day`) (PARTITION p20200215 VALUES LESS THAN ('2020-02-15'), PARTITION p20200216 VALUES LESS THAN ('2020-02-16'), PARTITION p20200217 VALUES LESS THAN ('2020-02-17')) DISTRIBUTED BY HASH(`group_name`) BUCKETS 30 PROPERTIES ( "replication_num" = "1" ); insert into afo_app_gpu_resource(`collect_day`, `collect_min`, `app_id`, `group_name`, `queue_name`, `app_name`, `app_type`, `app_mode`, `res_type`, res_schedule, res_used, res_util) values ('2020-02-16', '2020-02-16 00:00', 'application_1547876776012_46509900', 'root.serving.hadoop-peisong', 'root.serving.hadoop-peisong.p40prod', 'TFS','AFO-Serving', 'NULL', 'unknown', 2, 2,0); CREATE TABLE `yarn_groups_gpu_resource` ( `dt` date NULL COMMENT "日期", `scenes` varchar(20) NULL COMMENT "", `yarn_cluster` varchar(30) NULL COMMENT "", `tenant` varchar(50) NULL COMMENT "", `group_id` varchar(10) NULL COMMENT "", `group_code` varchar(200) NULL COMMENT "", `res_type` varchar(20) NULL COMMENT "", `res_min` int(11) SUM NULL COMMENT "" ) ENGINE=OLAP AGGREGATE KEY(`dt`, `scenes`, `yarn_cluster`, `tenant`, `group_id`, `group_code`, `res_type`) PARTITION BY RANGE(`dt`) (PARTITION p20200215 VALUES LESS THAN ('2020-02-15'), PARTITION p20200216 VALUES LESS THAN ('2020-02-16'), PARTITION p20200217 VALUES LESS THAN ('2020-02-17')) DISTRIBUTED BY HASH(`tenant`) BUCKETS 10 PROPERTIES ( "replication_num" = "1" ); insert into yarn_groups_gpu_resource(`dt`, `scenes`, `yarn_cluster`, `tenant`, `group_id`, `group_code`, `res_type`, res_min) values ('2020-02-16', 'Serving', 'gh_serving', 'hadoop-poistar', '3143', 'root.gh_serving.hadoop-poistar','gcores', 13); ``` 2 query **This query with date_format will fialed:** ``` select date_format(dt, '%Y%m%d') as dt from afo_app_gpu_resource as app left join ( select dt from yarn_groups_gpu_resource ) g on app.collect_day = g.dt group by dt; ``` **This query without date_format will success:** ``` select dt from afo_app_gpu_resource as app left join ( select dt from yarn_groups_gpu_resource ) g on app.collect_day = g.dt group by dt; ``` **Additional context** **The root reason is `tupleIds` in TupleIsNullPredicate is not right.** **The tupleId is 1 in TupleIsNullPredicate, but for AggregationNode, the tupleId is 3** ``` 17: tuple_is_null_pred (struct) = TTupleIsNullPredicate { 01: tuple_ids (list) = list<i32>[1] { [0] = 1, }, ``` ``` F0218 13:24:48.494895 31692 descriptors.cpp:370] Check failed: tuple_idx < _tuple_idx_nullable_map.size() (-1 vs. 1) RowDescriptor: tuple_desc_map: [Tuple(id=3 size=24 slots=[Slot(id=3 type=VARCHAR col=-1 offset=8 null=(offset=0 mask=80))] has_varlen_slots=1)] tuple_id_map: [-1, -1, -1, 0] tuple_is_nullable: [0] ``` The following is all TupleDescriptor: ``` TupleDescriptor{id=0, tbl=afo_app_gpu_resource, byte_size=0, is_materialized=true, slots=[SlotDescriptor{id=2, parent=0, col=colle ct_day, type=DATE, materialized=false, byteSize=0, byteOffset=-1, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0}]} TupleDescriptor{id=1, tbl=yarn_groups_gpu_resource, byte_size=0, is_materialized=true, slots=[SlotDescriptor{id=0, parent=1, col=d t, type=DATE, materialized=false, byteSize=0, byteOffset=-1, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0}]} TupleDescriptor{id=2, tbl=g, byte_size=0, is_materialized=false, slots=[SlotDescriptor{id=1, parent=2, col=dt, type=DATE, material ized=false, byteSize=0, byteOffset=-1, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0}]} TupleDescriptor{id=3, tbl=null, byte_size=0, is_materialized=true, slots=[SlotDescriptor{id=3, parent=3, col=null, type=VARCHAR(*) , materialized=false, byteSize=0, byteOffset=-1, nullIndicatorByte=0, nullIndicatorBit=0, slotIdx=0}]} ``` Partially fix this issue is improve `TupleIsNullPredicate::requiresNullWrapping`: ``` private static boolean requiresNullWrapping(Expr expr, Analyzer analyzer) { if (expr.isConstant()) { return false; } if (expr instanceof SlotRef) { SlotRef slotRef = (SlotRef) expr; Column column = slotRef.getDesc().getColumn(); if (column == null) { return true; } else { return !column.isAllowNull(); } } if (expr instanceof ArithmeticExpr || expr instanceof FunctionCallExpr || expr instanceof TimestampArithmeticExpr) { String functionName = expr.getFn().getFunctionName().getFunction(); return nonNullResultWithNullParamFunctions.contains(functionName); } return true; } ``` But which could not fix this issue completely，We should make the tupleIds in TupleIsNullPredicate is right.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
