zhouhoo opened a new issue, #15555: URL: https://github.com/apache/doris/issues/15555
### Discussed in https://github.com/apache/doris/discussions/15536 <div type='discussions-op-text'> <sup>Originally posted by **zhouhoo** December 30, 2022</sup> **_when I use doris 1.1.5 routine load to consume kafka data, the error reported as follow:_** `2022-12-30 19:05:17,159 WARN (Routine load task scheduler|37) [KafkaRoutineLoadJob.hasMoreDataToConsume():700] failed to get latest partition offset. errCode = 2, detailMessage = Failed to get latest offsets of kafka topic: gxy_lzx. error: errCode = 2, detailMessage = failed to get latest offsets: [failed to get latest offset for partition: 0, err: Local: Bad message format] org.apache.doris.common.LoadException: errCode = 2, detailMessage = Failed to get latest offsets of kafka topic: gxy_lzx. error: errCode = 2, detailMessage = failed to get latest offsets: [failed to get latest offset for partition: 0, err: Local: Bad message format] at org.apache.doris.common.util.KafkaUtil.getLatestOffsets(KafkaUtil.java:228) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.load.routineload.KafkaRoutineLoadJob.hasMoreDataToConsume(KafkaRoutineLoadJob.java:694) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.load.routineload.KafkaTaskInfo.hasMoreDataToConsume(KafkaTaskInfo.java:118) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadTaskScheduler.scheduleOneTask(RoutineLoadTaskScheduler.java:131) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadTaskScheduler.process(RoutineLoadTaskScheduler.java:110) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.load.routineload.RoutineLoadTaskScheduler.runAfterCatalogReady(RoutineLoadTaskScheduler.java:83) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.0-SNAPSHOT] at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.0-SNAPSHOT] ;` **the table is as fellow:** `CREATE TABLE `gxy_lzx_test` ( `A1` varchar(255) NOT NULL COMMENT "", `A2` varchar(255) NULL COMMENT "", `A3` varchar(255) NULL COMMENT "", `A4` varchar(255) NULL COMMENT "", `A5` varchar(255) NULL COMMENT "这是第五个字段", `A6` varchar(32) NULL COMMENT "" ) ENGINE=OLAP UNIQUE KEY(`A1`) COMMENT "测试" DISTRIBUTED BY HASH(`A1`) BUCKETS 1 PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "in_memory" = "false", "storage_format" = "V2" );` **the routine load is created as fellow:** ``CREATE ROUTINE LOAD gxy_test ON gxy_lzx_test COLUMNS TERMINATED BY "," PROPERTIES ( "max_batch_interval" = "10", "max_batch_rows" = "300000", "max_batch_size" = "209715200", "strict_mode" = "false", "format" = "json", "strip_outer_array" = "true", "json_root" = "$.data" ) FROM KAFKA ( "kafka_broker_list" = "10.30.162.53:9092", "kafka_topic" = "gxy_lzx" );`` **the data in kafka 'gxy_lzx' topic is just as fellow:** `{"data":[{"A1":"000","A2":"\u00003\u00002\u00002","A3":"311","A4":"43","A5":"33222","A6":"31"}],"database":"practice","es":1672323278000,"id":174,"isDdl":false,"mysqlType":{"A1":"varchar(255)","A2":"varchar(255)","A3":"varchar(255)","A4":"varchar(255)","A5":"varchar(255)","A6":"varchar(32)"},"old":[{"A2":"\u00003\u00002"}],"pkNames":["A1"],"sql":"","sqlType":{"A1":12,"A2":12,"A3":12,"A4":12,"A5":12,"A6":12},"table":"gxy_lzx_test","ts":1672323278630,"type":"UPDATE"} ` I had done all the steps in the official webset says, but did not work. </div> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
