learner1212 opened a new issue, #18535: URL: https://github.com/apache/doris/issues/18535
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description routine load now not support modify columns, so when my table schema changed, we must stop current routine load job and submit a new one. but doris will reset offset to lastest or earilest by property.kafka_default_offsets setting, that may cause data repeat or data lost. the only way we avoid this problem is appoint kafka_partitions and kafka_offsets settings, and this also bring new problems: 1. the configuration is difficult when we have a large number of partitions; 2. when we increase partiton in topic, we will loss data in the new partitions. in summary， we need to consume kafka topic with a exist consumer group an do not reset the offset. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
