xinyiZzz commented on PR #42421: URL: https://github.com/apache/doris/pull/42421#issuecomment-3440747952
> > 1. A query_id is randomly generated to replace t_query_plan_info.query_id. external query does not need to report anything to FE, so the query_id can be changed. > > Otherwise, multiple independent concurrent open tablet scanners have the same query_id. when one of the scanners ends, the other scanners will be canceled through FragmentMgr.cancel(query_id). > > 2. (query_id, fragment_id) is executed only on one BE, locks _pipeline_map. > > 3. External query (flink/spark read tablets) not need to report to FE. > > according to your describe，when multiple independent concurrent open tablet scanners ，must cause an error，why has this problem，I cannot understand it . > > https://doris-forum.org.cn/questions/D15j4/tong-guo-de-fang-shi-fang-wen-biao-ti-shi-bao-cuo In the Spark/Flink connector, `openScanner` executes to read a tablet from Doris. After the read, `closeScanner` executes `FragmentMgr.cancel(query_id)` to clean up the remaining state in Doris. Therefore, if multiple openScanners use the same query_id, the issue mentioned in the PR may occur. The actual error may occur randomly. because of the delay in executing `closeScanner -> FragmentMgr.cancel`, openScanners started at the same time may complete at the same time. Therefore, by the time the first completed scanner executes the cancel call, the other scanners have also completed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
