FrankChen021 commented on PR #20299: URL: https://github.com/apache/druid/pull/20299#issuecomment-6096051488
I think aboves findings from the AI should be addressed. Under the case that when a worker disappears after the snapshot is built, current code fallback to previous behaviour, that's true, but here we are going to solve the performance problem here right? and because the snapshot is built before each iteration, I think compared to the original code, it increases the probability that we will encounter above problem. 2nd, after this change, `getWorkersEligibleToRunTasks` can be called twice, it increases the overhead for a pending task scan that can find a worker. Instead of patching the current pending executor path, another way is to track the tasks of a worker when its announcement changes. we can do it in the `WorkerHolder#fullSync` and `WorkerHolder#deltaSync`, and update the `WorkerHolder#toImmutable` to return the computed result directly instead of computing in the fly as current implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
