Baymine opened a new issue, #66311: URL: https://github.com/apache/doris/issues/66311
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? On the FE, `ProfileManager.loadProfilesFromStorageIfFirstTime(boolean sync)` (the cold-load path that reads persisted query profiles from disk into memory) has two problems that can leak `profile-loader` threads: 1. **No in-flight guard.** Every call creates a brand-new `profile-loader` `Thread` with no check for a load already in progress. This method is reached from the periodic scheduler, so if a load has not yet finished a fresh loader thread is started on the next tick. 2. **`isProfileLoaded` is only set on success.** The flag `isProfileLoaded = true` is set *inside* the `try` block, *after* the batch loop completes. If the load throws, or the `profileIOExecutor` thread pool is saturated so the batch `future.get()` calls never return, the flag stays `false` forever. Combined with (1), a new `profile-loader` thread is then spawned on **every** scheduler tick — an unbounded thread leak that keeps growing until the FE is restarted. ```java Thread loadThread = new Thread(() -> { try { ... batch loop over profileIOExecutor futures ... isProfileLoadedLock.writeLock().lock(); try { this.isProfileLoaded = true; // only reached on success } finally { isProfileLoadedLock.writeLock().unlock(); } } catch (Exception e) { LOG.error("Failed to load query profile from storage", e); // flag left false } }); loadThread.setName("profile-loader"); loadThread.start(); ``` ### What You Expected? At most one `profile-loader` thread should run at a time, and a cold-load that fails or never completes should not cause a new loader thread to be spawned on every scheduler tick. ### How to Reproduce? Saturate the `profileIOExecutor` (or make `Profile.read`/`getOnStorageProfileInfos` block or throw) so `loadProfilesFromStorageIfFirstTime` never sets `isProfileLoaded = true`, then let the periodic scheduler keep calling it. The live `profile-loader` thread count grows without bound (observable via a thread dump / `Thread.getAllStackTraces`). ### Anything Else? Fix: add an `AtomicBoolean isProfileLoading` CAS guard so only one loader runs at a time (synchronous callers block until the in-flight load finishes), and mark the load finished in a `finally` block **even on failure** so a saturated/failed load does not respawn a loader every second. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
