lintingbin opened a new pull request, #4180: URL: https://github.com/apache/amoro/pull/4180
### Search before asking - [x] I have searched in the [issues](https://github.com/apache/amoro/issues?q=is%3Aissue) and found no similar issues. ### What type of PR is this? - [x] Improvement - [ ] Bug Fix - [ ] Feature - [ ] Refactoring ### What does this PR do? Add an automatic restart mechanism for optimizers that die unexpectedly. #### Problem When an optimizer process crashes, the `OptimizerKeeper` detects the heartbeat timeout and removes the optimizer record from the `optimizer` table. However, the corresponding resource record in the `resource` table is **not cleaned up**, leaving an "orphaned" resource with no active optimizer. Currently there is no mechanism to detect this situation and restart the optimizer. #### Solution Extend the existing `OptimizerGroupKeeper` to periodically detect orphaned resources and automatically restart the optimizer: 1. **Orphaned resource detection**: In `OptimizerGroupKeeper.processTask()`, cross-check the `resource` table against the `optimizer` table to find resources without any active optimizer instances. 2. **Automatic restart**: For each orphaned resource, call `container.requestResource(resource)` to restart the optimizer process. 3. **Retry tracking**: Track the number of restart attempts per orphaned resource. If a restart exceeds the configurable maximum retries, clean up the orphaned resource from the database instead of retrying indefinitely. #### Configuration Two new configuration options are added (disabled by default): | Config Key | Default | Description | |---|---|---| | `optimizer.auto-restart-enabled` | `false` | Whether to enable automatic restart of unexpectedly down optimizers | | `optimizer.auto-restart-max-retries` | `5` | Maximum restart attempts per orphaned resource before cleanup | #### Bug fix Also fixes `ResourceMapper.selectResourcesByGroup` result mapping where property names did not match `Resource` class fields (`group`→`groupName`, `container`→`containerName`, `totalMemory`→`memoryMb`). ### Checklist - [x] I have added corresponding tests for my changes. - [x] This PR only changes one thing, and it is clear from the title. - [ ] I have documented my changes in the relevant documentation. Closes #3988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
