pythaac opened a new pull request, #10619:
URL: https://github.com/apache/gravitino/pull/10619
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): Support xxx"
- "[#233] fix: Check null before access result in xxx"
- "[MINOR] refactor: Fix typo in variable name"
- "[MINOR] docs: Fix typo in README"
- "[#255] test: Fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
1. Modified `LancePartitionStatisticStorage.close()` to explicitly iterate
over all cached `Dataset` instances and call `.close()` before calling
`cache.invalidateAll()` and `allocator.close()`.
2. Updated `testCloseReleasesCachedDatasetBeforeAllocator` to verify both
the execution order (`InOrder`) and the actual release of native Arrow memory
(`getAllocatedMemory() == 0`).
### Why are the changes needed?
The original issue correctly states that closing the `allocator` before the
`Dataset` causes a shutdown exception.
However, simply swapping the closing order is not enough. If we rely solely
on `invalidateAll()`, `dataset.close()` is never called, silently leaking
native Arrow memory and still causing `allocator.close()` to crash
(`IllegalStateException: Allocator closed with outstanding buffers allocated`).
Therefore, explicitly closing the datasets in a loop is strictly required to
safely release off-heap resources.
Fix: #10598
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- Executed `testCloseReleasesCachedDatasetBeforeAllocator` using Mockito
`spy()` with a real `RootAllocator` and `VarCharVector`.
- Verified that `Dataset.close()` strictly happens before
`allocator.close()`.
- Asserted that `allocator.getAllocatedMemory()` successfully drops to `0
bytes` after `storage.close()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
