limc5462 opened a new pull request, #4336:
URL: https://github.com/apache/streampark/pull/4336
<!--
Thank you for contributing to StreamPark! Please make sure that your code
changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
## Contribution Checklist
- If this is your first time, please read our contributor guidelines:
[Submit Code](https://streampark.apache.org/community/submit_guide/submit_code).
- Make sure that the pull request corresponds to a [GITHUB
issue](https://github.com/apache/streampark/issues).
- Name the pull request in the form "[Feature] Title of the pull request",
where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- If the PR is unfinished, add `[WIP]` in your PR title, e.g.,
`[WIP][Feature] Title of the pull request`.
-->
## What changes were proposed in this pull request
<img width="2777" height="1351" alt="image"
src="https://github.com/user-attachments/assets/3aa9dc63-8230-4a75-b2d8-878e37753f6b";
/>
This is a new issue that surfaced following my previous fix for another E2E
failure.
Currently, the E2E tests are consistently failing due to "No space left on
device" errors. This issue is primarily caused by the
`easimon/maximize-build-space` action currently used in the workflow. To
improve disk space utilization, this action consolidates multiple disk spaces
into a single LVM mount, but leaves only about 10GB of available space on the
root directory after mounting. Ideally, all operations should be executed
within the LVM volume. However, there are some hard-to-track, unexpected disk
write behaviors in our workflow that do not use the LVM. As a result, the 10GB
root directory space is quickly exhausted, leading to E2E failures.
I tried directly adjusting the reservation parameters of
`easimon/maximize-build-space` (for example, setting `root-reserve-mb: 30720`
in hopes of preserving 30GB for the root directory), but no matter how it was
adjusted, the root directory still only had 10GB left after the LVM mount was
completed. This might be a bug in the action itself, or perhaps the parameters
failed to take effect as expected in this specific environment.
Looking at the overall environment, the total physical disk space of the
Runner is actually sufficient, and we don't need to introduce a complex LVM
mounting mechanism to consolidate space. Therefore, this PR replaces the
space-clearing solution with `jlumbroso/free-disk-space`. By directly removing
large, pre-installed packages that are not actually needed in the environment
(such as `dotnet`, `android`, `haskell`, etc.), we can free up ample disk space
in a much simpler and more direct way, thereby completely resolving the disk
space shortage issue in the E2E tests.
<!-- REMOVE this line if no issue to close -->
<!--(For example: This pull request proposed to add checkstyle plugin).-->
## Brief change log
- Replaced `easimon/maximize-build-space` with
`jlumbroso/[email protected]`
in the GitHub Actions workflow.
- Configured the new action to remove unnecessary pre-installed environments
(`dotnet`, `android`, `haskell`, `codeql`, `docker-images`, `large-packages`
) to directly free up Runner disk space instead of using LVM.
## Verifying this change
This change is primarily verified by the E2E tests in the CI workflow.
Please note: The previous "No space left on device" error was a
deterministic issue that happened every time, and this fix completely resolves
it. However, during verification, I found that the E2E tests might occasionally
fail due to other unrelated, flaky issues. Therefore, when verifying this PR,
if you encounter a failure that is not related to disk space, simply re-running
the jobs will highly likely result in a successful pass.
## Does this pull request potentially affect one of the following parts
-Dependencies (does it add or upgrade a dependency): no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
