luoyuxia commented on issue #6104:
URL: https://github.com/apache/iceberg/issues/6104#issuecomment-1310193327
> the repeated compression process of full data
The compaction won't be repeated. I mean during one compaction, some files
are compacted to one file. In next compaction, it won't compact these files
again. I don't think it's a repeated compression process. From my side, it's
still a incremental process.
Back to your problem, actuall, there's a pr #2680 to use disk to avoid OOM.
And I think it'll also fix your problem.
I think you can try to
1: increase the memorag of TM.
2: do compaction more frequently , so that every compaction won't compact
much data, which may well relieve the OOM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
