pvary commented on PR #12979:
URL: https://github.com/apache/iceberg/pull/12979#issuecomment-2872020495

   > > Do we want to have scheduler before running the compaction? If we 
proceed on this road, then we will run the compaction after every commit. I 
think it would be better to have a `TriggerEvaluator` before the actual task, 
and run the compaction task after ever X commit, or after a given time period.
   > > I understand that this would mean more configuration, but we at least 
should expose few of the scheduling possibilities to accommodate different 
use-cases
   > 
   > @pvary Because RewriteDataFiles/Builder has scheduling related settings, 
and TableMaintenance has TriggerEvaluator built-in, such as 
`scheduleOnCommitCount`, `scheduleOnDataFileCount`, `scheduleOnInterval`, 
`scheduleOnDataFileSize`, these settings can meet the need to run the 
compaction task after every X commits or after a certain time.
   > 
   > So I have exposed the following configs:
   > 
   > `flink-maintenance.rewrite.schedule-on-commit-count` 
`flink-maintenance.rewrite.schedule-on-data-file-count` 
`flink-maintenance.rewrite.schedule-on-data-file-size` 
`flink-maintenance.rewrite.schedule-on-interval-second`
   > 
   > Do these settings meet the above requirements? If my understanding is 
incorrect., please feel free to point it out.
   > 
   > Thank you very much.
   
   I started to review again, but I decided against it for now to prevent 
requesting multiple changes.
   First I think we should agree on the way how to configure the table 
maintenance. What to expose, what not to expose. How to expose the config for 
the Java API, and how to expose those to the SQL API.
   Here, I'm looking at @Guosmilesmile, @mxm and @gyfora who are more near to 
the actual users than me.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to