wmoustafa commented on PR #9830: URL: https://github.com/apache/iceberg/pull/9830#issuecomment-2067798780
> > However, if the materialized view is stale, the method simply returns to allow SparkCatalog's loadView to run. In turn, loadView returns the metadata for the virtual view itself, triggering the usual Spark view logic that computes the result set based on the current state of the base tables. > > 1/ was wondering if auto-refresh of MV on staleness detection should be an opt-in feature ? 2/ Any ideas / plans for incremental refresh ? These are very good questions. To me looks like if there is an external process that guarantees the freshness, then the current implementation still holds. Manual `REFRESH` will boil down to no-op, and `isFresh` will always return `true`. For (2): We have not discussed incremental refresh plans in the Iceberg community, but [there is some relevant work here](https://www.slideshare.net/walaa_eldin_moustafa/incremental-view-maintenance-with-coral-dbt-and-iceberg). You can review some of the test cases [here](https://github.com/linkedin/coral/blob/0d5dd3f300f48e48cd2404a49dbb799d7f4ce190/coral-incremental/src/test/java/com/linkedin/coral/incremental/RelToIncrementalSqlConverterTest.java#L28). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
