(doris) 02/02: [fix](hudi-mtmv) Fix the issue that Hudi materialized views are not involved in transparent rewriting (#49513)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-hudi-mtmv
in repository https://gitbox.apache.org/repos/asf/doris.git

commit f09f82590db62ead9e13bf3253e4e49eac0e888b
Author: Tiewei Fang <fangtie...@selectdb.com>
AuthorDate: Thu Mar 27 10:28:42 2025 +0800

    [fix](hudi-mtmv) Fix the issue that Hudi materialized views are not 
involved in transparent rewriting (#49513)
    
    It's because when Hudi performs transparent rewriting, the timestamp of
    the base table obtained by `loadSnapshot` is inconsistent with the
    timestamp stored after partition refresh, which causes the comparison of
    `tableSnapshot` to fail, resulting in the materialized view not being
    hit.
    
    Currently, the logic of `loadSnapshot` to obtain the timestamp of the
    base table is a bit strange and doesn't quite meet expectations. Further
    in - depth research is needed on how to modify it.
    
    For now, in the `getTableSnapshot` function, simply return `0L`
    constantly, indicating that the `tableSnapshot` is always synchronized,
    to bypass this problem. This modification is consistent with the
    original expectation of manual refresh. We'll deal with this issue
    later.
---
 .../java/org/apache/doris/datasource/hive/HudiDlaTable.java    | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HudiDlaTable.java 
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HudiDlaTable.java
index 952d8e3311e..24963ec10c5 100644
--- 
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HudiDlaTable.java
+++ 
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/HudiDlaTable.java
@@ -79,15 +79,17 @@ public class HudiDlaTable extends HMSDlaTable {
     @Override
     public MTMVSnapshotIf getPartitionSnapshot(String partitionName, 
MTMVRefreshContext context,
             Optional<MvccSnapshot> snapshot) throws AnalysisException {
-        Map<String, Long> partitionNameToLastModifiedMap = 
getOrFetchHudiSnapshotCacheValue(
-                snapshot).getPartitionNameToLastModifiedMap();
-        return new 
MTMVTimestampSnapshot(partitionNameToLastModifiedMap.get(partitionName));
+        // Map<String, Long> partitionNameToLastModifiedMap = 
getOrFetchHudiSnapshotCacheValue(
+        //         snapshot).getPartitionNameToLastModifiedMap();
+        // return new 
MTMVTimestampSnapshot(partitionNameToLastModifiedMap.get(partitionName));
+        return new MTMVTimestampSnapshot(0L);
     }
 
     @Override
     public MTMVSnapshotIf getTableSnapshot(MTMVRefreshContext context, 
Optional<MvccSnapshot> snapshot)
             throws AnalysisException {
-        return new 
MTMVTimestampSnapshot(getOrFetchHudiSnapshotCacheValue(snapshot).getLastUpdateTimestamp());
+        // return new 
MTMVTimestampSnapshot(getOrFetchHudiSnapshotCacheValue(snapshot).getLastUpdateTimestamp());
+        return new MTMVTimestampSnapshot(0L);
     }
 
     @Override


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org