morningman commented on code in PR #48723:
URL: https://github.com/apache/doris/pull/48723#discussion_r1986718971
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -294,6 +299,7 @@ public List<Split> getSplits(int numBackends) throws
UserException {
null,
PaimonSplit.PaimonSplitCreator.DEFAULT);
for (Split dorisSplit : dorisSplits) {
+ ((PaimonSplit)
dorisSplit).setSchemaFilePath(schemaFilePath);
Review Comment:
That could be issue if each split has a string to store schema path.
I suggest to only store the schema id in paimon split. and use a map to
store <id -> schema file path>
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -174,8 +174,10 @@ private void setPaimonParams(TFileRangeDesc rangeDesc,
PaimonSplit paimonSplit)
} else {
throw new RuntimeException("Unsupported file format: " +
fileFormat);
}
+ LocationPath schemaPath = new
LocationPath(paimonSplit.getSchemaFilePath(),
+ source.getCatalog().getProperties());
+
fileDesc.setSchemaFilePath(schemaPath.toStorageLocation().toString());
Review Comment:
See comment for `((PaimonSplit)
dorisSplit).setSchemaFilePath(schemaFilePath);`
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/paimon/source/PaimonScanNode.java:
##########
@@ -277,10 +279,13 @@ public List<Split> getSplits(int numBackends) throws
UserException {
splitStat.setType(SplitReadType.NATIVE);
splitStat.setRawFileConvertable(true);
List<RawFile> rawFiles = optRawFiles.get();
+ String tablePath =
source.getPaimonTable().options().get("path");
Review Comment:
This can be done outside the loop
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
