nsivabalan commented on PR #18132: URL: https://github.com/apache/hudi/pull/18132#issuecomment-4203607608
> for MOR reads: > > HoodieMergeOnReadRDD L89 -> where we broadcast the hadoop conf, lets inject the `hasLogicalTs` into it in the driver. > > and this eventually will get wired all the way to AbstractHoodieLogRecordReader.fs.getHadoopConf This piece of code will be executed in executor. We can rely on the boolean flag and decide whether to call repairSchema or not. hold off on this feedback. if (HoodieMergeOnReadRDD L89) is going to be invoked N no of times for N file groups in driver, then we might be parsing the schema N no of times unless we cache the value of `hasLogicalTs` at a table level. lets chat about this feedback f2f and take a call. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
