raghav-reglobe commented on PR #63192: URL: https://github.com/apache/doris/pull/63192#issuecomment-5066598695
Heads-up from running this patch in production against an Iceberg REST catalog (master-based build, 2026-07-22 base): the feature currently works only through the v1 file scanner. With `enable_file_scanner_v2 = true` (the master default — the session variable is also fuzz-randomized in tests), every read of a VARIANT-bearing parquet file fails at footer parse: ``` Metadata contains Thrift LogicalType that is not recognized ``` FileScannerV2 opens footers via the bundled arrow 17.0.0 parquet-cpp, which does not recognize the parquet VARIANT logical type, so the failure hits before any of this PR's read path is reached. Our mitigation is `SET GLOBAL enable_file_scanner_v2 = false` cluster-wide, which works but gives up the v2 scanner for everything. Since v2 is the default, it may be worth handling in this PR before merge — either a per-scan fallback to the v1 reader when the schema contains VARIANT, or teaching the v2 path the type (the thirdparty arrow pin is still 17.0.0, so footer parsing alone rules out v2 today). Happy to test either approach against our catalogs — we have VARIANT tables in the tens of millions of rows range. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
