[
https://issues.apache.org/jira/browse/IMPALA-12743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-12743.
-------------------------------------
Fix Version/s: Impala 4.4.0
Resolution: Fixed
> Incremental stats are filtered out by HMS due to HIVE-27114
> -----------------------------------------------------------
>
> Key: IMPALA-12743
> URL: https://issues.apache.org/jira/browse/IMPALA-12743
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 4.4.0
>
>
> HIVE-27114 adds a new property,
> "hive.metastore.partitions.parameters.exclude.pattern", which defaults to
> "impala_intermediate_stats_chunk%". This excludes the incremental stats in
> the parameters returned to HMS client.
> Impala is the only client that consumes this parameter. We should set it to
> an empty string. Otherwise, incremental stats can't be loaded when deploying
> Impala on newer versions of Hive. The error is
> {noformat}
> W0122 20:30:53.508314 17348 HdfsPartition.java:1444] Failed to set partition
> stats for table functional.alltypesagg partition year=2010/month=1/day=1
> Java exception follows:
> org.apache.impala.common.ImpalaRuntimeException: Missing stats chunk: 0
> at
> org.apache.impala.catalog.PartitionStatsUtil.partStatsBytesFromParameters(PartitionStatsUtil.java:108)
> at
> org.apache.impala.catalog.HdfsPartition$Builder.extractAndCompressPartStats(HdfsPartition.java:1441)
> at
> org.apache.impala.catalog.HdfsPartition$Builder.setMsPartition(HdfsPartition.java:1417)
> at
> org.apache.impala.catalog.HdfsTable.createOrUpdatePartitionBuilder(HdfsTable.java:959)
> at
> org.apache.impala.catalog.HdfsTable.createPartitionBuilder(HdfsTable.java:948)
> at
> org.apache.impala.catalog.HdfsTable.loadAllPartitions(HdfsTable.java:729)
> at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1299)
> at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1192)
> at org.apache.impala.catalog.TableLoader.load(TableLoader.java:144)
> at
> org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:248)
> at
> org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:245)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
