Quanlong Huang created IMPALA-12743:
---------------------------------------
Summary: Incremental stats are filtered out by HMS due to
HIVE-27114
Key: IMPALA-12743
URL: https://issues.apache.org/jira/browse/IMPALA-12743
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Assignee: Quanlong Huang
HIVE-27114 adds a new property,
"hive.metastore.partitions.parameters.exclude.pattern", which defaults to
"impala_intermediate_stats_chunk%". This excludes the incremental stats in the
parameters returned to HMS client.
Impala is the only client that consumes this parameter. We should set it to an
empty string. Otherwise, incremental stats can't be loaded when deploying
Impala on newer versions of Hive. The error is
{noformat}
W0122 20:30:53.508314 17348 HdfsPartition.java:1444] Failed to set partition
stats for table functional.alltypesagg partition year=2010/month=1/day=1
Java exception follows:
org.apache.impala.common.ImpalaRuntimeException: Missing stats chunk: 0
at
org.apache.impala.catalog.PartitionStatsUtil.partStatsBytesFromParameters(PartitionStatsUtil.java:108)
at
org.apache.impala.catalog.HdfsPartition$Builder.extractAndCompressPartStats(HdfsPartition.java:1441)
at
org.apache.impala.catalog.HdfsPartition$Builder.setMsPartition(HdfsPartition.java:1417)
at
org.apache.impala.catalog.HdfsTable.createOrUpdatePartitionBuilder(HdfsTable.java:959)
at
org.apache.impala.catalog.HdfsTable.createPartitionBuilder(HdfsTable.java:948)
at
org.apache.impala.catalog.HdfsTable.loadAllPartitions(HdfsTable.java:729)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1299)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1192)
at org.apache.impala.catalog.TableLoader.load(TableLoader.java:144)
at
org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:248)
at
org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:245)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750){noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
