Sai Hemanth Gantasala created IMPALA-12758:
----------------------------------------------
Summary: Event Processor is ignoring the prev_id while reloading
the existing partitions
Key: IMPALA-12758
URL: https://issues.apache.org/jira/browse/IMPALA-12758
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Sai Hemanth Gantasala
Assignee: Sai Hemanth Gantasala
Partitioned table Insert events consumed by the event processor reloads the
partitions to update file metadata. Currently while reloading the filemetadata,
'prev_id' of the old partitions is being ignored in the partition builder
{code:java}
HdfsPartition oldPartition = entry.getValue();
HdfsPartition.Builder partBuilder = createPartitionBuilder(
hmsPartition.getSd(), hmsPartition, permissionCache); {code}
As a result 'prev_id' of the partBuilder will always be -1, so when the
catalogDelta is sent from the state store to impala demons, because prev_id is
not valid, impalads will not know whether to invalidate the current partition
and then request the new partition information.
This might lead to data correctness issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
