lintingbin opened a new pull request, #4200:
URL: https://github.com/apache/amoro/pull/4200
## Summary
Closes #1503.
Adds INFO/WARN logs on the Mixed Hive Table partition operation paths
(create / drop / alter location). These operations are infrequent, so
logging them at INFO level makes after-the-fact diagnosis of missing
or mis-located partitions much easier with negligible runtime cost.
Logs are added at two layers:
- `HivePartitionUtil` — per-partition `LOG.info` on
`createPartitionIfAbsent`,
`dropPartition`, and `updatePartitionLocation`, plus matching `LOG.warn` on
the failure paths. The existing `alterPartition` log now reports both old
and new locations.
- `UpdateHiveFiles` / `ReplaceHivePartitions` — per-batch `LOG.info` for the
drop / create / alter sets emitted by `commitPartitionedTable`, with a
count and a sampled list of partitions (capped at 5, with `...` for the
rest) to avoid log explosion on large commits, plus matching `LOG.warn`
on commit failures.
Each log line carries the table identifier, partition values, location
(old → new on alter), and transaction id where available, so a single
log record is enough to identify the partition and trace it back to a
commit.
## Sample logs
Per-partition (single-partition path, `HivePartitionUtil`):
```
INFO o.a.a.h.utils.HivePartitionUtil - Creating Hive partition for table
catalog.db.tbl, partition values [2024-01-01], location
s3://bkt/db/tbl/dt=2024-01-01/.amoro_xact_42
INFO o.a.a.h.utils.HivePartitionUtil - Dropping Hive partition for table
catalog.db.tbl, partition values [2024-01-01], location
s3://bkt/db/tbl/dt=2024-01-01/.amoro_xact_41
INFO o.a.a.h.utils.HivePartitionUtil - Altering Hive partition location for
table catalog.db.tbl, partition dt=2024-01-01, location
s3://bkt/db/tbl/dt=2024-01-01/.amoro_xact_41 ->
s3://bkt/db/tbl/dt=2024-01-01/.amoro_xact_42
```
Per-batch (transaction commit path, `UpdateHiveFiles`):
```
INFO o.a.a.hive.op.UpdateHiveFiles - Creating 3 Hive partitions for table
catalog.db.tbl, txId 42, partitions [Partition(values: [2024-01-01], location:
s3://bkt/.../.amoro_xact_42), Partition(values: [2024-01-02], location:
s3://bkt/.../.amoro_xact_42), Partition(values: [2024-01-03], location:
s3://bkt/.../.amoro_xact_42)]
INFO o.a.a.hive.op.UpdateHiveFiles - Dropping 1 Hive partitions for table
catalog.db.tbl, txId 42, partitions [Partition(values: [2024-01-01], location:
s3://bkt/.../.amoro_xact_41)]
```
Failure path:
```
WARN o.a.a.h.utils.HivePartitionUtil - Failed to alter Hive partition
location for table catalog.db.tbl, partition dt=2024-01-01, location
s3://bkt/.../.amoro_xact_41 -> s3://bkt/.../.amoro_xact_42
org.apache.thrift.TException: ...
```
## Tests
No new unit tests; existing tests in the module continue to pass:
```
./mvnw test -pl amoro-format-mixed/amoro-mixed-hive \
-Dtest=TestRewritePartitions,TestOverwriteFiles,TestSyncHiveMeta
# Tests run: 76, Failures: 0, Errors: 0, Skipped: 18
```
`mvn spotless:check` and `mvn checkstyle:check` are clean for the
touched module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
