BsoBird commented on code in PR #10623:
URL: https://github.com/apache/iceberg/pull/10623#discussion_r1669870050
##########
core/src/main/java/org/apache/iceberg/hadoop/HadoopTableOperations.java:
##########
@@ -159,18 +169,124 @@ public void commit(TableMetadata base, TableMetadata
metadata) {
int nextVersion = (current.first() != null ? current.first() : 0) + 1;
Path finalMetadataFile = metadataFilePath(nextVersion, codec);
FileSystem fs = getFileSystem(tempMetadataFile, conf);
+ boolean versionCommitSuccess = false;
+ boolean useObjectStore =
+ metadata.propertyAsBoolean(
+ TableProperties.OBJECT_STORE_ENABLED,
TableProperties.OBJECT_STORE_ENABLED_DEFAULT);
+ int previousVersionsMax =
+ metadata.propertyAsInt(
+ TableProperties.METADATA_PREVIOUS_VERSIONS_MAX,
+ TableProperties.METADATA_PREVIOUS_VERSIONS_MAX_DEFAULT);
+ // todo:Currently, if the user is using an object store, we assume that he
must be using the
+ // global locking service. But we should support add some other
conditions in future.
+ boolean supportGlobalLocking = useObjectStore;
+ try {
+ tryLock(tempMetadataFile, metadataRoot());
+ versionCommitSuccess =
+ commitNewVersion(
+ fs, tempMetadataFile, finalMetadataFile, nextVersion,
supportGlobalLocking);
+ if (!versionCommitSuccess) {
+ throw new CommitFailedException(
+ "Can not commit newMetaData because version [%s] has already been
committed. tempMetaData=[%s],finalMetaData=[%s].Are there other clients running
in parallel with the current task?",
+ nextVersion, tempMetadataFile, finalMetadataFile);
+ }
+ validate(supportGlobalLocking, previousVersionsMax, nextVersion, fs,
finalMetadataFile);
+ this.shouldRefresh = true;
+ LOG.info("Committed a new metadata file {}", finalMetadataFile);
+ // update the best-effort version pointer
+ writeVersionHint(fs, nextVersion);
+ deleteRemovedMetadataFiles(base, metadata);
+ } catch (CommitStateUnknownException e) {
+ this.shouldRefresh = true;
+ throw e;
+ } catch (Exception e) {
+ this.shouldRefresh = versionCommitSuccess;
+ if (!versionCommitSuccess) {
Review Comment:
If the commitNewVersion method succeeds, we will ignore any exceptions
caught after that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
