BsoBird commented on issue #9327:
URL: https://github.com/apache/iceberg/issues/9327#issuecomment-1859649309
@RussellSpitzer
I have slightly modified HadoopTableOperations.commit(). What do you think
about this modification?
```
@Override
public void commit(TableMetadata base, TableMetadata metadata) {
Pair<Integer, TableMetadata> current = versionAndMetadata();
if (base != current.second()) {
throw new CommitFailedException("Cannot commit changes based on stale
table metadata");
}
if (base == metadata) {
LOG.info("Nothing to commit.");
return;
}
Preconditions.checkArgument(
base == null || base.location().equals(metadata.location()),
"Hadoop path-based tables cannot be relocated");
Preconditions.checkArgument(
!metadata.properties().containsKey(TableProperties.WRITE_METADATA_LOCATION),
"Hadoop path-based tables cannot relocate metadata");
int nowVersion = (current.first() != null ? current.first() : 0);
int nextVersion = nowVersion + 1;
boolean versionCommit = false;
try{
String codecName =
metadata.property(
TableProperties.METADATA_COMPRESSION,
TableProperties.METADATA_COMPRESSION_DEFAULT);
TableMetadataParser.Codec codec =
TableMetadataParser.Codec.fromName(codecName);
String fileExtension = TableMetadataParser.getFileExtension(codec);
Path tempMetadataFile = metadataPath(UUID.randomUUID().toString() +
fileExtension);
TableMetadataParser.write(metadata,
io().newOutputFile(tempMetadataFile.toString()));
Path finalMetadataFile = metadataFilePath(nextVersion, codec);
FileSystem fs = getFileSystem(tempMetadataFile, conf);
// this rename operation is the atomic commit operation
renameToFinal(fs, tempMetadataFile, finalMetadataFile, nextVersion);
LOG.info("Committed a new metadata file {}", finalMetadataFile);
// update the best-effort version pointer
writeVersionHint(nextVersion);
versionCommit = true;
//perhaps this step will throw exception
deleteRemovedMetadataFiles(base, metadata);
this.shouldRefresh = true;
}catch (Exception e){
if(versionCommit){
//try best roll back,if fail,user should roll back version by hem
self
writeVersionHint(nowVersion);
}
throw e;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
