[
https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773789#comment-17773789
]
ASF GitHub Bot commented on HADOOP-18930:
-----------------------------------------
steveloughran commented on PR #6168:
URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1756082091
creation tests fail. propose: parameterize
```
[ERROR] Tests run: 16, Failures: 3, Errors: 0, Skipped: 4, Time elapsed:
25.039 s <<< FAILURE! - in
org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate
[ERROR]
testOverwriteNonEmptyDirectory(org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate)
Time elapsed: 5.549 s <<< FAILURE!
java.lang.AssertionError: write of file over dir succeeded
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testOverwriteNonEmptyDirectory(AbstractContractCreateTest.java:194)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testOverwriteNonEmptyDirectory(AbstractContractCreateTest.java:210)
[ERROR]
testOverwriteEmptyDirectory(org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate)
Time elapsed: 1.51 s <<< FAILURE!
java.lang.AssertionError: Should be a directory -but isn't:
S3AFileStatus{path=s3a://stevel-london/fork-0001/test/testOverwriteEmptyDirectory;
isDirectory=false; length=256; replication=1; blocksize=33554432;
modification_time=1696965045000; access_time=0; owner=stevel; group=stevel;
permission=rw-rw-rw-; isSymlink=false; hasAcl=false; isEncrypted=true;
isErasureCoded=false} isEmptyDirectory=FALSE
eTag="fba66d3273eac117107832558bc9a363"
versionId=0u_U.3vOkC7.7F0kQHv9fHuNAIRxAo3f
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertIsDirectory(ContractTestUtils.java:585)
at
org.apache.hadoop.fs.contract.ContractTestUtils.assertIsDirectory(ContractTestUtils.java:577)
at
org.apache.hadoop.fs.contract.AbstractFSContractTestBase.assertIsDirectory(AbstractFSContractTestBase.java:352)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testOverwriteEmptyDirectory(AbstractContractCreateTest.java:145)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testOverwriteEmptyDirectory(AbstractContractCreateTest.java:161)
[ERROR]
testCreateFileOverExistingFileNoOverwrite(org.apache.hadoop.fs.contract.s3a.ITestS3AContractCreate)
Time elapsed: 1.467 s <<< FAILURE!
java.lang.AssertionError: writing without overwrite unexpectedly succeeded
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testCreateFileOverExistingFileNoOverwrite(AbstractContractCreateTest.java:92)
at
org.apache.hadoop.fs.contract.AbstractContractCreateTest.testCreateFileOverExistingFileNoOverwrite(AbstractContractCreateTest.java:106)
)
+ most of ITestCreateFileCost.
```
> make fs.s3a.create.performance an option you can set for the entire bucket
> --------------------------------------------------------------------------
>
> Key: HADOOP-18930
> URL: https://issues.apache.org/jira/browse/HADOOP-18930
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Affects Versions: 3.3.9
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> make the fs.s3a.create.performance option something you can set everywhere,
> rather than just in an openFile() option or under a magic path.
> this improves performance on apps like iceberg where filenames are generated
> with UUIDs in them, so we know there are no overwrites
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
