[
https://issues.apache.org/jira/browse/HADOOP-13449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709115#comment-15709115
]
Aaron Fabbri commented on HADOOP-13449:
---------------------------------------
Finished a run of the S3A integration tests. I see that fixing the
MockS3Client factory is not as simple as my last comment, as you use it for the
DynamoDBMetadataStore unit test. We can revisit this here or on HADOOP-13589.
Here are the integration test failures I see when I configure the
DynamoDBMetadataStore via core-site.xml:
{code}
Failed tests:
ITestS3AContractDelete>AbstractContractDeleteTest.testDeleteNonEmptyDirNonRecursive:78->Assert.fail:88
non recursive delete should have raised an exception, but completed with exit
code true
ITestS3AContractDelete>AbstractContractDeleteTest.testDeleteNonEmptyDirRecursive:94->AbstractFSContractTestBase.assertDeleted:349->Assert.fail:88
Deleted file: unexpectedly found
s3a://fabbri-dev/test/testDeleteNonEmptyDirNonRecursive as
S3AFileStatus{path=s3a://fabbri-dev/test/testDeleteNonEmptyDirNonRecursive;
isDirectory=true; modification_time=0; access_time=0; owner=fabbri;
group=fabbri; permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=false
ITestS3AConfiguration.testUsernameFromUGI:481 owner in
S3AFileStatus{path=s3a://fabbri-dev/; isDirectory=true; modification_time=0;
access_time=0; owner=fabbri; group=fabbri; permission=rwxrwxrwx;
isSymlink=false} isEmptyDirectory=false expected:<[alice]> but was:<[fabbri]>
ITestS3AFileOperationCost.testFakeDirectoryDeletion:254->Assert.assertEquals:555->Assert.assertEquals:118->Assert.failNotEquals:743->Assert.fail:88
after rename(srcFilePath, destFilePath): directories_created expected:<1> but
was:<0>
ITestS3AFileOperationCost.testCostOfGetFileStatusOnNonEmptyDir:139->Assert.fail:88
FileStatus says directory isempty:
S3AFileStatus{path=s3a://fabbri-dev/test/empty; isDirectory=true;
modification_time=0; access_time=0; owner=fabbri; group=fabbri;
permission=rwxrwxrwx; isSymlink=false} isEmptyDirectory=true
ls s3a://fabbri-dev/test/empty [00]
S3AFileStatus{path=s3a://fabbri-dev/test/empty/simple.txt; isDirectory=false;
length=0; replication=1; blocksize=33554432; modification_time=1480497225005;
access_time=0; owner=fabbri; group=fabbri; permission=rw-rw-rw-;
isSymlink=false} isEmptyDirectory=false
Tests in error:
ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRmEmptyRootDirNonRecursive:116
» PathIO
ITestS3AFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryAsNonExistentDirectory:1038->FileContextMainOperationsBaseTest.testRenameDirectoryAsNonExistentDirectory:1052->FileContextMainOperationsBaseTest.rename:1197
» IO
ITestS3AAWSCredentialsProvider.testAnonymousProvider:133 » AWSServiceIO
initia...
ITestS3AAWSCredentialsProvider.testBadCredentials:102->createFailingFS:76 »
AWSServiceIO
ITestS3ACredentialsInURL.testInstantiateFromURL:86 » AWSClientIO initializing
...
ITestS3AFileSystemContract>FileSystemContractBaseTest.testWriteReadAndDeleteOneBlock:266->FileSystemContractBaseTest.writeReadAndDelete:285->FileSystemContractBaseTest.writeAndRead:815
» FileAlreadyExists
ITestS3AFileSystemContract>FileSystemContractBaseTest.testRenameToDirWithSamePrefixAllowed:656->FileSystemContractBaseTest.rename:512
» AWSServiceIO
{code}
> S3Guard: Implement DynamoDBMetadataStore.
> -----------------------------------------
>
> Key: HADOOP-13449
> URL: https://issues.apache.org/jira/browse/HADOOP-13449
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Mingliang Liu
> Attachments: HADOOP-13449-HADOOP-13345.000.patch,
> HADOOP-13449-HADOOP-13345.001.patch, HADOOP-13449-HADOOP-13345.002.patch,
> HADOOP-13449-HADOOP-13345.003.patch, HADOOP-13449-HADOOP-13345.004.patch,
> HADOOP-13449-HADOOP-13345.005.patch, HADOOP-13449-HADOOP-13345.006.patch,
> HADOOP-13449-HADOOP-13345.007.patch, HADOOP-13449-HADOOP-13345.008.patch,
> HADOOP-13449-HADOOP-13345.009.patch, HADOOP-13449-HADOOP-13345.010.patch
>
>
> Provide an implementation of the metadata store backed by DynamoDB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
