[
https://issues.apache.org/jira/browse/HADOOP-19877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080732#comment-18080732
]
ASF GitHub Bot commented on HADOOP-19877:
-----------------------------------------
ajfabbri commented on code in PR #8467:
URL: https://github.com/apache/hadoop/pull/8467#discussion_r3228420057
##########
.github/gha-tests/hadoop-aws-localstack-excludes.txt:
##########
@@ -0,0 +1,79 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+# Initial stab at excluding tests that won't run in our
+# CI container environment with Localstack mocks of AWS services.
+
+# TODO see if we can enable any of these...
+
+# tests that depend on public S3 buckets
+**/org/apache/hadoop/fs/s3a/scale/ITestS3AInputStreamPerformance.java
+**/org/apache/hadoop/fs/s3a/ITestS3ARequesterPays.java
+**/org/apache/hadoop/fs/s3a/s3guard/ITestS3GuardTool.java
+**/org/apache/hadoop/fs/s3a/tools/ITestMarkerTool.java
+**/ITestS3AAnalyticsAcceleratorStreamReading.java
+**/ITestS3AEndpointRegion.java
+
+
+# Tests requiring IAM roles / STS
+# We should be able to re-enable some of these. See:
+# https://docs.localstack.cloud/aws/services/sts/
+**/org/apache/hadoop/fs/s3a/auth/ITestAssumeRole.java
+**/org/apache/hadoop/fs/s3a/auth/delegation/ITestDelegatedMRJob.java
+**/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationInFilesystem.java
+**/org/apache/hadoop/fs/s3a/auth/delegation/ITestSessionDelegationTokens.java
+**/org/apache/hadoop/fs/s3a/ITestS3ATemporaryCredentials.java
+
+
+# failures that need to be investigated
+
+# Two methods fail: 1. testUpdateDeepDirectoryStructureNoChange():
+# AssertionFailedError: Files Skipped value 0 too below minimum 1 ==>
+# expected: <true> but was: <false>
+# 2. testUpdateDeepDirectoryStructureToRemote():
+# AssertionFailedError: Files Copied value 2 above maximum 1 ==> expected:
<true> but was: <false>
+**/org/apache/hadoop/fs/contract/s3a/ITestS3AContractDistCp.java
+
+# A number of failures with vectored read tests
+**/org/apache/hadoop/fs/contract/s3a/ITestS3AContractVectoredRead.java
+
+# Access key errors:
+# (test
case)->AbstractS3ATestBase.setup:111->AbstractFSContractTestBase.setup:197->AbstractFSContractTestBase.mkdirs:355
+# » AccessDenied s3a://hadoop-ci/job-00/test: getFileStatus on
+# s3a://hadoop-ci/job-00/test:
+# software.amazon.awssdk.services.s3.model.S3Exception: The AWS Access Key Id
you
+# provided does not exist in our records. (Service: S3, Status Code: 403
+**/org/apache/hadoop/fs/s3a/ITestS3APrefetchingCacheFiles.java
+**/org/apache/hadoop/fs/s3a/ITestS3AFailureHandling.java
Review Comment:
Test discussion: Wonder if the client is still trying to talk to AWS. Will
validate which endpoint is being called.
> run s3a integration tests in CI
> -------------------------------
>
> Key: HADOOP-19877
> URL: https://issues.apache.org/jira/browse/HADOOP-19877
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Priority: Major
> Labels: pull-request-available
>
> * Get a decent portion of hadoop-aws (s3a) integration tests running in CI.
> * Use localstack (OSS license) or other S3 emulator as a target.
> * Update docs as needed.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
