This is an automated email from the ASF dual-hosted git repository. aswinshakil pushed a commit to branch HDDS-10239-container-reconciliation in repository https://gitbox.apache.org/repos/asf/ozone.git
commit 0a53c73de21306469aa9b2f3ad56b88fa6e7c8d9 Merge: 9a445ed01eb 5e273a48da9 Author: Aswin Shakil Balasubramanian <[email protected]> AuthorDate: Mon Jun 9 14:48:37 2025 +0530 Merge branch 'master' of https://github.com/apache/ozone into HDDS-10239-container-reconciliation Commits: 80 commits 5e273a48da HDDS-12977. Fail build on dependency problems (#8574) 5081ba23b4 HDDS-13034. Refactor DirectoryDeletingService to use ReclaimableDirFilter and ReclaimableKeyFilter (#8546) e936e4deb1 HDDS-12134. Implement Snapshot Cache lock for OM Bootstrap (#8474) 31d13de52a HDDS-13165. [Docs] Python client developer guide. (#8556) 9e6955ee4f HDDS-13205. Bump common-custom-user-data-maven-extension to 2.0.3 (#8581) 750b629b8b HDDS-13203. Bump Bouncy Castle to 1.81 (#8580) ba5177ec13 HDDS-13202. Bump build-helper-maven-plugin to 3.6.1 (#8579) 07ee5dd531 HDDS-13204. Bump awssdk to 2.31.59 (#8582) e1964f2eae HDDS-13201. Bump jersey2 to 2.47 (#8578) 81295a53cb HDDS-13013. [Snapshot] Add metrics and tests for snapshot operations. (#8436) b3d75ab801 HDDS-12976. Clean up unused dependencies (#8521) e0f08b2471 HDDS-13179. rename-generated-config fails on re-compile without clean (#8569) f388317a6c HDDS-12554. Support callback on completed reconfiguration (#8391) c13a3fed9e HDDS-13154 Link more Grafana dashboard json files to the Observability user doc (#8533) 2a761f79bc HDDS-11967. [Docs]DistCP Integration in Kerberized environment. (#8531) 81fc4c4f85 HDDS-12550. Use DatanodeID instead of UUID in NodeManager CommandQueue. (#8560) 2360af4c5b HDDS-13169. Intermittent failure in testSnapshotOperationsNotBlockedDuringCompaction (#8553) f19789dc88 HDDS-13170. Reclaimable filter should always reclaim entries when buckets and volumes have already been deleted (#8551) 315ef204d8 HDDS-13175. Leftover reference to OM-specific trash implementation (#8563) 902e715221 HDDS-13159. Refactor KeyManagerImpl for getting deleted subdirectories and deleted subFiles (#8538) 46a93d0150 HDDS-12817. Addendum rename ecIndex to replicaIndex in chunkinfo output (#8552) 19b9b9c936 HDDS-13166. Set pipeline ID in BlockExistenceVerifier to avoid cached pipeline with different node (#8549) b3ff67c4d5 HDDS-13068. Validate Container Balancer move timeout and replication timeout configs (#8490) 7a7b9a83c0 HDDS-13139. Introduce bucket layout flag in freon rk command (#8539) 3c25e7d634 HDDS-12595. Add verifier for container replica states (#8422) 6d592206a2 HDDS-13104. Move auditparser acceptance test under debug (#8527) 8e8c432bb7 HDDS-13071. Documentation for Container Replica Debugger Tool (#8485) 0e8c8d4d44 HDDS-13158. Bump junit to 5.13.0 (#8537) 8e552b468c HDDS-13157. Bump exec-maven-plugin to 3.5.1 (#8534) 168f6901fa HDDS-13155. Bump jline to 3.30.4 (#8535) cc1e4d13ad HDDS-13156. Bump awssdk to 2.31.54 (#8536) 3bfb7affaf HDDS-13136. KeyDeleting Service should not run for already deep cleaned snapshots (#8525) 006e691245 HDDS-12503. Compact snapshot DB before evicting a snapshot out of cache (#8141) 568b2286b8 HDDS-13067. Container Balancer delete commands should not be sent with an expiration time in the past (#8491) 53673c564d HDDS-11244. OmPurgeDirectoriesRequest should clean up File and Directory tables of AOS for deleted snapshot directories (#8509) 07f48686c5 HDDS-13099. ozone admin datanode list ignores --json flag when --id filter is used (#8500) 08c0ab84ba HDDS-13075. Fix default value in description of container placement policy configs (#8511) 58c87a862d HDDS-12177. Set runtime scope where missing (#8513) 10c470d800 HDDS-12817. Add EC block index in the ozone debug replicas chunk-info (#8515) 7027ab7e91 HDDS-13124. Respect config hdds.datanode.use.datanode.hostname when reading from datanode (#8518) b8b226c3c8 HDDS-12928. datanode min free space configuration (#8388) fd3d70c2a1 HDDS-13026. KeyDeletingService should also delete RenameEntries (#8447) 4c1c6cf630 HDDS-12714. Create acceptance test framework for debug and repair tools (#8510) fff80fc2d5 HDDS-13118. Remove duplicate mockito-core dependency from hdds-test-utils (#8508) 10d5555c49 HDDS-13115. Bump awssdk to 2.31.50 (#8505) 360d13907a HDDS-13017. Fix warnings due to non-test scoped test dependencies (#8479) 1db1cca78c HDDS-13116. Bump jline to 3.30.3 (#8504) 322ca93b4a HDDS-13025. Refactor KeyDeletingService to use ReclaimableKeyFilter (#8450) 988b447924 HDDS-5287. Document S3 ACL classes (#8501) 64bb29de1f HDDS-12777. Use module-specific name for generated config files (#8475) 54ed115888 HDDS-9210. Update snapshot chain restore test to incorporate snapshot delete. (#8484) 87dfa5ac2d HDDS-13014. Improve PrometheusMetricsSink#normalizeName performance (#8438) 7cdc865259 HDDS-13100. ozone admin datanode list --json should output a newline at the end (#8499) 9cc4194083 HDDS-13089. [snapshot] Add an integration test to verify snapshotted data can be read by S3 SDK client (#8495) cb9867bcd3 HDDS-13065. Refactor SnapshotCache to return AutoCloseSupplier instead of ReferenceCounted (#8473) a88ff710dd HDDS-10979. Support STANDARD_IA S3 storage class to accept EC replication config (#8399) 6ec8f85e59 HDDS-13080. Improve delete metrics to show number of timeout DN command from SCM (#8497) 3bb8858598 HDDS-12378. Change default hdds.scm.safemode.min.datanode to 3 (#8331) 0171befb3d HDDS-13073. Set pipeline ID in checksums verifier to avoid cached pipeline with different node (#8480) 5c7726a5ce HDDS-11539. OzoneClientCache `@PreDestroy` is never called (#8493) a8ed19bb23 HDDS-13031. Implement a Flat Lock resource in OzoneManagerLock (#8446) e9e8b303f9 HDDS-12935. Support unsigned chunked upload with STREAMING-UNSIGNED-PAYLOAD-TRAILER (#8366) 75902681be HDDS-13079. Improve logging in DN for delete operation. (#8489) 435fe7e241 HDDS-12870. Fix listObjects corner cases (#8307) eb5dabd4d3 HDDS-12926. Remove *.tmp.* exclusion in DU (#8486) eeb98c7921 HDDS-13030. Snapshot Purge should unset deep cleaning flag for next 2 snapshots in the chain (#8451) 6bf121e664 HDDS-13032. Support proper S3OwnerId representation (#8478) 5d1b43d44f HDDS-13076. Refactor OzoneManagerLock class to rename Resource class to LeveledResource (#8482) bafe6d9e44 HDDS-13064. [snapshot] Add test coverage for SnapshotUtils.isBlockLocationInfoSame() (#8476) 7035846377 HDDS-13040. Add user doc highlighting the difference between Ozone ACL and S3 ACL. (#8457) 1825cdf605 HDDS-13049. Deprecate VolumeName & BucketName in OmKeyPurgeRequest and prevent Key version purge on Block Deletion Failure (#8463) 211c76cd9f HDDS-13060. Change NodeManager.addDatanodeCommand(..) to use DatanodeID (#8471) f4102380df HDDS-13061. Add test for key ACL operations without permission (#8472) d1a2f48c8a HDDS-13057. Increment block delete processed transaction counts regardless of log level (#8466) 0cc6fcc4b0 HDDS-13043. Replace != with assertNotEquals in TestSCMContainerPlacementRackAware (#8470) e1c779abe7 HDDS-13051. Use DatanodeID in server-scm. (#8465) 35e1126bff HDDS-13042. [snapshot] Add future proofing test cases for unsupported file system API (#8458) 619c05d1ac HDDS-13008. Exclude same SST files when calculating full snapdiff (#8423) 21b49d32e1 HDDS-12965. Fix warnings about "used undeclared" dependencies (#8468) 8136119256 HDDS-13048. Create new module for Recon integration tests (#8464) Conflicts: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/NodeManager.java .github/workflows/ci.yml | 3 + .github/workflows/intermittent-test-check.yml | 15 +- .mvn/extensions.xml | 2 +- .../apache/hadoop/hdds/scm/XceiverClientGrpc.java | 18 +- hadoop-hdds/common/pom.xml | 69 +-- .../org/apache/hadoop/hdds/HddsConfigKeys.java | 2 +- .../hadoop/hdds/conf/OzoneConfiguration.java | 60 +-- .../hadoop/hdds/utils/BackgroundService.java | 43 +- .../org/apache/hadoop/ozone/OzoneConfigKeys.java | 2 +- .../common/src/main/resources/ozone-default.xml | 34 +- .../conf/TestGeneratedConfigurationOverwrite.java | 4 +- .../hadoop/hdds/protocol/MockDatanodeDetails.java | 31 +- hadoop-hdds/config/pom.xml | 3 + .../hadoop/hdds/conf/ConfigFileGenerator.java | 14 +- hadoop-hdds/container-service/pom.xml | 26 +- .../apache/hadoop/ozone/HddsDatanodeService.java | 2 + .../common/statemachine/DatanodeConfiguration.java | 49 +- .../commandhandler/DeleteBlocksCommandHandler.java | 32 +- .../protocol/commands/CommandForDatanode.java | 6 +- .../statemachine/TestDatanodeConfiguration.java | 25 +- .../common/volume/TestReservedVolumeSpace.java | 9 +- .../replication/TestReplicationSupervisor.java | 4 +- .../content/concept/StorageContainerManager.md | 4 +- .../content/concept/StorageContainerManager.zh.md | 2 +- .../content/design/dn-min-space-configuration.md | 108 ++++ hadoop-hdds/docs/content/feature/Observability.md | 2 +- hadoop-hdds/docs/content/integration/DistCp.md | 75 +++ hadoop-hdds/docs/content/interface/Ofs.md | 2 +- hadoop-hdds/docs/content/interface/Ofs.zh.md | 2 +- hadoop-hdds/docs/content/interface/Python.md | 239 +++++++++ hadoop-hdds/docs/content/recipe/Boto3Tutorial.md | 123 +++++ hadoop-hdds/docs/content/recipe/PyArrowTutorial.md | 141 ++++++ .../content/recipe/PythonRequestsOzoneHttpFS.md | 170 +++++++ hadoop-hdds/docs/content/security/SecurityAcls.md | 13 + .../docs/content/tools/ContainerLogParser.md | 251 ++++++++++ hadoop-hdds/erasurecode/pom.xml | 5 + hadoop-hdds/framework/pom.xml | 53 +- .../hadoop/hdds/conf/ReconfigurableBase.java | 202 ++++++++ .../hdds/conf/ReconfigurationChangeCallback.java | 31 +- .../hadoop/hdds/conf/ReconfigurationHandler.java | 54 +- .../java/org/apache/hadoop/hdds/fs/DUFactory.java | 3 +- .../hdds/utils/PrometheusMetricsSinkUtil.java | 54 +- .../org/apache/hadoop/hdds/utils/db/DBStore.java | 18 + .../org/apache/hadoop/hdds/utils/db/RDBStore.java | 16 + .../apache/hadoop/hdds/utils/db/TestRDBStore.java | 17 + hadoop-hdds/interface-client/pom.xml | 10 +- hadoop-hdds/interface-server/pom.xml | 6 + hadoop-hdds/managed-rocksdb/pom.xml | 8 + hadoop-hdds/rocks-native/pom.xml | 9 +- hadoop-hdds/rocksdb-checkpoint-differ/pom.xml | 25 +- .../org/apache/ozone/rocksdb/util/RdbUtil.java | 19 + hadoop-hdds/server-scm/pom.xml | 19 +- .../hadoop/hdds/scm/block/DeletedBlockLogImpl.java | 34 +- .../hdds/scm/block/SCMBlockDeletingService.java | 2 +- .../SCMDeletedBlockTransactionStatusManager.java | 111 ++--- .../scm/block/ScmBlockDeletingServiceMetrics.java | 27 +- .../balancer/AbstractFindTargetGreedy.java | 8 +- .../scm/container/balancer/ContainerBalancer.java | 19 + .../scm/container/balancer/FindSourceGreedy.java | 8 +- .../hdds/scm/container/balancer/MoveManager.java | 88 ++-- .../ContainerPlacementPolicyFactory.java | 3 +- .../container/replication/ReplicationManager.java | 2 +- .../apache/hadoop/hdds/scm/node/CommandQueue.java | 31 +- .../hadoop/hdds/scm/node/DatanodeUsageInfo.java | 5 + .../hadoop/hdds/scm/node/DeadNodeHandler.java | 2 +- .../apache/hadoop/hdds/scm/node/NodeManager.java | 16 +- .../hadoop/hdds/scm/node/SCMNodeManager.java | 25 +- .../hdds/scm/pipeline/PipelineManagerImpl.java | 27 +- .../hdds/scm/safemode/DataNodeSafeModeRule.java | 7 +- .../scm/server/SCMDatanodeHeartbeatDispatcher.java | 14 +- .../hdds/scm/server/StorageContainerManager.java | 2 + .../org/apache/hadoop/hdds/scm/HddsTestUtils.java | 2 +- .../hadoop/hdds/scm/block/TestDeletedBlockLog.java | 40 +- .../scm/block/TestSCMBlockDeletingService.java | 13 +- .../TestSCMDeleteBlocksCommandStatusManager.java | 26 +- .../hadoop/hdds/scm/container/MockNodeManager.java | 23 +- .../hdds/scm/container/SimpleMockNodeManager.java | 7 +- .../container/TestCloseContainerEventHandler.java | 6 +- .../container/balancer/TestContainerBalancer.java | 19 + .../scm/container/balancer/TestMoveManager.java | 51 ++ .../TestSCMContainerPlacementRackAware.java | 50 +- .../TestReconcileContainerEventHandler.java | 18 +- .../replication/TestReplicationManager.java | 26 +- .../TestReplicationManagerScenarios.java | 11 +- .../hadoop/hdds/scm/node/TestCommandQueue.java | 56 +-- .../hadoop/hdds/scm/node/TestDeadNodeHandler.java | 4 +- .../hadoop/hdds/scm/node/TestSCMNodeManager.java | 45 +- .../hdds/scm/pipeline/TestPipelineManagerImpl.java | 23 +- .../scm/pipeline/TestPipelinePlacementFactory.java | 21 +- .../safemode/TestHealthyPipelineSafeModeRule.java | 1 + .../hdds/scm/safemode/TestSCMSafeModeManager.java | 1 + .../server/TestSCMDatanodeHeartbeatDispatcher.java | 4 +- hadoop-hdds/test-utils/pom.xml | 6 +- hadoop-ozone/cli-admin/pom.xml | 1 + .../hdds/scm/cli/datanode/ListInfoSubcommand.java | 11 +- .../scm/cli/datanode/TestListInfoSubcommand.java | 83 ++++ hadoop-ozone/cli-shell/pom.xml | 15 +- .../hadoop/ozone/shell/ReplicationOptions.java | 2 +- .../ozone/shell/keys/ChecksumKeyHandler.java | 2 +- hadoop-ozone/client/pom.xml | 33 +- .../hadoop/ozone/client}/OzoneClientUtils.java | 5 +- .../hadoop/ozone/client}/TestOzoneClientUtils.java | 4 +- hadoop-ozone/common/pom.xml | 18 +- .../org/apache/hadoop/ozone/om/OMConfigKeys.java | 20 + .../hadoop/ozone/om/S3SecretLockedManager.java | 2 +- .../hadoop/ozone/om/helpers/SnapshotInfo.java | 4 +- .../hadoop/ozone/om/lock/IOzoneManagerLock.java | 108 +++- .../hadoop/ozone/om/lock/OmReadOnlyLock.java | 11 +- .../hadoop/ozone/om/lock/OzoneManagerLock.java | 408 ++++++++------- .../hadoop/ozone/om/lock/OzoneManagerLockUtil.java | 12 +- .../request/validation/RegisterValidator.java | 2 +- .../ozone/om/helpers/TestOmSnapshotInfo.java | 8 +- .../hadoop/ozone/om/lock/TestKeyPathLock.java | 20 +- .../hadoop/ozone/om/lock/TestOzoneManagerLock.java | 189 +++++-- hadoop-ozone/csi/pom.xml | 82 ++-- hadoop-ozone/datanode/pom.xml | 10 +- hadoop-ozone/dev-support/checks/junit.sh | 4 +- hadoop-ozone/dev-support/checks/unit.sh | 2 +- hadoop-ozone/dist/pom.xml | 20 + .../dist/src/main/compose/common/ec-test.sh | 2 + hadoop-ozone/dist/src/main/compose/ozone/test.sh | 2 - .../src/main/compose/ozonescripts/docker-config | 1 + .../dist/src/main/compose/ozonesecure-ha/.env | 1 + .../main/compose/ozonesecure-ha/debug-tools.yaml | 162 ++++++ .../test.sh => ozonesecure-ha/test-debug-tools.sh} | 53 +- .../src/main/k8s/definitions/ozone/config.yaml | 1 + .../examples/getting-started/config-configmap.yaml | 1 + .../k8s/examples/minikube/config-configmap.yaml | 1 + .../k8s/examples/ozone-dev/config-configmap.yaml | 1 + .../k8s/examples/ozone-ha/config-configmap.yaml | 1 + .../main/k8s/examples/ozone/config-configmap.yaml | 1 + hadoop-ozone/dist/src/main/license/bin/LICENSE.txt | 4 - hadoop-ozone/dist/src/main/license/jar-report.txt | 5 - .../{auditparser => debug}/auditparser.robot | 6 +- .../smoketest/debug/ozone-debug-tests-ec3-2.robot | 9 +- .../smoketest/debug/ozone-debug-tests-ec6-3.robot | 5 +- .../main/smoketest/debug/ozone-debug-tests.robot | 8 +- .../src/main/smoketest/debug/ozone-debug.robot | 5 +- .../src/main/smoketest/ec/awss3ecstorage.robot | 90 ++++ .../main/smoketest/httpfs/operations_tests.robot | 1 + .../dist/src/main/smoketest/s3/awss3.robot | 8 + .../dist/src/main/smoketest/s3/bucketlist.robot | 3 +- .../dist/src/main/smoketest/s3/objectlist.robot | 66 +++ .../src/main/smoketest/s3/s3_compatbility_check.sh | 1 + .../fault-injection-test/mini-chaos-tests/pom.xml | 55 ++- hadoop-ozone/freon/pom.xml | 11 +- .../hadoop/ozone/freon/RandomKeyGenerator.java | 30 +- hadoop-ozone/httpfsgateway/pom.xml | 63 ++- .../fs/http/server/HttpFSServerWebServer.java | 5 +- hadoop-ozone/insight/pom.xml | 42 +- .../ozone/insight/TestConfigurationSubCommand.java | 53 +- .../dev-support/findbugsExcludeFile.xml | 16 + .../pom.xml | 110 ++++- .../hadoop/ozone/StandardOutputTestBase.java | 0 .../apache/hadoop/ozone/recon/ReconService.java | 1 - .../hadoop/ozone/recon}/TestNSSummaryAdmin.java | 3 +- .../ozone/recon/TestReconAndAdminContainerCLI.java | 0 .../hadoop/ozone/recon/TestReconAsPassiveScm.java | 0 .../ozone/recon/TestReconContainerEndpoint.java | 0 .../hadoop/ozone/recon/TestReconEndpointUtil.java | 0 .../TestReconInsightsForDeletedDirectories.java | 0 .../hadoop/ozone/recon/TestReconScmSnapshot.java | 0 .../apache/hadoop/ozone/recon/TestReconTasks.java | 0 .../ozone/recon/TestReconWithOzoneManager.java | 0 .../ozone/recon/TestReconWithOzoneManagerFSO.java | 0 .../ozone/recon/TestReconWithOzoneManagerHA.java | 0 .../src/test/resources/ozone-site.xml | 20 +- hadoop-ozone/integration-test-s3/pom.xml | 30 ++ .../apache/hadoop/ozone/s3/S3GatewayService.java | 2 - .../ozone/s3/awssdk/v1/AbstractS3SDKV1Tests.java | 102 +++- .../ozone/s3/awssdk/v2/AbstractS3SDKV2Tests.java | 138 +++++- .../src/test/resources/ozone-site.xml | 5 +- hadoop-ozone/integration-test/pom.xml | 337 +++++++++++-- .../fs/ozone/AbstractOzoneFileSystemTest.java | 3 +- .../ozone/AbstractRootedOzoneFileSystemTest.java | 3 +- .../ozone/TestDirectoryDeletingServiceWithFSO.java | 28 +- .../hdds/scm/TestStorageContainerManager.java | 13 +- .../ozone/TestContainerBalancerOperations.java | 4 +- .../hadoop/ozone/TestOzoneConfigurationFields.java | 1 + .../rpc/TestContainerStateMachineFailures.java | 2 +- .../TestCloseContainerByPipeline.java | 8 +- .../commandhandler/TestCloseContainerHandler.java | 2 +- .../commandhandler/TestDeleteContainerHandler.java | 26 +- .../TestRefreshVolumeUsageHandler.java | 2 + .../ozoneimpl/TestOzoneContainerWithTLS.java | 3 +- .../hadoop/ozone/freon/TestOMSnapshotDAG.java | 8 +- .../hadoop/ozone/freon/TestRandomKeyGenerator.java | 23 + .../org/apache/hadoop/ozone/om/TestKeyPurging.java | 2 +- .../org/apache/hadoop/ozone/om/TestOmAcls.java | 56 +++ .../ozone/om/TestOmContainerLocationCache.java | 12 +- .../org/apache/hadoop/ozone/om/TestOmMetrics.java | 89 ++++ .../hadoop/ozone/om/snapshot/TestOmSnapshot.java | 101 ++++ .../om/snapshot/TestOzoneManagerHASnapshot.java | 61 ++- .../snapshot/TestSnapshotBackgroundServices.java | 5 +- ...TestSnapshotDeletingServiceIntegrationTest.java | 104 +++- .../TestSnapshotDirectoryCleaningService.java | 31 +- .../ozone/reconfig/TestOmReconfiguration.java | 2 + .../hadoop/ozone/shell/TestOzoneDebugShell.java | 3 +- .../hadoop/ozone/shell/TestOzoneShellHA.java | 10 +- .../hadoop/ozone/shell/TestReconfigShell.java | 55 ++- .../src/test/resources/ozone-site.xml | 12 + hadoop-ozone/interface-client/pom.xml | 24 +- .../src/main/proto/OmClientProtocol.proto | 5 +- hadoop-ozone/interface-storage/pom.xml | 38 +- hadoop-ozone/mini-cluster/pom.xml | 16 +- hadoop-ozone/ozone-manager/pom.xml | 26 +- .../apache/hadoop/ozone/om/BucketManagerImpl.java | 2 +- .../hadoop/ozone/om/DeletingServiceMetrics.java | 6 + .../org/apache/hadoop/ozone/om/KeyManager.java | 77 ++- .../org/apache/hadoop/ozone/om/KeyManagerImpl.java | 289 +++++------ .../org/apache/hadoop/ozone/om/ListIterator.java | 2 +- .../hadoop/ozone/om/OMDBCheckpointServlet.java | 2 +- .../java/org/apache/hadoop/ozone/om/OMMetrics.java | 30 ++ .../hadoop/ozone/om/OmMetadataManagerImpl.java | 156 ------ .../apache/hadoop/ozone/om/OmSnapshotManager.java | 23 +- .../org/apache/hadoop/ozone/om/OzoneManager.java | 69 ++- .../hadoop/ozone/om/PendingKeysDeletion.java | 18 +- .../apache/hadoop/ozone/om/PrefixManagerImpl.java | 2 +- .../hadoop/ozone/om/SstFilteringService.java | 12 +- .../apache/hadoop/ozone/om/TrashPolicyOzone.java | 7 +- .../apache/hadoop/ozone/om/VolumeManagerImpl.java | 4 +- .../ozone/om/lock/OBSKeyPathLockStrategy.java | 4 +- .../ozone/om/lock/RegularBucketLockStrategy.java | 2 +- .../hadoop/ozone/om/request/OMClientRequest.java | 6 +- .../om/request/bucket/OMBucketCreateRequest.java | 4 +- .../om/request/bucket/OMBucketDeleteRequest.java | 4 +- .../om/request/bucket/OMBucketSetOwnerRequest.java | 2 +- .../request/bucket/OMBucketSetPropertyRequest.java | 2 +- .../om/request/bucket/acl/OMBucketAclRequest.java | 2 +- .../om/request/file/OMDirectoryCreateRequest.java | 2 +- .../file/OMDirectoryCreateRequestWithFSO.java | 2 +- .../ozone/om/request/file/OMFileCreateRequest.java | 2 +- .../request/file/OMFileCreateRequestWithFSO.java | 2 +- .../ozone/om/request/file/OMFileRequest.java | 16 + .../om/request/file/OMRecoverLeaseRequest.java | 2 +- .../om/request/key/OMAllocateBlockRequest.java | 2 +- .../request/key/OMAllocateBlockRequestWithFSO.java | 2 +- .../key/OMDirectoriesPurgeRequestWithFSO.java | 2 +- .../ozone/om/request/key/OMKeyCommitRequest.java | 2 +- .../om/request/key/OMKeyCommitRequestWithFSO.java | 2 +- .../om/request/key/OMKeyCreateRequestWithFSO.java | 2 +- .../ozone/om/request/key/OMKeyDeleteRequest.java | 2 +- .../om/request/key/OMKeyDeleteRequestWithFSO.java | 2 +- .../ozone/om/request/key/OMKeyPurgeRequest.java | 6 +- .../ozone/om/request/key/OMKeyRenameRequest.java | 2 +- .../om/request/key/OMKeyRenameRequestWithFSO.java | 2 +- .../hadoop/ozone/om/request/key/OMKeyRequest.java | 2 +- .../ozone/om/request/key/OMKeySetTimesRequest.java | 2 +- .../request/key/OMKeySetTimesRequestWithFSO.java | 2 +- .../ozone/om/request/key/OMKeysDeleteRequest.java | 2 +- .../ozone/om/request/key/OMKeysRenameRequest.java | 2 +- .../om/request/key/OMOpenKeysDeleteRequest.java | 2 +- .../ozone/om/request/key/acl/OMKeyAclRequest.java | 2 +- .../om/request/key/acl/OMKeyAclRequestWithFSO.java | 2 +- .../request/key/acl/prefix/OMPrefixAclRequest.java | 2 +- .../S3ExpiredMultipartUploadsAbortRequest.java | 2 +- .../S3InitiateMultipartUploadRequest.java | 2 +- .../S3InitiateMultipartUploadRequestWithFSO.java | 2 +- .../multipart/S3MultipartUploadAbortRequest.java | 2 +- .../S3MultipartUploadCommitPartRequest.java | 2 +- .../S3MultipartUploadCompleteRequest.java | 2 +- .../s3/tagging/S3DeleteObjectTaggingRequest.java | 2 +- .../S3DeleteObjectTaggingRequestWithFSO.java | 2 +- .../s3/tagging/S3PutObjectTaggingRequest.java | 2 +- .../tagging/S3PutObjectTaggingRequestWithFSO.java | 2 +- .../s3/tenant/OMTenantAssignAdminRequest.java | 2 +- .../tenant/OMTenantAssignUserAccessIdRequest.java | 2 +- .../request/s3/tenant/OMTenantCreateRequest.java | 4 +- .../request/s3/tenant/OMTenantDeleteRequest.java | 2 +- .../s3/tenant/OMTenantRevokeAdminRequest.java | 2 +- .../tenant/OMTenantRevokeUserAccessIdRequest.java | 2 +- .../request/snapshot/OMSnapshotCreateRequest.java | 4 +- .../request/snapshot/OMSnapshotDeleteRequest.java | 4 +- .../request/snapshot/OMSnapshotPurgeRequest.java | 7 +- .../request/snapshot/OMSnapshotRenameRequest.java | 11 +- .../om/request/volume/OMQuotaRepairRequest.java | 4 +- .../om/request/volume/OMVolumeCreateRequest.java | 4 +- .../om/request/volume/OMVolumeDeleteRequest.java | 4 +- .../om/request/volume/OMVolumeSetOwnerRequest.java | 2 +- .../om/request/volume/OMVolumeSetQuotaRequest.java | 2 +- .../om/request/volume/acl/OMVolumeAclRequest.java | 2 +- .../key/OMDirectoriesPurgeResponseWithFSO.java | 39 +- .../ozone/om/response/key/OMKeyPurgeResponse.java | 17 +- .../OMSnapshotMoveDeletedKeysResponse.java | 8 +- .../snapshot/OMSnapshotMoveTableKeysResponse.java | 6 +- .../response/snapshot/OMSnapshotPurgeResponse.java | 8 +- .../om/service/AbstractKeyDeletingService.java | 192 ++++---- .../ozone/om/service/DirectoryDeletingService.java | 518 ++++++++++++------- .../ozone/om/service/KeyDeletingService.java | 546 +++++++-------------- .../ozone/om/service/SnapshotDeletingService.java | 8 +- .../service/SnapshotDirectoryCleaningService.java | 10 +- .../ozone/om/snapshot/MultiSnapshotLocks.java | 6 +- .../hadoop/ozone/om/snapshot/ReferenceCounted.java | 12 +- .../hadoop/ozone/om/snapshot/SnapshotCache.java | 134 ++++- .../ozone/om/snapshot/SnapshotDiffManager.java | 43 +- .../hadoop/ozone/om/snapshot/SnapshotUtils.java | 7 +- .../om/snapshot/filter/ReclaimableDirFilter.java | 4 +- .../om/snapshot/filter/ReclaimableFilter.java | 36 +- .../om/snapshot/filter/ReclaimableKeyFilter.java | 6 +- .../filter/ReclaimableRenameEntryFilter.java | 4 +- .../apache/hadoop/ozone/om/TestKeyManagerImpl.java | 32 +- .../ozone/om/request/OMRequestTestUtils.java | 37 +- .../TestOMDirectoriesPurgeRequestAndResponse.java | 127 ++++- .../key/TestOMKeyPurgeRequestAndResponse.java | 63 ++- .../ozone/om/request/key/TestOMKeyRequest.java | 26 +- .../s3/multipart/TestS3MultipartRequest.java | 6 +- .../TestOMSnapshotPurgeRequestAndResponse.java | 30 +- .../TestOMSnapshotMoveTableKeysResponse.java | 8 +- .../ozone/om/service/TestKeyDeletingService.java | 228 ++++++++- .../ozone/om/snapshot/TestMultiSnapshotLocks.java | 2 +- .../ozone/om/snapshot/TestSnapshotCache.java | 190 ++++++- .../ozone/om/snapshot/TestSnapshotDiffManager.java | 37 +- .../ozone/om/snapshot/TestSnapshotUtils.java | 172 +++++++ .../ozone/om/snapshot/TestSstFilteringService.java | 3 +- .../filter/AbstractReclaimableFilterTest.java | 39 +- .../snapshot/filter/TestReclaimableDirFilter.java | 8 +- .../om/snapshot/filter/TestReclaimableFilter.java | 34 +- .../snapshot/filter/TestReclaimableKeyFilter.java | 10 +- .../filter/TestReclaimableRenameEntryFilter.java | 8 +- .../fs/ozone/BasicOzoneClientAdapterImpl.java | 1 + .../hadoop/fs/ozone/BasicOzoneFileSystem.java | 1 + .../ozone/BasicRootedOzoneClientAdapterImpl.java | 1 + .../fs/ozone/BasicRootedOzoneFileSystem.java | 1 + hadoop-ozone/ozonefs-hadoop2/pom.xml | 1 + hadoop-ozone/ozonefs-hadoop3-client/pom.xml | 3 - hadoop-ozone/ozonefs-hadoop3/pom.xml | 1 + hadoop-ozone/ozonefs-shaded/pom.xml | 4 +- hadoop-ozone/ozonefs/pom.xml | 4 - hadoop-ozone/pom.xml | 1 + hadoop-ozone/recon-codegen/pom.xml | 49 +- hadoop-ozone/recon/pom.xml | 95 ++-- .../hadoop/ozone/recon/ReconControllerModule.java | 1 - .../hadoop/ozone/recon}/ReconSqlDbConfig.java | 2 +- .../ozone/recon/scm/TestReconNodeManager.java | 4 +- hadoop-ozone/s3-secret-store/pom.xml | 4 + hadoop-ozone/s3gateway/pom.xml | 111 ++--- .../apache/hadoop/ozone/s3/OzoneClientCache.java | 80 +-- .../hadoop/ozone/s3/OzoneClientProducer.java | 37 +- .../hadoop/ozone/s3/S3GatewayConfigKeys.java | 4 + .../hadoop/ozone/s3/SignedChunksInputStream.java | 86 +++- ...tStream.java => UnsignedChunksInputStream.java} | 115 +++-- .../hadoop/ozone/s3/endpoint/BucketEndpoint.java | 46 +- .../hadoop/ozone/s3/endpoint/EndpointBase.java | 3 +- .../s3/endpoint/ListMultipartUploadsResult.java | 4 +- .../hadoop/ozone/s3/endpoint/ObjectEndpoint.java | 121 +++-- .../hadoop/ozone/s3/endpoint/RootEndpoint.java | 2 +- .../org/apache/hadoop/ozone/s3/endpoint/S3Acl.java | 4 +- .../hadoop/ozone/s3/endpoint/S3BucketAcl.java | 13 +- .../apache/hadoop/ozone/s3/endpoint/S3Owner.java | 14 +- .../hadoop/ozone/s3/exception/S3ErrorTable.java | 6 + .../ozone/s3/signature/StringToSignProducer.java | 55 ++- .../org/apache/hadoop/ozone/s3/util/S3Consts.java | 8 +- .../apache/hadoop/ozone/s3/util/S3StorageType.java | 57 +-- .../org/apache/hadoop/ozone/s3/util/S3Utils.java | 175 ++++--- .../ozone/s3secret/S3SecretEndpointBase.java | 22 +- .../ozone/s3secret/S3SecretManagementEndpoint.java | 7 + ...ientProducer.java => TestOzoneClientCache.java} | 36 +- .../ozone/s3/TestSignedChunksInputStream.java | 231 ++++++--- .../ozone/s3/TestUnsignedChunkInputStream.java | 223 +++++++++ .../ozone/s3/endpoint/BucketEndpointBuilder.java} | 31 +- .../hadoop/ozone/s3/endpoint/EndpointBuilder.java | 2 +- .../hadoop/ozone/s3/endpoint/TestBucketList.java | 58 +++ .../hadoop/ozone/s3/endpoint/TestListParts.java | 3 + .../s3/endpoint/TestMultipartUploadComplete.java | 3 + .../s3/endpoint/TestMultipartUploadWithCopy.java | 5 + .../hadoop/ozone/s3/endpoint/TestObjectGet.java | 2 + .../hadoop/ozone/s3/endpoint/TestObjectPut.java | 13 +- .../ozone/s3/endpoint/TestObjectTaggingDelete.java | 3 + .../ozone/s3/endpoint/TestObjectTaggingGet.java | 3 + .../ozone/s3/endpoint/TestObjectTaggingPut.java | 4 +- .../hadoop/ozone/s3/endpoint/TestPartUpload.java | 7 + .../s3/endpoint/TestPartUploadWithStream.java | 3 + .../ozone/s3/endpoint/TestPermissionCheck.java | 3 + .../hadoop/ozone/s3/endpoint/TestRootList.java | 2 +- .../ozone/s3/endpoint/TestUploadWithStream.java | 2 + .../ozone/s3/metrics/TestS3GatewayMetrics.java | 3 + .../apache/hadoop/ozone/s3/util/TestS3Utils.java | 184 +++---- .../hadoop/ozone/s3secret/TestSecretGenerate.java | 2 +- .../hadoop/ozone/s3secret/TestSecretRevoke.java | 4 +- hadoop-ozone/tools/pom.xml | 99 ++-- .../debug/replicas/BlockExistenceVerifier.java | 1 + .../ozone/debug/replicas/ChecksumVerifier.java | 1 + .../debug/replicas/ContainerStateVerifier.java | 194 ++++++++ .../ozone/debug/replicas/ReplicasVerify.java | 18 + .../debug/replicas/chunk/ChunkKeyHandler.java | 17 +- pom.xml | 51 +- 386 files changed, 8622 insertions(+), 3599 deletions(-) diff --cc hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/reconciliation/TestReconcileContainerEventHandler.java index bb03529b7a1,00000000000..49fab0afe22 mode 100644,000000..100644 --- a/hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/reconciliation/TestReconcileContainerEventHandler.java +++ b/hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/reconciliation/TestReconcileContainerEventHandler.java @@@ -1,312 -1,0 +1,312 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hdds.scm.container.reconciliation; + +import static org.apache.hadoop.hdds.protocol.proto.HddsProtos.ReplicationFactor.ONE; +import static org.apache.hadoop.hdds.protocol.proto.HddsProtos.ReplicationFactor.THREE; +import static org.apache.hadoop.hdds.scm.events.SCMEvents.DATANODE_COMMAND; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.util.Arrays; +import java.util.HashSet; +import java.util.List; +import java.util.Set; - import java.util.UUID; +import java.util.stream.Collectors; +import org.apache.hadoop.hdds.client.ECReplicationConfig; +import org.apache.hadoop.hdds.client.RatisReplicationConfig; +import org.apache.hadoop.hdds.client.ReplicationConfig; +import org.apache.hadoop.hdds.protocol.DatanodeDetails; ++import org.apache.hadoop.hdds.protocol.DatanodeID; +import org.apache.hadoop.hdds.protocol.proto.HddsProtos.LifeCycleState; +import org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.ContainerReplicaProto.State; +import org.apache.hadoop.hdds.protocol.proto.StorageContainerDatanodeProtocolProtos.ReconcileContainerCommandProto; +import org.apache.hadoop.hdds.scm.HddsTestUtils; +import org.apache.hadoop.hdds.scm.container.ContainerID; +import org.apache.hadoop.hdds.scm.container.ContainerInfo; +import org.apache.hadoop.hdds.scm.container.ContainerManager; +import org.apache.hadoop.hdds.scm.container.ContainerNotFoundException; +import org.apache.hadoop.hdds.scm.container.ContainerReplica; +import org.apache.hadoop.hdds.scm.container.MockNodeManager; +import org.apache.hadoop.hdds.scm.container.reconciliation.ReconciliationEligibilityHandler.EligibilityResult; +import org.apache.hadoop.hdds.scm.container.reconciliation.ReconciliationEligibilityHandler.Result; +import org.apache.hadoop.hdds.scm.ha.SCMContext; +import org.apache.hadoop.hdds.server.events.EventPublisher; +import org.apache.hadoop.ozone.protocol.commands.CommandForDatanode; +import org.apache.hadoop.ozone.protocol.commands.SCMCommand; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.EnumSource; +import org.mockito.ArgumentCaptor; + +/** + * Tests that the ReconcileContainerEventHandler properly accepts and rejects reconciliation events based on + * container state, and dispatches commands to datanodes accordingly. + */ +public class TestReconcileContainerEventHandler { + private ContainerManager containerManager; + private EventPublisher eventPublisher; + private ReconcileContainerEventHandler eventHandler; + private SCMContext scmContext; + + private static final ContainerID CONTAINER_ID = ContainerID.valueOf(123L); + private static final long LEADER_TERM = 3L; + + private static final ReplicationConfig RATIS_THREE_REP = RatisReplicationConfig.getInstance(THREE); + private static final ReplicationConfig RATIS_ONE_REP = RatisReplicationConfig.getInstance(ONE); + private static final ReplicationConfig EC_REP = new ECReplicationConfig(3, 2); + + private ArgumentCaptor<CommandForDatanode<ReconcileContainerCommandProto>> commandCaptor; + + @BeforeEach + public void setup() throws Exception { + commandCaptor = ArgumentCaptor.forClass(CommandForDatanode.class); + containerManager = mock(ContainerManager.class); + scmContext = mock(SCMContext.class); + when(scmContext.isLeader()).thenReturn(true); + when(scmContext.getTermOfLeader()).thenReturn(LEADER_TERM); + eventPublisher = mock(EventPublisher.class); + eventHandler = new ReconcileContainerEventHandler(containerManager, scmContext); + } + + /** + * EC containers are not yet supported for reconciliation. + */ + @Test + public void testReconcileECContainer() throws Exception { + addContainer(EC_REP, LifeCycleState.CLOSED); + addReplicasToContainer(5); + + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + assertFalse(result.isOk()); + assertEquals(Result.INELIGIBLE_REPLICATION_TYPE, result.getResult()); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), any()); + } + + /** + * Ratis 1 containers are not currently supported for reconciliation. + */ + @Test + public void testReconcileRatisOneContainer() throws Exception { + addContainer(RATIS_ONE_REP, LifeCycleState.CLOSED); + addReplicasToContainer(1); + + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + assertFalse(result.isOk()); + assertEquals(Result.NOT_ENOUGH_REQUIRED_NODES, result.getResult()); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), any()); + } + + @Test + public void testReconcileWhenNotLeader() throws Exception { + addContainer(RATIS_THREE_REP, LifeCycleState.CLOSED); + addReplicasToContainer(3); + when(scmContext.isLeader()).thenReturn(false); + + // Container is eligible for reconciliation, but the request will not go through because this SCM is not the leader. + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + assertTrue(result.isOk()); + assertEquals(Result.OK, result.getResult()); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), any()); + } + + @Test + public void testReconcileNonexistentContainer() throws Exception { + // The step of adding the container to the mocked ContainerManager is intentionally skipped to simulate a + // nonexistent container. + // No exceptions should be thrown out of this test method when this happens. If they are, they will be propagated + // and the test will fail. + when(containerManager.getContainer(any())).thenThrow(new ContainerNotFoundException()); + + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + assertFalse(result.isOk()); + assertEquals(Result.CONTAINER_NOT_FOUND, result.getResult()); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), any()); + } + + @Test + public void testReconcileMissingContainer() throws Exception { + addContainer(RATIS_THREE_REP, LifeCycleState.CLOSED); + assertTrue(containerManager.getContainerReplicas(CONTAINER_ID).isEmpty(), + "Expected no replicas for this container"); + + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + assertFalse(result.isOk()); + assertEquals(Result.NO_REPLICAS_FOUND, result.getResult()); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), any()); + } + + @ParameterizedTest + @EnumSource(LifeCycleState.class) + public void testReconcileWithContainerStates(LifeCycleState state) throws Exception { + addContainer(RATIS_THREE_REP, state); + addReplicasToContainer(3); + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + switch (state) { + case OPEN: + case CLOSING: + case DELETING: + case DELETED: + case RECOVERING: + assertFalse(result.isOk()); + assertEquals(Result.INELIGIBLE_CONTAINER_STATE, result.getResult()); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), commandCaptor.capture()); + break; + default: + assertTrue(result.isOk()); + assertEquals(Result.OK, result.getResult()); + verify(eventPublisher, times(3)).fireEvent(eq(DATANODE_COMMAND), commandCaptor.capture()); + break; + } + } + + // TODO HDDS-10714 will change which datanodes are eligible to participate in reconciliation. + @Test + public void testReconcileSentToAllPeers() throws Exception { + addContainer(RATIS_THREE_REP, LifeCycleState.CLOSED); + Set<ContainerReplica> replicas = addReplicasToContainer(3); - Set<UUID> allNodeIDs = replicas.stream() - .map(r -> r.getDatanodeDetails().getUuid()) ++ Set<DatanodeID> allNodeIDs = replicas.stream() ++ .map(r -> r.getDatanodeDetails().getID()) + .collect(Collectors.toSet()); + + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + assertTrue(result.isOk()); + assertEquals(Result.OK, result.getResult()); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + assertEquals(3, replicas.size()); + assertEquals(allNodeIDs.size(), replicas.size()); + verify(eventPublisher, times(replicas.size())).fireEvent(eq(DATANODE_COMMAND), commandCaptor.capture()); + + // Check each reconcile command sent for correctness. - Set<UUID> nodesReceivingCommands = new HashSet<>(); ++ Set<DatanodeID> nodesReceivingCommands = new HashSet<>(); + for (CommandForDatanode<ReconcileContainerCommandProto> dnCommand: commandCaptor.getAllValues()) { + SCMCommand<ReconcileContainerCommandProto> reconcileCommand = dnCommand.getCommand(); + ReconcileContainerCommandProto reconcileProto = reconcileCommand.getProto(); + // All commands should use the latest term of SCM so the datanode does not drop them. + assertEquals(LEADER_TERM, reconcileCommand.getTerm()); + // All commands should have the same container ID. + assertEquals(CONTAINER_ID, ContainerID.valueOf(reconcileProto.getContainerID())); + // Container ID is also used as the command's identifier. + assertEquals(CONTAINER_ID, ContainerID.valueOf(reconcileCommand.getId())); + + // Every node should receive exactly one reconcile command. - UUID targetNodeID = dnCommand.getDatanodeId(); ++ DatanodeID targetNodeID = dnCommand.getDatanodeId(); + assertTrue(nodesReceivingCommands.add(targetNodeID), "Duplicate reconcile command sent to datanode."); + // All commands should have correctly constructed peer lists that exclude the node receiving the command. - Set<UUID> expectedPeerIDs = allNodeIDs.stream() - .filter(id -> id != targetNodeID) ++ Set<DatanodeID> expectedPeerIDs = allNodeIDs.stream() ++ .filter(id -> !id.equals(targetNodeID)) + .collect(Collectors.toSet()); - Set<UUID> actualPeerIDs = reconcileProto.getPeersList().stream() - .map(dn -> UUID.fromString(dn.getUuid())) ++ Set<DatanodeID> actualPeerIDs = reconcileProto.getPeersList().stream() ++ .map(dn -> DatanodeID.fromProto(dn.getId())) + .collect(Collectors.toSet()); + assertEquals(replicas.size() - 1, actualPeerIDs.size()); + assertEquals(expectedPeerIDs, actualPeerIDs); + } + + assertEquals(allNodeIDs, nodesReceivingCommands); + } + + @ParameterizedTest + @EnumSource(State.class) + public void testReconcileFailsWithIneligibleReplicas(State replicaState) throws Exception { + // Overall container state is eligible for reconciliation, but some replicas may not be. + // This means the container will not be considered eligible. + addContainer(RATIS_THREE_REP, LifeCycleState.CLOSED); + // Only one replica is in a different state. + addReplicasToContainer(replicaState, State.CLOSED, State.CLOSED); + + EligibilityResult result = + ReconciliationEligibilityHandler.isEligibleForReconciliation(CONTAINER_ID, containerManager); + + eventHandler.onMessage(CONTAINER_ID, eventPublisher); + switch (replicaState) { + case OPEN: + case INVALID: + case DELETED: + case CLOSING: + assertFalse(result.isOk()); + assertEquals(Result.INELIGIBLE_REPLICA_STATES, result.getResult()); + verify(eventPublisher, never()).fireEvent(eq(DATANODE_COMMAND), commandCaptor.capture()); + break; + default: + assertTrue(result.isOk()); + assertEquals(Result.OK, result.getResult()); + verify(eventPublisher, times(3)).fireEvent(eq(DATANODE_COMMAND), commandCaptor.capture()); + break; + } + } + + private ContainerInfo addContainer(ReplicationConfig repConfig, LifeCycleState state) throws Exception { + ContainerInfo container = new ContainerInfo.Builder() + .setContainerID(CONTAINER_ID.getId()) + .setReplicationConfig(repConfig) + .setState(state) + .build(); + when(containerManager.getContainer(CONTAINER_ID)).thenReturn(container); + return container; + } + + private Set<ContainerReplica> addReplicasToContainer(int count) throws Exception { + State[] replicaStates = new State[count]; + Arrays.fill(replicaStates, State.CLOSED); + return addReplicasToContainer(replicaStates); + } + + private Set<ContainerReplica> addReplicasToContainer(State... replicaStates) throws Exception { + // Add one container replica for each replica state specified. + // If no states are specified, replica list will be empty. + Set<ContainerReplica> replicas = new HashSet<>(); + try (MockNodeManager nodeManager = new MockNodeManager(true, replicaStates.length)) { + List<DatanodeDetails> nodes = nodeManager.getAllNodes(); + for (int i = 0; i < replicaStates.length; i++) { + replicas.addAll(HddsTestUtils.getReplicas(CONTAINER_ID, replicaStates[i], nodes.get(i))); + } + } + when(containerManager.getContainerReplicas(CONTAINER_ID)).thenReturn(replicas); + + return replicas; + } +} --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
