andyvuong commented on a change in pull request #1188: SOLR-14044: Support
collection and shard deletion in shared storage
URL: https://github.com/apache/lucene-solr/pull/1188#discussion_r375441526
##########
File path:
solr/core/src/java/org/apache/solr/cloud/api/collections/DeleteCollectionCmd.java
##########
@@ -142,6 +148,34 @@ public void call(ClusterState state, ZkNodeProps message,
NamedList results) thr
break;
}
}
+
+ // Delete the collection files from shared store. We want to delete all
of the files before we delete
+ // the collection state from ZooKeeper.
+ DocCollection docCollection =
zkStateReader.getClusterState().getCollectionOrNull(collection);
+ if (docCollection != null && docCollection.getSharedIndex()) {
+ SharedStoreManager sharedStoreManager =
ocmh.overseer.getCoreContainer().getSharedStoreManager();
+ BlobDeleteManager deleteManager =
sharedStoreManager.getBlobDeleteManager();
+ BlobDeleteProcessor deleteProcessor =
deleteManager.getOverseerDeleteProcessor();
+ // deletes all files belonging to this collection
+ CompletableFuture<BlobDeleterTaskResult> deleteFuture =
+ deleteProcessor.deleteCollection(collection, false);
+
+ try {
+ // TODO: Find a reasonable timeout value
+ BlobDeleterTaskResult result = deleteFuture.get(60,
TimeUnit.SECONDS);
+ if (!result.isSuccess()) {
+ log.warn("Deleting all files belonging to shared collection " +
collection +
+ " was not successful! Files belonging to this collection may
be orphaned.");
+ }
+ } catch (TimeoutException tex) {
+ // We can orphan files here if we don't delete everything in time
but what matters for potentially
+ // reusing the collection name is that the zookeeper state of the
collection gets deleted which
+ // will happen in the finally block
+ throw new SolrException(SolrException.ErrorCode.SERVER_ERROR, "Could
not complete deleting collection" +
Review comment:
I throw an exception in both cases so the client calling is aware if the
command fails and files are orphaned. In DeleteCollection the files are "truly
orphaned" because even if we error out here, the collection will always be
deleted from zookeeper in the finally block and it's effectively gone from
Solr's perspective. The DeleteShard will fail the whole command without doing
the same delete from zookeeper action and a subsequent delete shard command can
be called to try again which isn't the case in the former (unless the same
collection name is created again).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
