[
https://issues.apache.org/jira/browse/SOLR-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981188#comment-16981188
]
Ilan Ginzburg commented on SOLR-13932:
--------------------------------------
As part of Blob being "source of truth", code in
{{CorePusher.pushCoreToBlob()}} that compares the generation of local commit
point to the Blob generation should go away. As stated previously, the decision
to push should not be based on generation.
As a matter of fact, we can ignore the notion of generation altogether in the
Blob push/pull code. All that really matters is do we have the latest copy from
Blob, and once we do and we update it locally, while pushing we should make
sure no other conflicting update happened to Blob, to make sure updates can't
overwrite each other.
Also, the push strategy currently takes a snapshot i.e. copies all the files in
the current commit point into a new directory (parameter {{takeSnapshot}} in
the {{ServerSideMetadata}} constructor) to push them from there. This is used
by {{CorePusher.pushCoreToBlob()}} and could lead to inefficient push
operations. Pushes should instead be done from the existing index directory
without making copies.
{{ReplicationHandler}} does not do a copy and does its best to reserver the
commit point while the client is replicating files. The {{IndexFetcher}} has to
deal with Exceptions when getting the file stream (see for example
{{IndexFetcher.downloadIndexFiles()}}).
> Review directory locking for Blob interactions
> ----------------------------------------------
>
> Key: SOLR-13932
> URL: https://issues.apache.org/jira/browse/SOLR-13932
> Project: Solr
> Issue Type: Sub-task
> Reporter: Ilan Ginzburg
> Priority: Major
>
> Review resolution of local index directory content vs Blob copy.
> There has been wrong understanding of following line acquiring a lock on
> index directory.
> {{solrCore.getDirectoryFactory().get(indexDirPath,
> DirectoryFactory.DirContext.DEFAULT,
> solrCore.getSolrConfig().indexConfig.lockType);}}
> From Yonik:
> _A couple things about Directory locking.... the locks were only ever to
> prevent more than one IndexWriter from trying to modify the same index. The
> IndexWriter grabs a write lock once when it is created and does not release
> it until it is closed._
> _Directories are not locked on acquisition of the Directory from the
> DirectoryFactory. See the IndexWriter constructor, where the lock is
> explicitly grabbed._
> Review CorePushPull#pullUpdateFromBlob, ServerSideMetadata and other classes
> as relevant.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
