[
https://issues.apache.org/jira/browse/SOLR-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981175#comment-16981175
]
Ilan Ginzburg commented on SOLR-13932:
--------------------------------------
As a follow up to [the first comment on this
Jira|https://issues.apache.org/jira/browse/SOLR-13932?focusedCommentId=16974191&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16974191],
Blob metadata does not need to track the index generation and local index
generation does not have to be compared to the Blob generation. Decision to
*pull* from Blob should only be based on differences between local commit point
and Blob set of files (if the two are different, Blob is the source of truth).
Decision to *push* to Blob should only be made when local index was up to date
with Blob, an update occurred locally and we verify that no other replica (on
this or on another node) updated the Blob files in the meantime.
Therefore, as part of this Jira I'm going to remove the generation member
variable in {{BlobCoreMetadata}}, {{BlobCoreMetadataBuilder}} and the various
places where it is then used. It is used for resolution in
{{SharedStoreResolutionUtil}} and elsewhere for logging.
I believe the checks described in [my previous post
above|https://issues.apache.org/jira/browse/SOLR-13932?focusedCommentId=16977555&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16977555]
are sufficient to make sure we avoid conflicts between local and pulled files
regardless of which generation each index is at.
> Review directory locking for Blob interactions
> ----------------------------------------------
>
> Key: SOLR-13932
> URL: https://issues.apache.org/jira/browse/SOLR-13932
> Project: Solr
> Issue Type: Sub-task
> Reporter: Ilan Ginzburg
> Priority: Major
>
> Review resolution of local index directory content vs Blob copy.
> There has been wrong understanding of following line acquiring a lock on
> index directory.
> {{solrCore.getDirectoryFactory().get(indexDirPath,
> DirectoryFactory.DirContext.DEFAULT,
> solrCore.getSolrConfig().indexConfig.lockType);}}
> From Yonik:
> _A couple things about Directory locking.... the locks were only ever to
> prevent more than one IndexWriter from trying to modify the same index. The
> IndexWriter grabs a write lock once when it is created and does not release
> it until it is closed._
> _Directories are not locked on acquisition of the Directory from the
> DirectoryFactory. See the IndexWriter constructor, where the lock is
> explicitly grabbed._
> Review CorePushPull#pullUpdateFromBlob, ServerSideMetadata and other classes
> as relevant.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
