[
https://issues.apache.org/jira/browse/SOLR-13932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974191#comment-16974191
]
Ilan Ginzburg commented on SOLR-13932:
--------------------------------------
Commented code in {{ServerSideMetadata}} constructor regarding
{{DirectoryReader.listCommits()}} mentions "But in blob context we serialize
commits and pulls by proper locking therefore we should be good here". I
believe this is no longer true (a previous Salesforce version of that code used
to serialize updates). We do not serialize commits but let Lucene do it the way
it wants (we serialize pushes to Blob).
OTOH, I don't see why we try to detect inconsistency using past files that are
no longer part of the commit point. If we rely on such detections, we're at
risk because files of past commit points can be deleted at any time. Our
strategy must make sure (I believe it does) that local is always consistent, or
local should be deleted and the Blob version adopted instead. The only time
where local is "a source of truth" over blob is after indexing. Consistency in
such a case is guaranteed by verifying that local was consistent with Blob
before indexing started (blob being source of truth), and that Blob hasn't
changed by the time indexing finished and got pushed back to Blob (allowing
parallel indexing from given node/replica, this is done by the conditional
update into ZK).
> Review directory locking for Blob interactions
> ----------------------------------------------
>
> Key: SOLR-13932
> URL: https://issues.apache.org/jira/browse/SOLR-13932
> Project: Solr
> Issue Type: Sub-task
> Reporter: Ilan Ginzburg
> Priority: Major
>
> Review resolution of local index directory content vs Blob copy.
> There has been wrong understanding of following line acquiring a lock on
> index directory.
> {{solrCore.getDirectoryFactory().get(indexDirPath,
> DirectoryFactory.DirContext.DEFAULT,
> solrCore.getSolrConfig().indexConfig.lockType);}}
> From Yonik:
> _A couple things about Directory locking.... the locks were only ever to
> prevent more than one IndexWriter from trying to modify the same index. The
> IndexWriter grabs a write lock once when it is created and does not release
> it until it is closed._
> _Directories are not locked on acquisition of the Directory from the
> DirectoryFactory. See the IndexWriter constructor, where the lock is
> explicitly grabbed._
> Review CorePushPull#pullUpdateFromBlob, ServerSideMetadata and other classes
> as relevant.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
