[
https://issues.apache.org/jira/browse/LUCENE-3373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540701#comment-17540701
]
Vigya Sharma commented on LUCENE-3373:
--------------------------------------
I doubt if disabling ConcurrentMergeScheduler throttling would help here, as
the code path I mentioned, gets triggered regardless of whether throttling is
enabled or not.
Separately, I don't have a good feeling about the whole unbounded wait in
IndexWriter shutdown(). I guess it was written this way, because we don't seem
to have a good time upper bound imposed on running merges (although merges are
transactional, such that aborting them midway would only be wasted work, but
not cause any corruption as far as I know).
I think we should have a timeout on this wait, at least on the wait for
pendingMerges. If we are done with runningMerges but still have pending merges
left, we could exit from this thread and abort them (the same way we
[abort|https://github.com/apache/lucene/blob/d17c6056d8caada6db6c1c4f280f54960e058ee2/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java#L2438]
in {{rollbackInternalNoCommit()}} ). This should be an easy fix. I can take
this up if Lucene experts in this area think this is a good idea.
> waitForMerges deadlocks if background merge fails
> -------------------------------------------------
>
> Key: LUCENE-3373
> URL: https://issues.apache.org/jira/browse/LUCENE-3373
> Project: Lucene - Core
> Issue Type: Bug
> Components: core/index
> Affects Versions: 3.0.3
> Reporter: Tim Smith
> Priority: Major
>
> waitForMerges can deadlock if a merge fails for ConcurrentMergeScheduler
> this is because the merge thread will die, but pending merges are still
> available
> normally, the merge thread will pick up the next merge once it finishes the
> previous merge, but in the event of a merge exception, the pending work is
> not resumed, but waitForMerges won't complete until all pending work is
> complete
> i worked around this by overriding doMerge() like so:
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException
> {
> try {
> super.doMerge(merge);
> } catch (Throwable exc) {
> // Just logging the exception and not rethrowing
> // insert logging code here
> }
> }
> {code}
> Here's the rough steps i used to reproduce this issue:
> override doMerge like so
> {code}
> protected final void doMerge(MergePolicy.OneMerge merge) throws IOException
> {
> try {Thread.sleep(500L);} catch (InterruptedException e) { }
> super.doMerge(merge);
> throw new IOException("fail");
> }
> {code}
> then, if you do the following:
> loop 50 times:
> addDocument // any doc
> commit
> waitForMerges // This will deadlock sometimes
> SOLR-2017 may be related to this (stack trace for deadlock looked related)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
