On Thu, May 23, 2013 at 7:00 PM, AlexeyK <lex.kudi...@gmail.com> wrote:
<snip /> > from what I understood from the code, for each 'add' command there is a > test > for a 'delete by query'. if there is an older dbq, it's run after the 'add' > operation if its version > 'add' version. > in my case, there are a lot of documents to be inserted, and a single large > DBQ. My question is: shouldn't this be done in bulks? Why is it necessary > to > run the DBQ after each insertion? Supposedly there are 1000 insertions it's > run 1000 times. > > > As I understand it, this is done to handle out-of-order updates. Suppose a client makes a few add requests and then invokes a DBQ but the DBQ reaches the replicas before the last add request. In such a case, the DBQ is executed after the add request to preserve consistency. We don't do that in bulk because we don't know how long to wait for all add requests to arrive. Also, the individual add requests may arrive via different threads (think connection reset from leader to replica). That being said, the scenario you describe of a 1000 insertions causing DBQs to be run a large number of times (on recovery after restarting) could be optimized. Note that the bug you discovered (SOLR-4870) does not affect log replay because log replay on startup will replay all of the last two transaction logs (unless they end with a commit). Only PeerSync is affected by SOLR-4870. You say that both nodes are leaders but the comment inside DirectUpdateHandler2.addDoc() says that deletesAfter (i.e. reordered DBQs) should always be null on leaders. So there's definitely something fishy here. A quick review of the code leads me to believe that reordered DBQs can happen on a leader as well. I'll investigate further. > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-4-3-node-is-seen-as-active-in-Zk-while-in-recovery-mode-endless-recovery-tp4065549p4065628.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Regards, Shalin Shekhar Mangar.
