Thanks, just created SOLR-14581 as an entry point. And sure, beers sound good! ;-)
On 17/06/2020 23:13, Erick Erickson wrote: > Please raise a JIRA and attach your patch to that…. > > Best, > Erick > > P.S. Buy me some beers sometime if we’re even in the same place... > >> On Jun 17, 2020, at 5:00 PM, Bram Van Dam <bram.van...@intix.eu> wrote: >> >> Thanks for pointing that out. I'm attaching a patch for the ref-guide >> which summarizes what you said. Maybe other people will find this useful >> as well? >> >> Oh and Erick, thanks for your ever thoughtful replies. Given all the >> hours of your time I've soaked up over the years, you should probably >> start invoicing me :-) >> >> - Bram >> >> On 17/06/2020 13:55, Erick Erickson wrote: >>> Each node has its own timer that starts when it receives an update. >>> So in your situation, 60 seconds after any give replica gets it’s first >>> update, all documents that have been received in the interval will >>> be committed. >>> >>> But note several things: >>> >>> 1> commits will tend to cluster for a given shard. By that I mean >>> they’ll tend to happen within a few milliseconds of each other >>> ‘cause it doesn’t take that long for an update to get from the >>> leader to all the followers. >>> >>> 2> this is per replica. So if you host replicas from multiple collections >>> on some node, their commits have no relation to each other. And >>> say for some reason you transmit exactly one document that lands >>> on shard1. Further, say nodeA contains replicas for shard1 and shard2. >>> Only the replica for shard1 would commit. >>> >>> 3> Solr promises eventual consistency. In this case, due to all the >>> timing variables it is not guaranteed that every replica of a single >>> shard has the same document available for search at any given time. >>> Say doc1 hits the leader at time T and a follower at time T+10ms. >>> Say doc2 hits the leader and gets indexed 5ms before the >>> commit is triggered, but for some reason it takes 15ms for it to get >>> to the follower. The leader will be able to search doc2, but the >>> follower won’t until 60 seconds later. >>> >>> Best, >>> Erick >>> >>>> On Jun 17, 2020, at 5:36 AM, Bram Van Dam <bram.van...@intix.eu> wrote: >>>> >>>> 'morning :-) >>>> >>>> I'm wondering how autocommits work in Solr. >>>> >>>> Say I have a cluster with many nodes and many colections with many >>>> shards. If each collection's config has a hard autocommit configured >>>> every minute, does that mean that SolrCloud (presumably the leader?) >>>> will dish out commit requests to each node on that schedule? Or does >>>> each node have its own timed trigger? >>>> >>>> If it's the former, doesn't that mean the load will spike dramatically >>>> across the whole cluster every minute? >>>> >>>> I tried reading the code, but I don't quite understand the way >>>> CommitTracker and the UpdateHandlers interact with SolrCloud. >>>> >>>> Thanks, >>>> >>>> - Bram >>> >> >> <doc.patch> >
