about <1>. Well, at a high level you're right, of course. Having the EFF stuff in a single place seems more elegant. But then ugly details crop up. I.e. "one place" implies that you'd have to fetch them over the network, potentially a very expensive operation every time there was a commit. Is this really a good tradeoff? With high network latency, this could be a performance killer. But I suspect that the real reason is that nobody has found a compelling use-case for this kind of thing. Until and unless someone does, and is willing to make a patch, it'll be theory :).
bq: modifications also sent to replicas with this kind of commits brief review: Update process: 1> Update goes to a node. 2> node forwards to all leaders 3> leader forward to replicas 4> replicas respond to their leader. 5> leader responds to originating node. 6> originating node responds to caller. At this point all the replicas for your entire cluster have the update. This is entirely independent of commits. Whenever a commit is issued the documents currently pending on a node are committed and made visible to a searcher. If one is relying on solrconfig settings, then the commit happens a little bit out of synch. Let's say that the commit (hard with opensearcher=true or soft) is set to 60 seconds. Each node may have a different commit time, depending upon when it was started. So there may be a slight difference in when documents are visible. You'll probably never notice. If you issue commits from a client, then the commit is propagated to all nodes in the cluster. HTH, Erick On Fri, Nov 22, 2013 at 7:23 PM, Flavio Pompermaier <pomperma...@okkam.it>wrote: > On Fri, Nov 22, 2013 at 2:21 PM, Erick Erickson <erickerick...@gmail.com > >wrote: > > > 1> I'm not quite sure I understand. External File Fields are keyed > > by the unique id of the doc. So every shard _must_ have the > > eff available for at least the documents in that shard. At first glance > > this doesn't look simple. Perhaps a bit more explanation of what > > you're using EFF for? > > > Thanks Erick for the reply, I use EFF for boosting results by popularity. > So I was right, I should put popularity in every shard data dir..right? But > why not keeping that file in just one place (obviously the file should be > reachable by all solrcloud nodes...) and allow external fields to be > outside data dir? > > > > > 2> Let's be sure we're talking about the same thing here. In Solr, > > a "commit" is the command that makes documents visible, often > > controlled by the autoCommit and autoSoftCommit settings in > > solrconfig.xml. You will not be able to issue 100 commits/second. > > > > If you're using "commit" to mean adding a document to the index, > > then 100/s should be no problem. I regularly see many times that > > ingestion rate. The documents won't be visible to search until > > you do a commit however. > > > Yeah, now it is more clear. Still a question: for my client is not a > problem to soft commit but, are the modifications also sent to replicas > with this kind of commits? > > > > > Best > > Erick > > > > > > On Fri, Nov 22, 2013 at 4:44 AM, Flavio Pompermaier < > pomperma...@okkam.it > > >wrote: > > > > > Hi to all, > > > we're migrating from solr 3.x to solr 4.x to use Solrcloud and I have > two > > > big doubts: > > > > > > 1) External fields. When I compute such a file do I have to copy it in > > the > > > data directory of shards..? The external fields boosts the results of > > the > > > query to a specific collection, for me it doesn't make sense to put it > in > > > all shard's data dir, it should be something related to the collection > > > itself. > > > Am I wrong or missing something? Is there a simple way to upload the > > > popularity file (for the external field) at one in all shards? > > > > > > 2) My index requires frequently commits (i.e. sometimes up to 100/s). > How > > > do I have to manage this? Do I have to use soft commits..? Any simple > > > configuration/code snippet to use them? Is it true that external fields > > > affect performance on commit? > > > > > > Best, > > > Flavio > > > > > >
