Bhawna, I think you need to reconcile yourself to the fact that what you want to achieve is not going to be possible.
Solr (and Lucene underneath it) is HEAVILY optimised for high read/low write situations, and that leads to some latency in content reaching the index. If you wanted to change this, you'd have to get into some heavy Java/Lucene coding, as I believe Twitter have done on Lucene itself. I'd say, rather than attempting to change this, I'd say you need to work out a way in your UI to handle this situation. E.g. have a "refresh on stale results" button, or "not seeing your data, try here". Or, if a user submits data, then wants to search for it in the same session, have your UI enforce a minimum 10s delay before it sends a request to Solr, or something like that. Efforts to solve this at the Solr end, without spending substantial sums and effort on it, will be futile as it isn't what Solr/Lucene are designed for. Upayavira On Tue, Jul 21, 2015, at 06:02 AM, Bhawna Asnani wrote: > Thanks, I tried turning off auto softCommits but that didn't help much. > Still seeing stale results every now and then. Also load on the server > very light. We are running this just on a test server with one or two > users. I don't see any warning in logs whole doing softCommits and it > says it successfully opened new searcher and registered it as main > searcher. Could this be due to caching? I have tried to disable all in my > solrconfig. > > Sent from my iPhone > > > On Jul 20, 2015, at 12:16 PM, Shawn Heisey <apa...@elyograg.org> wrote: > > > >> On 7/20/2015 9:29 AM, Bhawna Asnani wrote: > >> Thanks for your suggestions. The requirement is still the same , to be > >> able to make a change to some solr documents and be able to see it on > >> subsequent search/facet calls. > >> I am using softCommit with waitSearcher=true. > >> > >> Also I am sending reads/writes to a single solr node only. > >> I have tried disabling caches and warmup time in logs is '0' but every > >> once in a while I do get the document just updated with stale data. > >> > >> I went through lucene documentation and it seems opening the > >> IndexReader with the IndexWriter should make the changes visible to > >> the reader. > >> > >> I checked solr logs no errors. I see this in logs each time > >> 'Registered new searcher Searcher@xxxxx' even before searches that had > >> the stale document. > >> > >> I have attached my solrconfig.xml for reference. > > > > Your attachment made it through the mailing list processing. Most > > don't, I'm surprised. Some thoughts: > > > > maxBooleanClauses has been set to 400000. This is a lot. If you > > actually need a setting that high, then you are sending some MASSIVE > > queries, which probably means that your Solr install is exceptionally > > busy running those queries. > > > > If the server is fairly busy, then you should increase maxTime on > > autoCommit. I use a value of five minutes (300000) ... and my server is > > NOT very busy most of the time. A commit with openSearcher set to false > > is relatively fast, but it still has somewhat heavy CPU, memory, and > > disk I/O resource requirements. > > > > You have autoSoftCommit set to happen after five seconds. If updates > > happen frequently or run for very long, this is potentially a LOT of > > committing and opening new searchers. I guess it's better than trying > > for one second, but anything more frequent than once a minute is likely > > to get you into trouble unless the system load is extremely light ... > > but as already discussed, your system load is probably not light. > > > > For the kind of Near Real Time setup you have mentioned, where you want > > to do one or more updates, commit, and then query for the changes, you > > probably should completely remove autoSoftCommit from the config and > > *only* open new searchers with explicit soft commits. Let autoCommit > > (with a maxTime of 1 to 5 minutes) handle durability concerns. > > > > A lot of pieces in your config file are set to depend on java system > > properties just like the example does, but since we do not know what > > system properties have been set, we can't tell for sure what those parts > > of the config are doing. > > > > Thanks, > > Shawn > >
