1 ms autocommit is far too frequent. And it's not
helping you anyway.
There is some lag between when a commit happens
and when the docs are really available. The sequence is:
1> commit (soft or hard-with-opensearcher=true doesn't matter).
2> a new searcher is opened and autowarming starts
3> until the new searcher is opened, queries continue to be served by
the old searcher
4> the new searcher is fully opened
5> _new_ requests are served by the new searcher.
6> the last request is finished by the old searcher and it's closed.
So what's probably happening is that you send docs and then send a
query and Solr is still in step <3>. You can look at your admin UI
pluginst/stats page or your log to see how long it takes for a
searcher to open and adjust your expectations accordingly.
If you want to fetch only the document (not try to get it by a
search), Real Time Get is designed to insure that you always get the
most recent copy whether it's searchable or not.
All that said, Solr wasn't designed for autocommits that are that
frequent. That's why the documentation talks about _Near_ Real Time.
You may need to adjust your expectations.
Best,
Erick
On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com> wrote:
> There's a very high probability that you're using the wrong tool for the
> job if you need 1ms softCommit time. Especially when you always need it (ex
> there are apps where you need commit-after-insert very rarely).
>
> So explain what you're using it for ?
>
> On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <watt...@gmail.com>
> wrote:
>
>> Hi Furkan,
>>
>> Thanks for the links. I had read the first one but not the second one. I
>> did read it after you sent. So in my current solrconfig.xml settings below
>> are the configurations,
>>
>> <autoSoftCommit>
>> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
>> </autoSoftCommit>
>>
>>
>> <autoCommit>
>> <maxTime>15000</maxTime>
>> <openSearcher>false</openSearcher>
>> </autoCommit>
>>
>> The problem i'm facing is, just after adding the documents to solr using
>> solrj, when I retrieve data from solr I am not getting the updated results.
>> This happens time to time. Most of the time I get the correct data but in
>> some occasions I get wrong results. so as you suggest, what the best
>> practice to use here ? , should I wait 1 mili second before calling for
>> updated results ?
>>
>> Regards,
>> Lasitha
>>
>> Lasitha Wattaladeniya
>> Software Engineer
>>
>> Mobile : +6593896893
>> Blog : techreadme.blogspot.com
>>
>> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com>
>> wrote:
>>
>> > Hi Lasitha,
>> >
>> > First of all, did you check these:
>> >
>> > https://cwiki.apache.org/confluence/display/solr/Near+
>> Real+Time+Searching
>> > https://lucidworks.com/blog/2013/08/23/understanding-
>> > transaction-logs-softcommit-and-commit-in-sorlcloud/
>> >
>> > after that, if you cannot adjust your configuration you can give more
>> > information and we can find a solution.
>> >
>> > Kind Regards,
>> > Furkan KAMACI
>> >
>> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
>> watt...@gmail.com>
>> > wrote:
>> >
>> >> Hi furkan,
>> >>
>> >> Thanks for your reply, it is generally a query heavy system. We are
>> using
>> >> realtime indexing for editing the available data
>> >>
>> >> Regards,
>> >> Lasitha
>> >>
>> >> Lasitha Wattaladeniya
>> >> Software Engineer
>> >>
>> >> Mobile : +6593896893 <+65%209389%206893>
>> >> Blog : techreadme.blogspot.com
>> >>
>> >> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <furkankam...@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi Lasitha,
>> >>>
>> >>> What is your indexing / querying requirements. Do you have an index
>> >>> heavy/light - query heavy/light system?
>> >>>
>> >>> Kind Regards,
>> >>> Furkan KAMACI
>> >>>
>> >>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
>> >>> watt...@gmail.com>
>> >>> wrote:
>> >>>
>> >>> > Hello devs,
>> >>> >
>> >>> > I'm here with another problem i'm facing. I'm trying to do a commit
>> >>> (soft
>> >>> > commit) through solrj and just after the commit, retrieve the data
>> from
>> >>> > solr (requirement is to get updated data list).
>> >>> >
>> >>> > I'm using soft commit instead of the hard commit, is previously I got
>> >>> an
>> >>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
>> >>> because of
>> >>> > too many commit requests. Now I have removed the explicit commit and
>> >>> has
>> >>> > let the solr to do the commit using autoSoftCommit *(1 mili second)*
>> >>> and
>> >>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
>> >>> errors
>> >>> > when i'm committing frequently.
>> >>> >
>> >>> > The problem i'm facing now is, I'm not getting the updated data when
>> I
>> >>> > fetch from solr just after the soft commit. So in this case what are
>> >>> the
>> >>> > best practices to use ? to wait 1 mili second before retrieving data
>> >>> after
>> >>> > soft commit ? I don't feel like waiting from client side is a good
>> >>> option.
>> >>> > Please give me some help from your expert knowledge
>> >>> >
>> >>> > Best regards,
>> >>> > Lasitha Wattaladeniya
>> >>> > Software Engineer
>> >>> >
>> >>> > Mobile : +6593896893
>> >>> > Blog : techreadme.blogspot.com
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>
