Hi all,
Thanks for your replies,
@dorian : the requirement is, we are showing a list of entries on a page.
For each user there's a read / unread flag. The data for listing is
fetched from solr. And you can see the entry was previously read or not. So
when a user views an entry by clicking. We are updating the database flag
to READ and use real time indexing to update solr entry. So when the user
close the full view of the entry and go back to entry listing page, the
data fetched from solr should be updated to READ. That's the use case we
are trying to fix.
@eric : thanks for the lengthy reply. So let's say I increase the
autosoftcommit time out to may be 100 ms. In that case do I have to wait
much that time from client side before calling search ?. What's the
correct way of achieving this?
Regards,
Lasitha
On 18 Dec 2016 23:52, "Erick Erickson" <erickerick...@gmail.com> wrote:
> 1 ms autocommit is far too frequent. And it's not
> helping you anyway.
>
> There is some lag between when a commit happens
> and when the docs are really available. The sequence is:
> 1> commit (soft or hard-with-opensearcher=true doesn't matter).
> 2> a new searcher is opened and autowarming starts
> 3> until the new searcher is opened, queries continue to be served by
> the old searcher
> 4> the new searcher is fully opened
> 5> _new_ requests are served by the new searcher.
> 6> the last request is finished by the old searcher and it's closed.
>
> So what's probably happening is that you send docs and then send a
> query and Solr is still in step <3>. You can look at your admin UI
> pluginst/stats page or your log to see how long it takes for a
> searcher to open and adjust your expectations accordingly.
>
> If you want to fetch only the document (not try to get it by a
> search), Real Time Get is designed to insure that you always get the
> most recent copy whether it's searchable or not.
>
> All that said, Solr wasn't designed for autocommits that are that
> frequent. That's why the documentation talks about _Near_ Real Time.
> You may need to adjust your expectations.
>
> Best,
> Erick
>
> On Sun, Dec 18, 2016 at 6:49 AM, Dorian Hoxha <dorian.ho...@gmail.com>
> wrote:
> > There's a very high probability that you're using the wrong tool for the
> > job if you need 1ms softCommit time. Especially when you always need it
> (ex
> > there are apps where you need commit-after-insert very rarely).
> >
> > So explain what you're using it for ?
> >
> > On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Wattaladeniya <
> watt...@gmail.com>
> > wrote:
> >
> >> Hi Furkan,
> >>
> >> Thanks for the links. I had read the first one but not the second one. I
> >> did read it after you sent. So in my current solrconfig.xml settings
> below
> >> are the configurations,
> >>
> >> <autoSoftCommit>
> >> <maxTime>${solr.autoSoftCommit.maxTime:1}</maxTime>
> >> </autoSoftCommit>
> >>
> >>
> >> <autoCommit>
> >> <maxTime>15000</maxTime>
> >> <openSearcher>false</openSearcher>
> >> </autoCommit>
> >>
> >> The problem i'm facing is, just after adding the documents to solr using
> >> solrj, when I retrieve data from solr I am not getting the updated
> results.
> >> This happens time to time. Most of the time I get the correct data but
> in
> >> some occasions I get wrong results. so as you suggest, what the best
> >> practice to use here ? , should I wait 1 mili second before calling for
> >> updated results ?
> >>
> >> Regards,
> >> Lasitha
> >>
> >> Lasitha Wattaladeniya
> >> Software Engineer
> >>
> >> Mobile : +6593896893
> >> Blog : techreadme.blogspot.com
> >>
> >> On Sun, Dec 18, 2016 at 8:46 PM, Furkan KAMACI <furkankam...@gmail.com>
> >> wrote:
> >>
> >> > Hi Lasitha,
> >> >
> >> > First of all, did you check these:
> >> >
> >> > https://cwiki.apache.org/confluence/display/solr/Near+
> >> Real+Time+Searching
> >> > https://lucidworks.com/blog/2013/08/23/understanding-
> >> > transaction-logs-softcommit-and-commit-in-sorlcloud/
> >> >
> >> > after that, if you cannot adjust your configuration you can give more
> >> > information and we can find a solution.
> >> >
> >> > Kind Regards,
> >> > Furkan KAMACI
> >> >
> >> > On Sun, Dec 18, 2016 at 2:28 PM, Lasitha Wattaladeniya <
> >> watt...@gmail.com>
> >> > wrote:
> >> >
> >> >> Hi furkan,
> >> >>
> >> >> Thanks for your reply, it is generally a query heavy system. We are
> >> using
> >> >> realtime indexing for editing the available data
> >> >>
> >> >> Regards,
> >> >> Lasitha
> >> >>
> >> >> Lasitha Wattaladeniya
> >> >> Software Engineer
> >> >>
> >> >> Mobile : +6593896893 <+65%209389%206893>
> >> >> Blog : techreadme.blogspot.com
> >> >>
> >> >> On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI <
> furkankam...@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Hi Lasitha,
> >> >>>
> >> >>> What is your indexing / querying requirements. Do you have an index
> >> >>> heavy/light - query heavy/light system?
> >> >>>
> >> >>> Kind Regards,
> >> >>> Furkan KAMACI
> >> >>>
> >> >>> On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya <
> >> >>> watt...@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>> > Hello devs,
> >> >>> >
> >> >>> > I'm here with another problem i'm facing. I'm trying to do a
> commit
> >> >>> (soft
> >> >>> > commit) through solrj and just after the commit, retrieve the data
> >> from
> >> >>> > solr (requirement is to get updated data list).
> >> >>> >
> >> >>> > I'm using soft commit instead of the hard commit, is previously I
> got
> >> >>> an
> >> >>> > error "Exceeded limit of maxWarmingSearchers=2, try again later"
> >> >>> because of
> >> >>> > too many commit requests. Now I have removed the explicit commit
> and
> >> >>> has
> >> >>> > let the solr to do the commit using autoSoftCommit *(1 mili
> second)*
> >> >>> and
> >> >>> > autoCommit *(30 seconds)* configurations. Now I'm not getting any
> >> >>> errors
> >> >>> > when i'm committing frequently.
> >> >>> >
> >> >>> > The problem i'm facing now is, I'm not getting the updated data
> when
> >> I
> >> >>> > fetch from solr just after the soft commit. So in this case what
> are
> >> >>> the
> >> >>> > best practices to use ? to wait 1 mili second before retrieving
> data
> >> >>> after
> >> >>> > soft commit ? I don't feel like waiting from client side is a good
> >> >>> option.
> >> >>> > Please give me some help from your expert knowledge
> >> >>> >
> >> >>> > Best regards,
> >> >>> > Lasitha Wattaladeniya
> >> >>> > Software Engineer
> >> >>> >
> >> >>> > Mobile : +6593896893
> >> >>> > Blog : techreadme.blogspot.com
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
>
