Re: Soft commit and reading data just after the commit

2016-12-20 Thread Shawn Heisey
On 12/19/2016 7:12 PM, Lasitha Wattaladeniya wrote: > *Requirement *is, we are showing a list of entries on a page. For each > user there's a read / unread flag. The data for listing is fetched > from solr. And you can see the entry was previously read or not. So > when a user views an entry by cli

Re: Soft commit and reading data just after the commit

2016-12-19 Thread Ere Maijala
Hi, so, the app already has a database connection because it updates the READ flag when the user clicks an entry, right? If you only need the flag for display purposes, it sounds like it would make sense to also fetch it directly from the database when displaying the listing. Of course if you

Re: Soft commit and reading data just after the commit

2016-12-19 Thread Walter Underwood
You probably need a database instead of a search engine. What requirement makes you want to do this with a search engine? wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 19, 2016, at 6:34 PM, Lasitha Wattaladeniya wrote: > > Hi Hendrik, > > T

Re: Soft commit and reading data just after the commit

2016-12-19 Thread Lasitha Wattaladeniya
Hi Hendrik, Thanks for your input. Previously I was using the hard commit (SolrClient.commit()) but then I got some error when there are concurrent real time index requests from my app. The error was "Exceeded limit of maxWarmingSearchers=2, try again later", then i changed the code to use only s

Re: Soft commit and reading data just after the commit

2016-12-19 Thread Lasitha Wattaladeniya
Hi Shawn, Thanks for your well detailed explanation. Now I understand, I won't be able to achieve the 100ms softcommit timeout with my hardware setup. However let's say someone has a requirement as below (quoted from my previous mail) *Requirement *is, we are showing a list of entries on a page.

Re: Soft commit and reading data just after the commit

2016-12-19 Thread Hendrik Haddorp
Hi, the SolrJ API has this method: SolrClient.commit(String collection, boolean waitFlush, boolean waitSearcher, boolean softCommit). My assumption so far was that when you set waitSearcher to true that the method call only returns once a search would find the new data, which sounds what you w

Re: Soft commit and reading data just after the commit

2016-12-19 Thread Shawn Heisey
On 12/18/2016 7:09 PM, Lasitha Wattaladeniya wrote: > @eric : thanks for the lengthy reply. So let's say I increase the > autosoftcommit time out to may be 100 ms. In that case do I have to > wait much that time from client side before calling search ?. What's > the correct way of achieving this?

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Lasitha Wattaladeniya
I didn't look much onto REALTIME GET handler. Thanks for mentioning that. I'm checking it now On 19 Dec 2016 10:09, "Lasitha Wattaladeniya" wrote: > Hi all, > > Thanks for your replies, > > @dorian : the requirement is, we are showing a list of entries on a page. > For each user there's a re

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Lasitha Wattaladeniya
Hi all, Thanks for your replies, @dorian : the requirement is, we are showing a list of entries on a page. For each user there's a read / unread flag. The data for listing is fetched from solr. And you can see the entry was previously read or not. So when a user views an entry by clicking. We

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Erick Erickson
1 ms autocommit is far too frequent. And it's not helping you anyway. There is some lag between when a commit happens and when the docs are really available. The sequence is: 1> commit (soft or hard-with-opensearcher=true doesn't matter). 2> a new searcher is opened and autowarming starts 3> until

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Dorian Hoxha
There's a very high probability that you're using the wrong tool for the job if you need 1ms softCommit time. Especially when you always need it (ex there are apps where you need commit-after-insert very rarely). So explain what you're using it for ? On Sun, Dec 18, 2016 at 3:38 PM, Lasitha Watta

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Lasitha Wattaladeniya
Hi Furkan, Thanks for the links. I had read the first one but not the second one. I did read it after you sent. So in my current solrconfig.xml settings below are the configurations, ${solr.autoSoftCommit.maxTime:1} 15000 false The problem i'm facing is, just after adding the do

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Furkan KAMACI
Hi Lasitha, First of all, did you check these: https://cwiki.apache.org/confluence/display/solr/Near+Real+Time+Searching https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ after that, if you cannot adjust your configuration you can give more

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Lasitha Wattaladeniya
Hi furkan, Thanks for your reply, it is generally a query heavy system. We are using realtime indexing for editing the available data Regards, Lasitha Lasitha Wattaladeniya Software Engineer Mobile : +6593896893 Blog : techreadme.blogspot.com On Sun, Dec 18, 2016 at 8:12 PM, Furkan KAMACI wro

Re: Soft commit and reading data just after the commit

2016-12-18 Thread Furkan KAMACI
Hi Lasitha, What is your indexing / querying requirements. Do you have an index heavy/light - query heavy/light system? Kind Regards, Furkan KAMACI On Sun, Dec 18, 2016 at 11:35 AM, Lasitha Wattaladeniya wrote: > Hello devs, > > I'm here with another problem i'm facing. I'm trying to do a com