Re: Auto Commit and Hard Commit being differentiated

2015-02-22 Thread Shawn Heisey
On 2/22/2015 3:01 PM, Deepak Vohra wrote: > In the SOLR-7133 issue hard commit and auto commit are being differentiated, > but aren't they the same? Doesn't the following setting apply a hard commit? > > ${solr.autoCommit.maxTime:15000} > false > > https://issues.apache.org/jira/browse/S

Re: Auto Commit and Hard Commit being differentiated

2015-02-22 Thread Erick Erickson
No. This has openSearcher set to false, which means that the index segment will be closed, the docs are not visible for search. Please take the time, as Shawn suggested on the JIRA, to read this blog. https://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Auto Commit and Hard Commit being differentiated

2015-02-22 Thread Deepak Vohra
In the SOLR-7133 issue hard commit and auto commit are being differentiated, but aren't they the same? Doesn't the following setting apply a hard commit? ${solr.autoCommit.maxTime:15000} false https://issues.apache.org/jira/browse/SOLR-7133

Re: ignoring bad documents during index

2015-02-22 Thread SolrUser1543
What i tried is to make an update processor , with try / catch inside of ProcessAdd. This update processor was the last one last in update chain . in catch statement I tried to add to response the id of failed item . This information ( about failed items ) is lost somewhere when request redirected

Re: FW: NRTCachingDirectory threads stuck

2015-02-22 Thread Mikhail Khludnev
On Sun, Feb 22, 2015 at 1:54 PM, Moshe Recanati wrote: > Hi Mikhail, > Thank you. > 1. Regarding jetty threads - How I can reduce them? > https://wiki.eclipse.org/Jetty/Howto/High_Load#Thread_Pool note, you'll get 503 or something when pool size is exceeded. > 2. Is it related to the fact we'r

Re: ignoring bad documents during index

2015-02-22 Thread Mikhail Khludnev
Can you use CloudSolrServer to submit XMLs that ommit the intermediate relay, and might make simple to respond additional info. regarding experimenting with DistributingUpdateProcessor you can copy it and make its' factory implement DistributingUpdateProcessorFactory and add your processor factory

RE: FW: NRTCachingDirectory threads stuck

2015-02-22 Thread Moshe Recanati
Hi, Another question. We're using single Is it related? Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile  + 972-52-6194481 Skype    :  recanati More at:  www.kmslh.com | LinkedIn | FB -Original Message- From: Moshe Recanati Sent: Sunday, February 22, 2015 12:55

Re: ignoring bad documents during index

2015-02-22 Thread SolrUser1543
I'm not using a replicas. Does this class relevant anyway? Is there any way to not change this class ,but inherit it and do try catch on ProcessAdd? -- View this message in context: http://lucene.472066.n3.nabble.com/ignoring-bad-documents-during-index-tp4176947p4188008.html Sent from the So

Re: ignoring bad documents during index

2015-02-22 Thread SolrUser1543
We are working with the following configuration : There is Indexer service that prepares a bulk of xmls . Those XMLs received by a shard , which used only for distributing a request among a shards. ( let's call it GW) Some of shards could return OK, some 400 ( wrong field value ) some 500 ( becau

RE: FW: NRTCachingDirectory threads stuck

2015-02-22 Thread Moshe Recanati
Hi Mikhail, Thank you. 1. Regarding jetty threads - How I can reduce them? 2. Is it related to the fact we're running Solr 4.0 in parallel on this machine? Thank you Regards, Moshe Recanati SVP Engineering Office + 972-73-2617564 Mobile  + 972-52-6194481 Skype    :  recanati More at:  www.kmslh

Re: ignoring bad documents during index

2015-02-22 Thread Mikhail Khludnev
On Sun, Feb 22, 2015 at 12:20 PM, SolrUser1543 wrote: > Does anyone know where is it? local update on leader happens first (assuming you use CloudSolrServer), https://github.com/apache/lucene-solr/blob/trunk/solr/core/src/java/org/apache/solr/update/processor/DistributedUpdateProcessor.java#L70

Re: ignoring bad documents during index

2015-02-22 Thread SolrUser1543
I think , you did not understand the question. The problem is indexing via cloud. When one shard gets a request, distributes it among others and in case of error on one of them this information is not passed to request initiator. Does anyone know where is it? -- View this message in cont

Re: FW: NRTCachingDirectory threads stuck

2015-02-22 Thread Mikhail Khludnev
Hello, I checked 20020.tdump. From the update perspective, it's ok, I see the single thread committed and awaits for opening a searcher. There are a few very bad evidences: - there are many threads executing search requests in parallel. let;s say it's roughly hundred of them. This is dead end. Con