Re: SolrCloud Scale Struggle

2014-08-10 Thread anand.mahajan
Hello all, Thank you for your suggestions. With the autoCommit (every 10 mins) and softCommit (every 10 secs) frequencies reduced things work much better now. The CPU usages has gone down considerably too (by about 60%) and the read/write throughput is showing considerable improvements too. Ther

Re: what os env you use to develop lucene or solr?

2014-08-10 Thread Shawn Heisey
On 8/10/2014 7:49 PM, rulinma wrote: > I want know this, if linux is the best choosen? and Doug Cutting use what, > and centos or ubuntu or others, even mac? My clients are all Windows, my servers are all Linux. I've used Linux clients, but I do enough stuff with my computers that is either har

Re: SolrCloud Scale Struggle

2014-08-10 Thread rulinma
should not autoCommit openSearcher too freq. 360 true 1000 100 1 -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Scale-Struggle-tp4150592p4152229.html Sent from the Solr - User mailing list archive

RE: Content-Charset header in HttpSolrServer

2014-08-10 Thread Michael Ryan
Done. https://issues.apache.org/jira/browse/SOLR-6360 -Michael -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: Wednesday, August 06, 2014 7:55 PM To: solr-user@lucene.apache.org Subject: Re: Content-Charset header in HttpSolrServer : I was reviewing the

Re: Disabling transaction logs

2014-08-10 Thread rulinma
good. -- View this message in context: http://lucene.472066.n3.nabble.com/Disabling-transaction-logs-tp4151721p415.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Delta Import - Cleaning Index

2014-08-10 Thread rulinma
good. -- View this message in context: http://lucene.472066.n3.nabble.com/Delta-Import-Cleaning-Index-tp4151217p4152221.html Sent from the Solr - User mailing list archive at Nabble.com.

what os env you use to develop lucene or solr?

2014-08-10 Thread rulinma
HI everybody, I want know this, if linux is the best choosen? and Doug Cutting use what, and centos or ubuntu or others, even mac? thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/what-os-env-you-use-to-develop-lucene-or-solr-tp4152219.html Sent from the Solr

Re: How to grab matching stats in Similarity class

2014-08-10 Thread Hafiz Mian M Hamid
Follow-up, hoping someone will help. On Wednesday, August 6, 2014 5:40 PM, Hafiz Mian M Hamid wrote: We're using solr 4.2.1 and use an extension of Lucene's DefaultSimilarity as our similarity class. I am trying to figure out how we could get hold of the matching stats (i.e. how many/whic

Re: How can I request a big list of values ?

2014-08-10 Thread Jack Krupansky
The issue is not whether or how to do a massive request, but to recognize that a single massive request across the network is very clearly an anti-pattern for modern distributed systems. Instead of searching for ways to do something "bad", it is better to figure out how to exploit the positive

Re: How can I request a big list of values ?

2014-08-10 Thread Jack Krupansky
Not safe? In what way? It might be nice to have a specialized SolrJ API for this particular kind of request, so the API can do the merge. Maybe do it as a class so that you could have a method that gets invoked as documents trickle back from the various requests, again so that it is not a mass

Re: How can I request a big list of values ?

2014-08-10 Thread Bruno Mannina
Hi Anshum, I can do it with 3.6 release no ? my main problem, it's that I have around 2000 values, so I can't use one request with these values, it's too wide. :'( I will take a look to generate (like Jack proposes me) several requests, but even in this case it seems to be not safe... Le 1

Re: How can I request a big list of values ?

2014-08-10 Thread Bruno Mannina
Hi Jack, ok but for 2000 values, it means that I must do 40 requests if I choose to have 50 values by requests :'( and in my case, user can choose about 8 topics, so it can generate 8 times 40 requests... humm... is it not possible to send a text, json, xml file ? Le 10/08/2014 17:38, Jack K

Re: How can I request a big list of values ?

2014-08-10 Thread Anshum Gupta
Hi Bruno, If you would have been on a more recent release, https://issues.apache.org/jira/browse/SOLR-6318 would have come in handy perhaps. You might want to look at patching your version with this though (as a work around). On Sat, Aug 9, 2014 at 4:18 PM, Bruno Mannina wrote: > Hi All, > > I'm

Re: How can I request a big list of values ?

2014-08-10 Thread Jack Krupansky
Generally, "large requests" are an anti-pattern in modern distributed systems. Better to have a number of smaller requests executing in parallel and then merge the results in the application layer. -- Jack Krupansky -Original Message- From: Bruno Mannina Sent: Saturday, August 9, 201

Re: SqlEntityProcessor

2014-08-10 Thread Alexandre Rafalovitch
Custom UpdateRequestProcessor that collects the IDs of submitted documents and updates the database on commit. DataImportHandler itself is strictly one-way read operation. But you can add URP chain after it. Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources an

SqlEntityProcessor

2014-08-10 Thread Christof Lorenz
Hi folks, i am searching for a way to update a certain column in the rdbms for each item as soon as the item was indexed by solr. The column will be the indicator in the delta-query to select un-indexed items. We don't want to use the timestamp based mechanism that is default. Any ideas how we c