Thanks guys for the quick responses. I need to take the suggestions,
incorporate them, figure out how is that we are doing the fetching etc and
reply back on this post. The suggestions have been very helpful in taking this
forward for us here.
Thanks
-Peri.S
> On Dec 22, 2014, at 10:32 AM, Er
Just to pile on
_very_ frequently in my experience the problem
is not Solr at all, but acquiring the data in the
first place, i.e. often executing the DB query.
A very simple test is (in the SolrJ world) just comment
out the server.add(doclist).
Assuming you're using SolrJ, you _are_ indexin
What your indexer is build on? Do you use SolrJ, just REST, or
DataImportHandler? What's you DB schema is briefly?
Frankly speaking, there are few approaches to handle indexing concurrently,
details depends on the details mentioned above.
On Mon, Dec 22, 2014 at 5:54 PM, Peri Subrahmanya <
peri.su
Hi Peri,
You can always send concurrent update requests to solr.
Usually data acquisition takes more time than indexing time. You can dump your
db record into several csv files and you can feed them to solr in parallel.
Ahmet
On Monday, December 22, 2014 4:55 PM, Peri Subrahmanya
wrote:
H
Can you tell more about "You can index from a MapReduce job "? I use
nutch and it says Solr to index and reindex. I know that I can use Map
Reduce jobs at nutch side however can I use Map Reduce jobs at Solr side
(i.e for indexing etc.)?
2013/3/29 Otis Gospodnetic
> Yes. You can index from
Yes. You can index from any app that can hit SOlr with multiple
threads. You can use StreamingUpdateSolrServer, at least in older
Solrs, to handle multi-threading for you. You can index from a
MapReduce job
Otis
--
Solr & ElasticSearch Support
http://sematext.com/
On Fri, Mar 29, 2013
On 29 March 2013 14:56, Furkan KAMACI wrote:
> Does Solr allows parallelism (parallel computing) for indexing?
What do you mean by parallel computing in this context?
Solr can use multiple threads for indexing if that is what
you are asking.
Regards,
Gora
There is an open issue somewhere for this type of support. We don't have a
simple way to do it currently.
We also will be looking at adding index alias', which is probably another
feature you could use to solve this.
Currently, you would need some kind of load balancer to achieve this nicely I
You could try to isolate the bottleneck by testing the indexing speed
from the local machine hosting Solr. Also tools like iostat or sar
might give you more details about the disk side.
Yes, I am doing different stuff to isolate bottleneck. Im also profiling
JVM. And I am using iostat, top a
On Mon, Feb 6, 2012 at 5:55 PM, Per Steffensen wrote:
> Sami Siren skrev:
>
>> On Mon, Feb 6, 2012 at 2:53 PM, Per Steffensen
>> wrote:
>>
>>
>>
>>>
>>> Actually right now, I am trying to find our what my bottleneck is. The
>>> setup
>>> is more complex, than I would bother you with, but basicall
. I've had recurring discussions with "executive level folks" that no
matter how many VMs you host on a machine, and no matter how big that
machine is, there really, truly, *is* some hardware underlying it all that
really, truly, *does* have some limits.
And adding more VMs doesn't somehow get aro
Sami Siren skrev:
On Mon, Feb 6, 2012 at 2:53 PM, Per Steffensen wrote:
Actually right now, I am trying to find our what my bottleneck is. The setup
is more complex, than I would bother you with, but basically I have servers
with 80-90% IO-wait and only 5-10% "real CPU usage". It might not
So SolrJ with CommonsHttpSolrServer will not support handling several
requests concurrently?
Nope. Use StreamingUpdateSolrServer, it should be just a drop-in with
a different constructor.
I will try to do that. It is a little bit difficult for me, as we are
actually not dealing with
On Mon, Feb 6, 2012 at 2:53 PM, Per Steffensen wrote:
> Actually right now, I am trying to find our what my bottleneck is. The setup
> is more complex, than I would bother you with, but basically I have servers
> with 80-90% IO-wait and only 5-10% "real CPU usage". It might not be a
> Solr-relat
Right. See below.
On Mon, Feb 6, 2012 at 7:53 AM, Per Steffensen wrote:
> See response below
>
> Erick Erickson skrev:
>
>> Unfortunately, the answer is "it depends(tm)".
>>
>> First question: How are you indexing things? SolrJ? post.jar?
>>
>
> SolrJ, CommonsHttpSolrServer
>
>> But some observat
See response below
Erick Erickson skrev:
Unfortunately, the answer is "it depends(tm)".
First question: How are you indexing things? SolrJ? post.jar?
SolrJ, CommonsHttpSolrServer
But some observations:
1> sure, using multiple cores will have some parallelism. So will
using a single co
Unfortunately, the answer is "it depends(tm)".
First question: How are you indexing things? SolrJ? post.jar?
But some observations:
1> sure, using multiple cores will have some parallelism. So will
using a single core but using something like SolrJ and
StreamingUpdateSolrServer. Especial
17 matches
Mail list logo
