Re: SolrCloud distributed indexing

2014-06-03 Thread rulinma
use one is ok. solrCloud will route it, but use cloudserver is a good choice. -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-distributed-indexing-tp4138600p4139700.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud distributed indexing

2014-05-29 Thread Shalin Shekhar Mangar
If you are using Java to index/query, then use CloudSolrServer which accepts the ZooKeeper connection string as a constructor parameter and it will take care of routing requests and failover. On Thu, May 29, 2014 at 2:41 PM, Priti Solanki wrote: > Hi, > > How to achieve distributed in

SolrCloud distributed indexing

2014-05-29 Thread Priti Solanki
Hi, How to achieve distributed indexing in solr cloud.I have external Zookeeper with two separate machines acting as leader. In researching further I found As of now we are specifying the port id in our "update" call and if the leader is down zookeeper do not forward the request to ot

Re: Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Edd Grant
Aah I see - very useful. Thanks! On 3 May 2013 15:49, Shawn Heisey wrote: > On 5/3/2013 8:35 AM, Edd Grant wrote: > > Thanks, that's exactly what I was worried about. If I take your suggested > > approach of using SolrCloudServer and the feeder learns which shard > leader > > to target, then if

Re: Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Shawn Heisey
On 5/3/2013 8:35 AM, Edd Grant wrote: > Thanks, that's exactly what I was worried about. If I take your suggested > approach of using SolrCloudServer and the feeder learns which shard leader > to target, then if the shard leader goes down midway through indexing then > I've lost my ability to index

Re: Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Edd Grant
2013 13:17, Furkan KAMACI wrote: > > > > > Do you use CloudSolrServer when you push documnts into SolrCloud to be > > > indexed? > > > > > > 2013/5/3 Edd Grant > > > > > > > Hi all, > > > > > > > > I have been pl

Re: Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Furkan KAMACI
y away. > > Cheers, > > Edd > > > On 3 May 2013 13:17, Furkan KAMACI wrote: > > > Do you use CloudSolrServer when you push documnts into SolrCloud to be > > indexed? > > > > 2013/5/3 Edd Grant > > > > > Hi all, > > > > >

Re: Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Edd Grant
xed? > > 2013/5/3 Edd Grant > > > Hi all, > > > > I have been playing with Solr Cloud recently and am enjoying the > > distributed indexing capability. > > > > At the moment my SolrCloud consists of 2 leaders and 2 replicas which are > > fronted by an

Re: Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Furkan KAMACI
Do you use CloudSolrServer when you push documnts into SolrCloud to be indexed? 2013/5/3 Edd Grant > Hi all, > > I have been playing with Solr Cloud recently and am enjoying the > distributed indexing capability. > > At the moment my SolrCloud consists of 2 leaders and 2

Performance considerations when using distributed indexing + loadbalancing with Solr cloud

2013-05-03 Thread Edd Grant
Hi all, I have been playing with Solr Cloud recently and am enjoying the distributed indexing capability. At the moment my SolrCloud consists of 2 leaders and 2 replicas which are fronted by an HAProxy instance. I want to maximise performance for indexing and it occurred to me that the model I

Re: SolrCloud non-distributed indexing (update.chain)

2012-06-18 Thread Mark Miller
chain=notdistributed > > I tried update.distrib=NONE. The indexing still being distributed and > ignoring the update.chain (as specified below in Solr config). > > > > > > > How do I get the above chain and non-distrib

SolrCloud non-distributed indexing (update.chain)

2012-06-18 Thread Boon Low
E. The indexing still being distributed and ignoring the update.chain (as specified below in Solr config). How do I get the above chain and non-distributed indexing to work again? Regards, Boon - Boon Low Search UX and Engine Developer (SO

Re: Distributed indexing

2011-11-11 Thread Rafał Kuć
Hello! Thanks for the answer :) -- Regards, Rafał Kuć > On Nov 9, 2011, at 12:52 PM, Rafał Kuć wrote: >> Hello! >> >> I was looking for a way to implement distributed indexing in Solr. >> From looking at the https://issues.apache.org/jira/browse/SOLR-2358 >

Re: Distributed indexing

2011-11-11 Thread Mark Miller
On Nov 9, 2011, at 12:52 PM, Rafał Kuć wrote: > Hello! > > I was looking for a way to implement distributed indexing in Solr. > From looking at the https://issues.apache.org/jira/browse/SOLR-2358 > there was some work done to enable Solr to distribute the documents to > shard

Re: Distributed indexing

2011-11-11 Thread Mark Miller
>> From: Erick Erickson >> To: solr-user@lucene.apache.org >> Sent: Friday, November 11, 2011 8:47 AM >> Subject: Re: Distributed indexing >> >> As you can see, there are a lot of related issues. Solr Cloud >> is where this act

Re: Distributed indexing

2011-11-11 Thread Otis Gospodnetic
Nutch Lucene ecosystem search :: http://search-lucene.com/ > >From: Erick Erickson >To: solr-user@lucene.apache.org >Sent: Friday, November 11, 2011 8:47 AM >Subject: Re: Distributed indexing > >As you can see, there are a lot of related issues.

Re: Distributed indexing

2011-11-11 Thread Erick Erickson
se date for all this is uncertain Best Erick 2011/11/9 Rafał Kuć : > Hello! > > I was looking for a way to implement distributed indexing in Solr. > From looking at the https://issues.apache.org/jira/browse/SOLR-2358 > there was some work done to enable Solr to distribute the doc

Distributed indexing

2011-11-09 Thread Rafał Kuć
Hello! I was looking for a way to implement distributed indexing in Solr. From looking at the https://issues.apache.org/jira/browse/SOLR-2358 there was some work done to enable Solr to distribute the documents to shards without the need of 3rd party software before Solr. What I would like to know

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-09 Thread Yonik Seeley
On Thu, Sep 9, 2010 at 11:51 AM, Grant Ingersoll wrote: > On Sep 6, 2010, at 10:41 AM, Yonik Seeley wrote: >> For SolrCloud, I don't think we'll end up using consistent hashing - >> we don't need it (although some of the concepts may still be useful). > > Can you elaborate on why we don't need it?

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-09 Thread Grant Ingersoll
On Sep 6, 2010, at 10:41 AM, Yonik Seeley wrote: > > For SolrCloud, I don't think we'll end up using consistent hashing - > we don't need it (although some of the concepts may still be useful). Can you elaborate on why we don't need it?

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-07 Thread MitchK
I must add something to my last post: When saying it could be used together with techniques like consistent hashing, I mean it could be used at indexing time for indexing documents, since I assumed that the number of shards does not change frequently and therefore an ODV-case becomes relatively i

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-07 Thread MitchK
What if we do not care about the version of a document at index-time? When it comes to distributed search, we currently decide aggregating documents based on their uniqueKey. But what would be, if we decide additionally decide on uniqueKey plus indexingDate, so that we only aggregate the last ind

RE: Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Dennis Gearon
e: > From: Markus Jelsma > Subject: RE: Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?) > To: solr-user@lucene.apache.org > Date: Monday, September 6, 2010, 2:53 PM > The remainder of an arithmetic > division > > http://en.wikipedia.org/wiki/Modulo_

RE: Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Markus Jelsma
The remainder of an arithmetic division http://en.wikipedia.org/wiki/Modulo_operation -Original message- From: Dennis Gearon Sent: Mon 06-09-2010 22:04 To: solr-user@lucene.apache.org; Subject: Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?) What is a 'simpl

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Andrzej Bialecki
On 2010-09-06 22:03, Dennis Gearon wrote: What is a 'simple MOD'? md5(docId) % numShards -- Best regards, Andrzej Bialecki <>< ___. ___ ___ ___ _ _ __ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Dennis Gearon
gt; Subject: Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?) > To: solr-user@lucene.apache.org > Date: Monday, September 6, 2010, 11:30 AM > On 2010-09-06 16:41, Yonik Seeley > wrote: > > On Mon, Sep 6, 2010 at 10:18 AM, MitchK  > wrote: > > [...consistent has

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Andrzej Bialecki
On 2010-09-06 16:41, Yonik Seeley wrote: On Mon, Sep 6, 2010 at 10:18 AM, MitchK wrote: [...consistent hashing...] But it doesn't solve the problem at all, correct me if I am wrong, but: If you add a new server, let's call him IP3-1, and IP3-1 is nearer to the current ressource X, than doc x wi

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Yonik Seeley
On Mon, Sep 6, 2010 at 10:18 AM, MitchK wrote: [...consistent hashing...] > But it doesn't solve the problem at all, correct me if I am wrong, but: If > you add a new server, let's call him IP3-1, and IP3-1 is nearer to the > current ressource X, than doc x will be indexed at IP3-1 - even if IP2-1

Re: SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread MitchK
Andrzej, thank you for sharing your experiences. > b) use consistent hashing as the mapping schema to assign documents to a > changing number of shards. There are many explanations of this schema on > the net, here's one that is very simple: > Boom. With the given explanation, I understan

SolrCloud distributed indexing (Re: anyone use hadoop+solr?)

2010-09-06 Thread Andrzej Bialecki
(I adjusted the subject to better reflect the content of this discussion). On 2010-09-06 14:37, MitchK wrote: Thanks for your detailed feedback Andzej! From what I understood, SOLR-1301 becomes obsolete ones Solr becomes cloud-ready, right? Who knows... I certainly didn't expect this code

RE: Distributed Indexing

2010-07-08 Thread Yuval Feinstein
uniformly to different shards. -- Yuval -Original Message- From: Li Li [mailto:fancye...@gmail.com] Sent: Thursday, July 08, 2010 2:44 PM To: solr-user@lucene.apache.org Subject: Distributed Indexing Is there any tools for "Distributed Indexing"? It refers to KattaIntegr

Distributed Indexing

2010-07-08 Thread Li Li
Is there any tools for "Distributed Indexing"? It refers to KattaIntegration and ZooKeeperIntegration in http://wiki.apache.org/solr/DistributedSearch. But it seems that they concern more on error processing and replication. I need a dispatcher that dispatch different docs by