Re: Master/Slave

2020-10-08 Thread Eric Pugh
o be supported for the >>> foreseeable future? Or are we forced to assume that Master/Slave support >>> will evaporate shortly after the now-evaporated CDCR support? >>> >>> -Original Message- >>> From: David Hastings >>> Sent: Wedne

Re: Master/Slave

2020-10-08 Thread Erick Erickson
r/Slave support >> will evaporate shortly after the now-evaporated CDCR support? >> >> -Original Message- >> From: David Hastings >> Sent: Wednesday, September 30, 2020 3:10 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Master/Slave >

Re: Master/Slave

2020-10-07 Thread Jan Høydahl
vid Hastings > Sent: Wednesday, September 30, 2020 3:10 PM > To: solr-user@lucene.apache.org > Subject: Re: Master/Slave > >> whether we should expect Master/Slave replication also to be deprecated > > it better not ever be depreciated. it has been the most reliable

RE: Master/Slave

2020-10-06 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
d CDCR support? -Original Message- From: David Hastings Sent: Wednesday, September 30, 2020 3:10 PM To: solr-user@lucene.apache.org Subject: Re: Master/Slave >whether we should expect Master/Slave replication also to be deprecated it better not ever be depreciated. it has been the most

Re: Master/Slave

2020-09-30 Thread Walter Underwood
We do this sort of thing outside of Solr. The indexing process includes creating a feed file with one JSON object per line. The feed files are stored in S3 with names that are ISO 8601 timestamps. Those files are picked up and loaded into Solr. Because S3 is cross-region in AWS, those files are als

Re: Master/Slave

2020-09-30 Thread David Hastings
>whether we should expect Master/Slave replication also to be deprecated it better not ever be depreciated. it has been the most reliable mechanism for its purpose, solr cloud isnt going to replace standalone, if it does, thats when I guess I stop upgrading or move to elastic On Wed, Sep 30, 202

Re: Master Slave Terminology

2020-06-17 Thread Walter Underwood
I’ve long thought that master/slave was not the right metaphor for a pull model anyway. We probably should not use “replica” since that already has a use in Solr Cloud. Where is the discussion? wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jun 16

Re: Master Slave Terminology

2020-06-17 Thread Doug Turnbull
+1 to name change. Also 'overseer' which doesn't go well with Master/Slave! On Wed, Jun 17, 2020 at 11:16 AM David Smiley wrote: > priv...@lucene.apache.org but it should have been public and expect it to > spill out to the dev list today. > > ~ David > > > On Wed, Jun 17, 2020 at 11:14 AM Mike

Re: Master Slave Terminology

2020-06-17 Thread David Smiley
priv...@lucene.apache.org but it should have been public and expect it to spill out to the dev list today. ~ David On Wed, Jun 17, 2020 at 11:14 AM Mike Drob wrote: > Hi Jan, > > Can you link to the discussion? I searched the dev list and didn’t see > anything, is it on slack or a jira or some

Re: Master Slave Terminology

2020-06-17 Thread Mike Drob
Hi Jan, Can you link to the discussion? I searched the dev list and didn’t see anything, is it on slack or a jira or somewhere else? Mike On Wed, Jun 17, 2020 at 1:51 AM Jan Høydahl wrote: > Hi Kaya, > > Thanks for bringing it up. The topic is already being discussed by > developers, so expect

Re: Master Slave Terminology

2020-06-16 Thread Jan Høydahl
Hi Kaya, Thanks for bringing it up. The topic is already being discussed by developers, so expect to see some change in this area; Not over-night, but incremental. Also, if you want to lend a helping hand, patches are more than welcome as always. Jan > 17. jun. 2020 kl. 04:22 skrev Kayak28 : >

Re: Master Slave Replication Issue

2018-11-12 Thread Erick Erickson
Hmmm, afraid I'm out of my depth, perhaps some of the Lucene folks can chime in. Sorry I can't be more help. Erick On Mon, Nov 12, 2018 at 12:27 AM damian.pawski wrote: > > Hi > > I had to re-create the index as some tokenizers are not longer supported on > the 7.x version. > I have a fresh 7.x

Re: Master Slave Replication Issue

2018-11-12 Thread damian.pawski
Hi I had to re-create the index as some tokenizers are not longer supported on the 7.x version. I have a fresh 7.x index. Thank you Damian -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Master Slave Replication Issue

2018-11-12 Thread damian.pawski
Hi I had to re-create the index, as some Tokenizers are no longer supported on 7.X, so I have a fresh 7.x index, but still having issues with the replication. Thank you Damian -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Master Slave Replication Issue

2018-11-09 Thread Erick Erickson
Damian: You say you've switched from 5x to 7x. Did you try to use an index created with 5x or did you index fresh with 7x? Solr/Lucene do not guarantee backward compatibility across more than one major version. Best, Erick On Fri, Nov 9, 2018 at 2:34 AM damian.pawski wrote: > > Hi, > We have swi

Re: Master Slave Replication Issue

2018-11-09 Thread damian.pawski
Hi, We have switched from 5.4 to 7.2.1 and we have started to see more issues with the replication. I think it may be related to the fact that a delta import was started during a full import (not the case for the Solr 5.4). I am getting below error: XXX: java.lang.IllegalArgumentException:java.l

Re: Master Slave Replication of Solr with Basic Authentication

2018-03-25 Thread Shawn Heisey
On 3/25/2018 1:56 AM, Basheeruddin Ahmed (syedbahm) wrote: Seems even when we use Secuirty.json with BasicAuthentication Plugin as documented here -- https://lucene.apache.org/solr/guide/7_2/basic-authentication-plugin.html , which nicely encrypts the user password using SHA256 encryption, whe

Re: Master Slave Replication Issue

2018-02-01 Thread den_nal
Done https://issues.apache.org/jira/browse/SOLR-11938. Thanks! -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Master Slave Replication Issue

2018-02-01 Thread Tomas Fernandez Lobbe
This seems pretty serious. Please create a Jira issue Sent from my iPhone > On Feb 1, 2018, at 12:15 AM, dennis nalog > wrote: > > Hi, > We are using Solr 7.1 and are solr setup is master-slave replication. > We encounter this issue that when we disable the replication in master via UI > or U

Re: master slave replication taking time

2017-06-29 Thread Erick Erickson
So you say that master/slave replication takes up to 20 seconds but it takes 8-10 minutes just to copy the entire index? Solr is _already_ speeding up your copy enormously by just copying changed segments. There's nothing magic you can do to make it faster. And if your master happens to merge many

Re: master slave replication taking time

2017-06-29 Thread Midas A
Erick, when we copy entire index it takes 8- 10 mins . On Wed, Jun 28, 2017 at 9:22 PM, Erick Erickson wrote: > How long it takes to copy the entire index from one machine to another > over your network. Solr can't go any faster than your network can > support. Consider SolrCloud if you need s

Re: master slave replication taking time

2017-06-28 Thread Erick Erickson
How long it takes to copy the entire index from one machine to another over your network. Solr can't go any faster than your network can support. Consider SolrCloud if you need something closer to NRT. Best, Erick On Tue, Jun 27, 2017 at 11:31 PM, Midas A wrote: > Hi, > > we have around 2000

RE: Master/Slave out of sync

2017-06-27 Thread Pouliot, Scott
[mailto:erickerick...@gmail.com] Sent: Tuesday, June 27, 2017 11:17 AM To: solr-user Subject: Re: Master/Slave out of sync First, attachments are almost always stripped by the mail program, so we can't see anything. Hmmm, does look odd. What happens if you issue a commit against the slave via a

Re: Master/Slave out of sync

2017-06-27 Thread Erick Erickson
First, attachments are almost always stripped by the mail program, so we can't see anything. Hmmm, does look odd. What happens if you issue a commit against the slave via a url? I.e. http://server:port/solr/core/update?commit=true? And what are the autocommit settings on the slave? Best, Erick

Re: Master Slave set up in Solr Cloud

2014-11-02 Thread Meraj A. Khan
Eirck, I meant having a replication factor of 1 in SolrCloud would not be scalable, I ended up using a replication factor of 1 , because of the disjoint replicas issue for replication factor greater than 1 , that I mentioned previously in the other email thread. Are you saying that a SolrCloud cl

Re: Master Slave set up in Solr Cloud

2014-11-02 Thread Erick Erickson
I _strongly_ urge you to try it before assuming it won't work. bq: It obviously seems will not be scalable due to the fact that the same core will be indexed and queried at the same time as this is a long running indexing task. What evidence do you have for this? SolrCloud is designed for this sc

Re: Master Slave set up in Solr Cloud

2014-11-02 Thread S.L
Resending this as I might have not been clear in my earlier query. I want to use SolrCloud for everything except the replication , is it possible to set up the master-slave configuration using different Solr instances and still be able to use the sharding feature provided by SolrCloud ? On Thu,

Re: Master-Slave setup using SolrCloud

2014-10-04 Thread Sachin Kale
Apparently, there is a bug in Solr 4.10.0 which was causing the NullPointerExceptions. SOLR-6501 We have updated our production SOLR to 4.10.1 On Thu, Oct 2, 2014 at 8:13 PM, Sachin Kale wrote: > If I look into the logs, many times I get only fo

Re: Master-Slave setup using SolrCloud

2014-10-02 Thread Sachin Kale
If I look into the logs, many times I get only following line without any stacktrace: *ERROR - 2014-10-02 19:35:25.516; org.apache.solr.common.SolrException; java.lang.NullPointerException* These exceptions are not coming continuously. Once in every 10-15 minutes. But once it starts, there are co

Re: Master-Slave setup using SolrCloud

2014-10-02 Thread Shawn Heisey
On 10/2/2014 6:58 AM, Sachin Kale wrote: > We are trying to move our traditional master-slave Solr configuration to > SolrCloud. As our index size is very small (around 1 GB), we are having > only one shard. > So basically, we are having same master-slave configuration with one leader > and 6 repli

Re: Master / Slave Set Up Documentation

2013-08-26 Thread Jared Griffith
Ha, I guess I didn't see that page listed in the Table of contents it's definitely Monday. Thanks. On Mon, Aug 26, 2013 at 10:36 AM, Andrea Gazzarini < andrea.gazzar...@gmail.com> wrote: > You mean this > > http://wiki.apache.org/solr/SolrReplication > > ? > > What's wrong with this page?

Re: Master / Slave Set Up Documentation

2013-08-26 Thread Andrea Gazzarini
You mean this http://wiki.apache.org/solr/SolrReplication ? What's wrong with this page? It seems clear. I'm widely using replication and the first time I set up a 1 master + 2 slaves by simply following that page On 26 Aug 2013 18:54, "Jared Griffith" wrote: > Hello, > I'm new to this Solr th

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-05-01 Thread Furkan KAMACI
Shawn thanks for the detailed answer. I have 5 shards and 1 leader - 1 replica for each. I mean I have 10 Solr nodes. When I look at admin gui of one of the shards leader I see that its replica has less MB of index than leader. I don't update the data, I don't index new ones. I think that after a t

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Shawn Heisey
On 4/30/2013 8:33 AM, Furkan KAMACI wrote: I think that replication occurs after commit by default. It has been long time however there is still mismatch between leader and replica (approximately 5 MB). I tried to pull indexes from leader but it is still same. My mail server has been down most

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI
It would be nice if I learn what is a follower means and how to define them (I know the example of replica but didn't see an example of follower yet) 2013/4/30 Michael Della Bitta > I could be getting this wrong, and the wiki is down at the moment, but I > think a replica can be a leader, wherea

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta
I could be getting this wrong, and the wiki is down at the moment, but I think a replica can be a leader, whereas a follower is definitely not. Michael Della Bitta Appinions 18 East 41st Street, 2nd Floor New York, NY 10017-6271 www.appinions.com

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI
If we talk about SolrCloud terminology does follower and replica means same? Is there any documentation at wiki for that? 2013/4/30 Michael Della Bitta > I'd say a follower is a participant in a shard that's not the leader. > > > Michael Della Bitta > > --

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta
I'd say a follower is a participant in a shard that's not the leader. Michael Della Bitta Appinions 18 East 41st Street, 2nd Floor New York, NY 10017-6271 www.appinions.com Where Influence Isn’t a Game On Tue, Apr 30, 2013 at 1:27 PM, Furkan K

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI
However I am using SolrCloud with 5 shards. Every leader has a replica. What do you mean with "followers"? 2013/4/30 Michael Della Bitta > Then there is no replication, and no slaves nor masters. There's a leader > and followers. Documents themselves are sent from the leader to followers, > not

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta
Then there is no replication, and no slaves nor masters. There's a leader and followers. Documents themselves are sent from the leader to followers, not cores or segments. You should not expect the bits on the disk across leaders and followers to be the same because of the reasons I mentioned alrea

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI
Shawn, why they don't have same data byte per byte? Can I force slave to pull them, I tried but didn't work. 2013/4/30 Furkan KAMACI > I use Solr 4.2.1 as SolrCloud > > > 2013/4/30 Michael Della Bitta > >> I'm a little confused. Are you using Solr Cloud, or ordinary replication? >> >> >> Michae

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI
I use Solr 4.2.1 as SolrCloud 2013/4/30 Michael Della Bitta > I'm a little confused. Are you using Solr Cloud, or ordinary replication? > > > Michael Della Bitta > > > Appinions > 18 East 41st Street, 2nd Floor > New York, NY 10017-6271 > > www.ap

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta
I'm a little confused. Are you using Solr Cloud, or ordinary replication? Michael Della Bitta Appinions 18 East 41st Street, 2nd Floor New York, NY 10017-6271 www.appinions.com Where Influence Isn’t a Game On Tue, Apr 30, 2013 at 10:33 AM, Fur

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI
I think that replication occurs after commit by default. It has been long time however there is still mismatch between leader and replica (approximately 5 MB). I tried to pull indexes from leader but it is still same. 2013/4/30 Michael Della Bitta > In Solr Cloud, commits can happen at different

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta
In Solr Cloud, commits can happen at different times across replicas. Which means merges also may happen at different times. So there's no expectation of the cores of different replicas being totally similar. Michael Della Bitta Appinions 18 East

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Shawn Heisey
On 4/30/2013 6:13 AM, Furkan KAMACI wrote: > I use SolrCloud, 4.2.1 of Solr. > > Here is a detail from my admin page: > > Replication (Slave) Version Gen Size > Master: 1367309548534 84 779.87 MB > Slave: 1367307649512 82 784.44 MB > > When I use command=abortfetch still file size are not same.

Re: Master slave replication with digest authentication

2013-04-17 Thread Shawn Heisey
On 4/17/2013 1:20 AM, Maciej Pestka wrote: Hi, I've configured basic authentication on tomcat & my slave solr instance and it works. Any idea how to configure slave to replicate properly with digest authentication? on Solr WIKI I could find only basic authentication example: http://wiki.apach

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Sujatha Arun
Thanks Otis . Yes true but considering that the Indexing is via a queue, there would actually be minimal load on the machine. And we are planning to replicate this set up via adding more machines when the server reaches about 80% capacity for adding more cores. Regards, Sujatha On Wed, Mar 6, 20

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Otis Gospodnetic
Hello, This is not recommended because people typically don't want the load from indexing affect queries/user experience. If your numbers are low, then this may not be a big deal. If you already need to create a core on 2 machines, creating it on 3 doesn't seem a big deal. There is a slight conf

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Sujatha Arun
Hi Otis,Michael, Thanks for your input and suggestions . Yes, we were considering the sticky session for pagination and we are not planning for having index on EBS I would like to understand why its not the recommended approach, can you please explain? Till now we were having a single server

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Michael Della Bitta
If your index is on EBS, you'll see big iowait percentages when merges happen. I'm not sure what that's going to do to your master's ability to service requests. You should test. Alternatively, you might figure out the size of machine you need to index vs. the size of machine you need to service

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Otis Gospodnetic
Hi, That my be fine. I'd use the sticky session setting in ELB to avoid having the same user's query hit both master and slave, say while paging through results, and risking seeing inconsistent results. THis will also help with cache utilization. This said, this is not a recommended setup.

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Sujatha Arun
Is there anything wrong with set up? On Tue, Mar 5, 2013 at 5:43 PM, Sujatha Arun wrote: > Hi Otis, > > Since currently we are planning for only one slave due to cost > considerations, can we have an ELB fronting the master and slave for HA. > >1. All index requests will go to the master .

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Sujatha Arun
Hi Otis, Since currently we are planning for only one slave due to cost considerations, can we have an ELB fronting the master and slave for HA. 1. All index requests will go to the master . 2. Slave replicates from master . 3. Search request can go either to master /slave via ELB. is

Re: Master /Slave Set up on AWS - 3.6.1

2013-03-05 Thread Otis Gospodnetic
Hi Sujatha, If I understand correctly, you will have only 1 slave (and 1 master), so that's not really a HA architecture. You could manually turn master into slave, but that's going to mean some down time... Otis -- Solr & ElasticSearch Support http://sematext.com/ On Tue, Mar 5, 2013 at 3:

Re: Master /Slave Architecture3.6.1

2013-01-10 Thread Upayavira
In the end, the best advice is "try it". You'll save the effort of indexing with this master/slave setup, but you'll still need to warm your caches on each slave, which is a reasonable portion of the work done on a commit. However, with a master/slave setup, you get the option to go to two slaves

Re: Master /Slave Architecture3.6.1

2013-01-10 Thread Sujatha Arun
Thanks,Otis.. But then what exactly is the advantage for a master slave architecture for multicore ,when replication has the same effect as that of a commit and if I am going to have worse performance by moving to master/ slave over a single server with sequential indexing?Am I missing anythi

Re: Master /Slave Architecture3.6.1

2013-01-10 Thread Otis Gospodnetic
Hi, You are going in the right direction and your assumptions are correct. In short, if the performance hit is too big then you simply need more ec2 instances (some have high cpu, some memory, some disk IO ... pick wisely). Otis Solr & ElasticSearch Support http://sematext.com/ On Jan 10, 2013 4:

Re: Master/Slave switch on teh fly. Replication

2012-03-16 Thread Michael Kuhlmann
Am 16.03.2012 15:05, schrieb stockii: i have 8 cores ;-) i thought that replication is defined in solrconfig.xml and this file is only load on startup and i cannot change master to slave and slave to master without restarting the servlet-container ?!?!?! No, you can reload the whole core at an

Re: Master/Slave switch on teh fly. Replication

2012-03-16 Thread stockii
i have 8 cores ;-) i thought that replication is defined in solrconfig.xml and this file is only load on startup and i cannot change master to slave and slave to master without restarting the servlet-container ?!?!?! - --- System ---

Re: Master/Slave switch on teh fly. Replication

2012-03-16 Thread Erick Erickson
What's the use-case? Presumably you have different configs... I'm actually not sure if you can do a reload see: http://wiki.apache.org/solr/CoreAdmin#RELOAD without a core, but you could try. Best Erick On Thu, Mar 15, 2012 at 4:59 AM, stockii wrote: > Hello. > > Is it possible to switch master

RE: Master Slave Question

2011-09-12 Thread Jaeger, Jay - DOT
: Patrick Sauts [mailto:patrick.via...@gmail.com] Sent: Saturday, September 10, 2011 11:26 AM To: solr-user@lucene.apache.org Subject: Re: Master Slave Question Real Time indexing (solr 4) or decrease replication poll and auto commit time. 2011/9/10 Jamie Johnson > Is it appropriate to query the mas

Re: Master Slave Question

2011-09-10 Thread Jamie Johnson
Is this feature on Trunk currently? On Sat, Sep 10, 2011 at 12:26 PM, Patrick Sauts wrote: > Real Time indexing (solr 4) or decrease replication poll and auto commit > time. > > 2011/9/10 Jamie Johnson > >> Is it appropriate to query the master servers when replicating?  I ask >> because there c

Re: Master Slave Question

2011-09-10 Thread Patrick Sauts
Real Time indexing (solr 4) or decrease replication poll and auto commit time. 2011/9/10 Jamie Johnson > Is it appropriate to query the master servers when replicating? I ask > because there could be a case where we index say 50 documents to the > master, they have not yet been replicated and a

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

2011-06-20 Thread Mark Schoy
You're right, thanks! 2011/6/20 Erick Erickson : > Hmmm, be a little careful here with terminology. > Shards may be unnecessary if you  can put your whole index > on a single searcher. It's preferable to   simply have each > slave hold a complete copy of the index, no sharding necessary. > > Best

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

2011-06-20 Thread Erick Erickson
Hmmm, be a little careful here with terminology. Shards may be unnecessary if you can put your whole index on a single searcher. It's preferable to simply have each slave hold a complete copy of the index, no sharding necessary. Best Erick On Mon, Jun 20, 2011 at 9:45 AM, Mark Schoy wrote: >

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

2011-06-20 Thread Mark Schoy
Thanks for your answer Erick. So the easiest way will be to set up 2 shard cluster with shard replicas ;) 2011/6/20 Erick Erickson : > No, there's nothing built into Solr to automatically promote a slave > to a master. > > You have several choices here. One is to build a new master and > reindex

Re: Master Slave Replication in Solr Cloud - What happens if the master is not available?

2011-06-20 Thread Erick Erickson
No, there's nothing built into Solr to automatically promote a slave to a master. You have several choices here. One is to build a new master and reindex from scratch. Another is to configure your slave as a new master and then bring up a new machine and have it replicate. Now make that new machi

Re: Master Slave help

2011-06-07 Thread Rohit Gupta
thanks Jayendra.. From: Jayendra Patil To: solr-user@lucene.apache.org Sent: Tue, 7 June, 2011 6:55:58 AM Subject: Re: Master Slave help Do you mean the replication happens everytime you restart the server ? If so, you would need to modify the events you

Re: Master Slave help

2011-06-06 Thread Jayendra Patil
Do you mean the replication happens everytime you restart the server ? If so, you would need to modify the events you want the replication to happen. Check for the replicateAfter tag and remove the startup option, if you don't need it. startup commit

Re: Master/Slave High CPU Usage

2010-11-23 Thread Lance Norskog
You're welcome. On Tue, Nov 23, 2010 at 3:55 PM, Ofer Fort wrote: > ok, we ran some tests and doing the commit for the "slave" as a post commit > event of the "master" reloaded the index and allowed us to achieve a master > slave configuration, without replication > This is useful only if your ma

Re: Master/Slave High CPU Usage

2010-11-23 Thread Ofer Fort
ok, we ran some tests and doing the commit for the "slave" as a post commit event of the "master" reloaded the index and allowed us to achieve a master slave configuration, without replication This is useful only if your master and slave are on the same machine, and it helps reducing the resources

Re: Master/Slave High CPU Usage

2010-11-21 Thread Ofer Fort
do i really need a commit? or can i use the *readercycle*script? since i don't need to comit anything, just reopen the reader. thanks On Sun, Nov 21, 2010 at 12:17 PM, Lance Norskog wrote: > Yes, the Solr commit operations always reloads the index

Re: Master/Slave High CPU Usage

2010-11-21 Thread Ofer Fort
ok, i'll try that and update the group thanks On Sun, Nov 21, 2010 at 12:17 PM, Lance Norskog wrote: > Yes, the Solr commit operations always reloads the index. And it > always throws away the Solr caches: queryresult, document, filter > query. > > If you do this, please post your results. > > O

Re: Master/Slave High CPU Usage

2010-11-21 Thread Lance Norskog
Yes, the Solr commit operations always reloads the index. And it always throws away the Solr caches: queryresult, document, filter query. If you do this, please post your results. On Sat, Nov 20, 2010 at 11:16 PM, Ofer Fort wrote: > OK, > so to make sure i understand, even though the "slave" doe

Re: Master/Slave High CPU Usage

2010-11-20 Thread Ofer Fort
OK, so to make sure i understand, even though the "slave" doesn't do any indexing, i will call commit and it will do nothing to the index itself, but will reload it? thanks On Sun, Nov 21, 2010 at 8:26 AM, Lance Norskog wrote: > Ah! If the program doing the indexing has manual commits, the progr

Re: Master/Slave High CPU Usage

2010-11-20 Thread Lance Norskog
Ah! If the program doing the indexing has manual commits, the program could send a commit to the slave. If the indexer uses automatic commits, there is a trick: you can add a program as a postCommit event in solrconfig.xml. This can just be a shell script or a curl command that sends a commit to th

Re: Master/Slave High CPU Usage

2010-11-20 Thread Ofer Fort
thanks Erick, but my question was regard the configuration Lance suggested, a configuration where i have two servers, set set logical master and slave, not as a true replication. Since both are running on the same machine, just have one only doing updates, and the other only queries, but both are u

Re: Master/Slave High CPU Usage

2010-11-20 Thread Erick Erickson
The slave polls. See: http://wiki.apache.org/solr/SolrReplication Best Erick On Sat, Nov 20, 2010 at 1:13 PM, Ofer Fort wrote: > Another question on that configuration, when the "master" commits, how does > the "slave" knows that the index has changed? Does it check the index and > finds out th

Re: Master/Slave High CPU Usage

2010-11-20 Thread Ofer Fort
Another question on that configuration, when the "master" commits, how does the "slave" knows that the index has changed? Does it check the index and finds out that it has a newer version? Thanks again for the help, Ofer ב-19 בנוב 2010, בשעה 05:30, Lance Norskog כתב/ה: If they are on the same

Re: Master/Slave High CPU Usage

2010-11-19 Thread Ofer Fort
That sounds like a great option, and it will also free some storage space on that sever (now each index is about 130GB). Other than the lock policy (we use single), any other things to look out to? Thanks ב-19 בנוב 2010, בשעה 05:30, Lance Norskog כתב/ה: If they are on the same server, you do no

Re: Master/Slave High CPU Usage

2010-11-18 Thread Lance Norskog
If they are on the same server, you do not need to replicate. If you only do queries, the query server can use the same index directory as the master. Works quite well. Both have to have the same LockPolicy in solrconfig.xml. For security reasons, I would run the query server as a different user w

Re: Master/Slave High CPU Usage

2010-11-17 Thread Ofer Fort
anybody? On Wed, Nov 17, 2010 at 12:09 PM, Ofer Fort wrote: > > Hi, I'm working with Erez, > we experienced this again, and this time the slave index folder didn't > contain the index.XXX folder, only one index folder. > if we shutdown the slave, the CPU on the master was normal, as soon as we

Re: Master/Slave High CPU Usage

2010-11-17 Thread Ofer Fort
Hi, I'm working with Erez, we experienced this again, and this time the slave index folder didn't contain the index.XXX folder, only one index folder. if we shutdown the slave, the CPU on the master was normal, as soon as we started the slave again, the CPU went up to 100% again. thanks for any hel

Re: master/slave

2010-03-09 Thread Peter Sturge
Hi Dino, I suppose you could write your own ReplicationHandler to do the replication yourself, but I should think the effort involved would be better spent deploying the existing Solr http replication or using a Hadoop-based solution, or UNIX scripting. By far, the easiest path to replication is

Re: master/slave

2010-03-09 Thread Dino Di Cola
Ok Peter for script-based replication; I forgot to mention I already verified that mechanism. When I configure the slave as follows http://localhost:8983/solr/admin/replication 00:00:20 ... SOLR uses the org.apache.solr.handler.ReplicationHandler to access ev

Re: master/slave

2010-03-09 Thread Peter Sturge
The SolrEmbededServer doesn't have any http, and so you can't use the http replication. You can use the script-based replication if you're on LUNIX. See: http://wiki.apache.org/solr/CollectionDistribution It would be worth looking at using Solr in a Jetty container and using the http replicati

Re: master/slave replication issue

2009-08-26 Thread Noble Paul നോബിള്‍ नोब्ळ्
The log messages are shown when you hit the admin page. So on't worry about that. Keep a minimal configuration of Replication. All you need is masterUrl and pollInterval. On Thu, Aug 27, 2009 at 5:52 AM, J G wrote: > > > > > > > > Hello, > > I'm having an issue getting the master to replicate it

Re: Master Slave data distribution | rsync fail issue

2009-07-02 Thread Bill Au
You can add the "-V" option to both your automatic and manual invocation of snappuller and snapinstaller tor both core and compare the debug info. Bill On Thu, Jul 2, 2009 at 11:02 AM, Vicky_Dev wrote: > > Yes. Permissions are same across cores > > ~Vikrant > > > Bill Au wrote: > > > > Are the u

Re: Master Slave data distribution | rsync fail issue

2009-07-02 Thread Vicky_Dev
Yes. Permissions are same across cores ~Vikrant Bill Au wrote: > > Are the user/group/permissions on the snapshot files the same for both > cases > (manual vs postCommit/postOptimize events)? > > Bill > > On Tue, May 5, 2009 at 12:54 PM, tushar kapoor < > tushar_kapoor...@rediffmail.com> wro

Re: master/slave failure scenario

2009-05-21 Thread Otis Gospodnetic
ith little effort. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: nk 11 > To: solr-user@lucene.apache.org > Sent: Thursday, May 21, 2009 12:44:55 PM > Subject: Re: master/slave failure scenario > > You are right... I just

Re: master/slave failure scenario

2009-05-21 Thread nk 11
You are right... I just don't like the idea of stopping the indexing process if the master fails until a new one is started (more or less by hand). On Thu, May 21, 2009 at 6:49 PM, Bryan Talbot wrote: > Indexing is usually much more expensive that replication so it won't scale > well as you add m

Re: master/slave failure scenario

2009-05-21 Thread Bryan Talbot
Indexing is usually much more expensive that replication so it won't scale well as you add more servers. Also, what would a client do if it was able to send the update to only some of the servers because others were down (for maintenance, etc)? -Bryan On May 21, 2009, at May 21, 6:04

Re: master/slave failure scenario

2009-05-21 Thread nk 11
Just curious. What would be the disadvantages of a no replication / multi master (no slave) setup? The client code should do the updates for evey master ofc, but if one machine would fail then I can imediatly continue the indexing process and also I can query the index on any machine for a valid re

Re: master/slave failure scenario

2009-05-14 Thread nk 11
wow! that was just a couple of days old! thanks as lot! 2009/5/14 Noble Paul നോബിള്‍ नोब्ळ् > yeah there is a hack > > https://issues.apache.org/jira/browse/SOLR-1154?focusedCommentId=12708316&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12708316 > > On Thu, May

Re: master/slave failure scenario

2009-05-14 Thread Noble Paul നോബിള്‍ नोब्ळ्
yeah there is a hack https://issues.apache.org/jira/browse/SOLR-1154?focusedCommentId=12708316&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12708316 On Thu, May 14, 2009 at 6:07 PM, nk 11 wrote: > sorry for the mail. I wanted to hit reply :( > > On Thu, May 14, 2

Re: master/slave failure scenario

2009-05-14 Thread nk 11
oh, so the configuration must be manualy changed? Can't something be passed at (re)start time? > > 2009/5/14 Noble Paul നോബിള്‍ नोब्ळ् > >> On Thu, May 14, 2009 at 4:07 PM, nk 11 wrote: >> > Ok so the VIP will point to the new master. but what makes a slave >> promoted >> > to a master? Only t

Re: master/slave failure scenario

2009-05-14 Thread nk 11
Ok so the VIP will point to the new master. but what makes a slave promoted to a master? Only the fact that it will receive add/update requests? And I suppose that this "hot" promotion is possible only if the slave is convigured as master also... 2009/5/14 Noble Paul നോബിള്‍ नोब्ळ् > ideally , w

Re: master/slave failure scenario

2009-05-13 Thread Noble Paul നോബിള്‍ नोब्ळ्
ideally , we don't do that. you can just keep the master host behind a VIP so if you wish to change the master make the VIP point to the new host On Wed, May 13, 2009 at 10:52 PM, nk 11 wrote: > This is more interesting.Such a procedure would involve taking down and > reconfiguring the slave? > >

Re: master/slave failure scenario

2009-05-13 Thread nk 11
This is more interesting.Such a procedure would involve taking down and reconfiguring the slave? On Wed, May 13, 2009 at 7:55 PM, Bryan Talbot wrote: > Or ... > > 1. Promote existing slave to new master > 2. Add new slave to cluster > > > > > -Bryan > > > > > > On May 13, 2009, at May 13, 9:48 AM

  1   2   >