I have the luxury of JMS in my environment, so that may be a simple way to solve this...
Sent from my iPhone On Oct 13, 2011, at 4:02 PM, "Robert Stewart" <bstewart...@gmail.com> wrote: > Yes that is a good point. Thanks. > > I think I will avoid using NAS/SAN and use two masters, one setup as a > repeater (slave and master). In case of very rare master failure, some minor > manual intervention will be required to re-configure remaining master or > bring other one back up. > > My only concern in that case is losing new documents from solrj client since > there is no broker/buffer/queue between solrj client and SOLR master. It > would be nice if there was some open source broker/queue which could sit > between solrj and SOLR and queue up messages (publish/subscribe). > > Bob > > On Oct 13, 2011, at 3:56 PM, Jaeger, Jay - DOT wrote: > >> One thing to consider is the case where the JVM is up, but the system is >> otherwise unavailable (say, a NIC failure, firewall failure, load balancer >> failure) - especially if you use a SAN (whose connection is different from >> the normal network). >> >> In such a case the old master might have uncommitted updates. >> >> JRJ >> >> -----Original Message----- >> From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] >> Sent: Tuesday, October 11, 2011 3:17 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Replication with an HA master >> >> Hello, >> ----- Original Message ----- >> >>> From: Robert Stewart <bstewart...@gmail.com> >>> To: solr-user@lucene.apache.org >>> Cc: >>> Sent: Tuesday, October 11, 2011 3:37 PM >>> Subject: Re: Replication with an HA master >>> >>> In the case of using a shared (SAN) index between 2 masters, what happens >>> if the >>> live master fails in such a way that the index remains "locked" (such >>> as if some hardware failure and it did not unlock/close index). Will the >>> other >>> master be able to open/write to the index as new documents are added? >> >> >> You'd use native locks, which should disappear if the JVM dies. If it does >> not, then I'm not 100% sure what happens, but in the worst case there would >> be a need for a quick manual (or scripted) intervention. But your index >> would be up to date! >> >>> Also, if that can work ok, would it work if you have a LB (VIP) from both >>> indexing and replication sides of the 2 masters, such that some VIP used by >>> solrj for indexing new documents via HTTP, and the same VIP used by slave >>> searchers for replication? That sounds like it would work. >> >> >> Precisely what you should do. e.g. "master-vip" is the "hostname" that both >> SolrJ would post new docs to and the master "server" slaves would poll for >> index changes. >> >> Otis >> ---- >> >> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch >> Lucene ecosystem search :: http://search-lucene.com/ >> >> >> >> >>> On Oct 11, 2011, at 3:16 PM, Otis Gospodnetic wrote: >>> >>>> Hello, >>>> >>>> Yes, you've read about NFS, which is why I gave the example of a SAN >>> (which can have multiple power supplies, controllers, etc.) >>>> >>>> Yes, should be OK to have multiple Solr instances have the same index >>>> open, >>> since only one of them will actually be writing to it, thanks to LB. >>>> >>>> Otis >>>> ---- >>>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch >>>> Lucene ecosystem search :: http://search-lucene.com/ >>>> >>>> >>>>> ________________________________ >>>>> From: Brandon Ramirez <brandon_rami...@elementk.com> >>>>> To: "solr-user@lucene.apache.org" >>> <solr-user@lucene.apache.org> >>>>> Sent: Tuesday, October 11, 2011 2:55 PM >>>>> Subject: RE: Replication with an HA master >>>>> >>>>> Using a shared volume crossed my mind too, but I discarded the idea >>> because of literature I have read about Lucene performing poorly against >>> remote >>> file systems. But then I suppose a SAN wouldn't be a remote file system in >>> the same sense as an NFS-mounted NAS or similar. >>>>> >>>>> Should I be concerned about two solr instances on two machines having >>> the same SAN-based index open, as long as only one of them is receiving >>> requests? I would think in theory it would work, but I don't have any >>> production-level experience with Solr yet, only textbook knowledge. >>>>> >>>>> >>>>> Brandon Ramirez | Office: 585.214.5413 | Fax: 585.295.4848 >>>>> Software Engineer II | Element K | www.elementk.com >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: Otis Gospodnetic [mailto:otis_gospodne...@yahoo.com] >>>>> Sent: Tuesday, October 11, 2011 2:28 PM >>>>> To: solr-user@lucene.apache.org >>>>> Subject: Re: Replication with an HA master >>>>> >>>>> A few alternatives: >>>>> * Have the master keep the index on a shared disk (e.g. SAN) >>>>> * Use LB to easily switch to between masters, potentially even >>> automatically if LB can detect the primary is down >>>>> >>>>> Otis >>>>> ---- >>>>> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene >>> ecosystem search :: http://search-lucene.com/ >>>>> >>>>> >>>>>> ________________________________ >>>>>> From: Robert Stewart <bstewart...@gmail.com> >>>>>> To: solr-user@lucene.apache.org >>>>>> Sent: Friday, October 7, 2011 10:22 AM >>>>>> Subject: Re: Replication with an HA master >>>>>> >>>>>> Your idea sounds like the correct path. Setup 2 masters, one >>> running >>>>>> in "slave" mode which pulls replicas from the live >>> master. When/if live master goes down, you just reconfigure and restart >>> the >>> backup master to be the live master. You'd also need to then start data >>> import on the backup master (enable/start cron job?), and redirect slave >>> searchers to pull replicas from the new live master. All that could be >>> done >>> using scripts or something like puppet possibly. >>>>>> >>>>>> Another solution maybe is to run 2 "live" masters, which >>> both index the same content from the same data source. If one goes down, >>> then >>> you just need to redirect slave searchers to the backup master for >>> replication. >>>>>> >>>>>> I am also starting a similar project which needs some disaster >>> recovery processes in place, so any other info would be useful to me as >>> well. >>>>>> >>>>>> Bob >>>>>> >>>>>> On Oct 7, 2011, at 9:53 AM, Brandon Ramirez wrote: >>>>>> >>>>>>> We are getting ready to start a project using Solr as our >>> backend search engine and I am trying to devise a deployment architecture >>> that >>> works for us. We definitely need a master/slave replication strategy, >>> that's for sure, but my concern is the master becomes a single point of >>> failure. >>>>>>> >>>>>>> Fortunately, real-time search is not a requirement for us. If >>> search results are a few minutes out of sync with our database, it's not a >>> big deal. >>>>>>> >>>>>>> So what I would like to do is have a set of query servers >>> (slaves) that are only used for querying, no indexing and have them use >>> Solr's HTTP replication mechanism on a 2 or 3 minute interval. To get HA >>> indexing, I'd like to have 2 masters: a primary and a standby. All >>> indexing >>> requests go to the primary unless it's taken out of service. To keep the >>> standby ready to takeover if it needs to, it needs to be more up to date >>> than >>> the slaves. I'd like to have it replicate every 30 seconds or so. >>>>>>> >>>>>>> The reason I'm asking about it on this list is that I >>> haven't seen any Solr documentation or even anything that talks about this. >>> >>> I can't be the only one concerned about having a single point of failure, >>> so >>> I'm reaching out to see what others have done in this case before I go with >>> my own solution. >>>>>>> >>>>>>> >>>>>>> Brandon Ramirez | Office: 585.214.5413 | Fax: 585.295.4848 >>> Software >>>>>>> Engineer II | Element K | >>> www.elementk.com<http://www.elementk.com/> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> >>> > >
