ok, we ran some tests and doing the commit for the "slave" as a post commit event of the "master" reloaded the index and allowed us to achieve a master slave configuration, without replication This is useful only if your master and slave are on the same machine, and it helps reducing the resources needed, as you don't have 2 indexes and you don't need to copy the data from one to the other.
Thanks Lance for that proposal. On Sun, Nov 21, 2010 at 2:12 PM, Ofer Fort <ofer...@gmail.com> wrote: > do i really need a commit? or can i use the > *readercycle*<http://wiki.apache.org/solr/SolrOperationsTools>script? since i > don't need to comit anything, just reopen the reader. > thanks > > > On Sun, Nov 21, 2010 at 12:17 PM, Lance Norskog <goks...@gmail.com> wrote: > >> Yes, the Solr commit operations always reloads the index. And it >> always throws away the Solr caches: queryresult, document, filter >> query. >> >> If you do this, please post your results. >> >> On Sat, Nov 20, 2010 at 11:16 PM, Ofer Fort <ofer...@gmail.com> wrote: >> > OK, >> > so to make sure i understand, even though the "slave" doesn't do any >> > indexing, i will call commit and it will do nothing to the index itself, >> but >> > will reload it? >> > thanks >> > >> > On Sun, Nov 21, 2010 at 8:26 AM, Lance Norskog <goks...@gmail.com> >> wrote: >> > >> >> Ah! If the program doing the indexing has manual commits, the program >> >> could send a commit to the slave. If the indexer uses automatic >> >> commits, there is a trick: you can add a program as a postCommit event >> >> in solrconfig.xml. This can just be a shell script or a curl command >> >> that sends a commit to the slave Solr. >> >> >> >> Be sure to make all of the wait options false to this command; you >> >> don't want the master to block while the slave loads up the new index. >> >> Or, to control the maximum load on your server, you might actually >> >> want to make the master wait while the slave loads up >> >> >> >> Lance >> >> >> >> On Sat, Nov 20, 2010 at 2:13 PM, Ofer Fort <ofer...@gmail.com> wrote: >> >> > thanks Erick, >> >> > but my question was regard the configuration Lance suggested, a >> >> > configuration where i have two servers, set set logical master and >> slave, >> >> > not as a true replication. Since both are running on the same >> machine, >> >> just >> >> > have one only doing updates, and the other only queries, but both are >> >> using >> >> > the same index files. >> >> > >> >> > Ofer >> >> > >> >> > >> >> > On Sat, Nov 20, 2010 at 8:52 PM, Erick Erickson < >> erickerick...@gmail.com >> >> >wrote: >> >> > >> >> >> The slave polls. See: http://wiki.apache.org/solr/SolrReplication >> >> >> >> >> >> Best >> >> >> Erick >> >> >> >> >> >> On Sat, Nov 20, 2010 at 1:13 PM, Ofer Fort <ofer...@gmail.com> >> wrote: >> >> >> >> >> >> > Another question on that configuration, when the "master" commits, >> how >> >> >> does >> >> >> > the "slave" knows that the index has changed? Does it check the >> index >> >> and >> >> >> > finds out that it has a newer version? >> >> >> > Thanks again for the help, >> >> >> > Ofer >> >> >> > >> >> >> > >> >> >> > >> >> >> > ב-19 בנוב 2010, בשעה 05:30, Lance Norskog <goks...@gmail.com> >> כתב/ה: >> >> >> > >> >> >> > If they are on the same server, you do not need to replicate. >> >> >> > >> >> >> > If you only do queries, the query server can use the same index >> >> >> > directory as the master. Works quite well. Both have to have the >> same >> >> >> > LockPolicy in solrconfig.xml. For security reasons, I would run >> the >> >> >> > query server as a different user who has read-only access to the >> >> >> > index; that way it cannot touch the index. >> >> >> > >> >> >> > On Wed, Nov 17, 2010 at 11:28 PM, Ofer Fort <ofer...@gmail.com> >> >> wrote: >> >> >> > >> >> >> > anybody? >> >> >> > >> >> >> > >> >> >> > On Wed, Nov 17, 2010 at 12:09 PM, Ofer Fort <ofer...@gmail.com> >> >> wrote: >> >> >> > >> >> >> > >> >> >> > Hi, I'm working with Erez, >> >> >> > >> >> >> > we experienced this again, and this time the slave index folder >> didn't >> >> >> > contain the index.XXX folder, only one index folder. >> >> >> > >> >> >> > if we shutdown the slave, the CPU on the master was normal, as >> soon as >> >> we >> >> >> > started the slave again, the CPU went up to 100% again. >> >> >> > >> >> >> > thanks for any help >> >> >> > >> >> >> > ofer >> >> >> > >> >> >> > >> >> >> > On Wed, Nov 17, 2010 at 11:15 AM, Erez Zarum <e...@icinga.org.il> >> >> wrote: >> >> >> > >> >> >> > >> >> >> > Hi all, >> >> >> > >> >> >> > We've been seeing this for the second time already. >> >> >> > >> >> >> > I have a solr (1.4.1) master and a slave. both are located on the >> same >> >> >> > machine (16GB RAM, 4GB allocated to the slave and 3GB to the >> master) >> >> >> > >> >> >> > All our updates are going towards the master, and all the queries >> are >> >> >> > towards the slave. >> >> >> > >> >> >> > Once in a while the slave gets OutOfMemoryError. This is not the >> big >> >> >> > problem >> >> >> > (i have a about 100M documents) >> >> >> > >> >> >> > The problem is that from that moment the CPU of the slave AND the >> >> master >> >> >> is >> >> >> > almost 100%. >> >> >> > >> >> >> > If i shutdown the slave, the CPU of the master drops. >> >> >> > >> >> >> > If i start the slave again, the CPU is 100% again. >> >> >> > >> >> >> > I have the replication set on commit and startup. >> >> >> > >> >> >> > I see that in the data folder contains three index folders: index, >> >> >> > index.XXXYYY and index.XXXYYY.ZZZ >> >> >> > >> >> >> > >> >> >> > The only way i was able to get pass it (worked two times already), >> is >> >> to >> >> >> > shutdown the two servers, and to copy all the index of the master >> to >> >> the >> >> >> > slave, and start them again. >> >> >> > >> >> >> > From that moment and on, they continue to work and replicate with >> a >> >> very >> >> >> > reasonable CPU usage. >> >> >> > >> >> >> > >> >> >> > Our guess is that it failed to replicate due to the OOM and since >> then >> >> >> > tries >> >> >> > to do a full replication again and again? >> >> >> > >> >> >> > but why is the CPU of the master so high? >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > Lance Norskog >> >> >> > goks...@gmail.com >> >> >> > >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> Lance Norskog >> >> goks...@gmail.com >> >> >> > >> >> >> >> -- >> Lance Norskog >> goks...@gmail.com >> > >
