Right, but if you want to take periodic backups and ship them to tape or some DR site, you need to be able to tell when the backup is actually complete.
It's seems very strange to me that you can actually track the replication progress on a slave, but you can't track the backup progress on a master. To me that suggests that the only reliable way of performing backups is to set up replication to some slave without a regular polling interval. Then force a poll, wait for the sync to complete, and ship the slave's index to redundant storage. Seems like a pretty backwards way of doing things... -----Original Message----- From: noble.p...@gmail.com [mailto:noble.p...@gmail.com] On Behalf Of Noble Paul ??????? ?????? Sent: Thursday, January 07, 2010 5:56 AM To: solr-user Subject: Re: replication --> missing field data file actually it does not. BTW, FYI, backup is just to take periodics backups not necessary for the Replicationhandler to work On Thu, Jan 7, 2010 at 2:37 AM, Giovanni Fernandez-Kincade <gfernandez-kinc...@capitaliq.com> wrote: > How can you tell when the backup is done? > > -----Original Message----- > From: noble.p...@gmail.com [mailto:noble.p...@gmail.com] On Behalf Of Noble > Paul ??????? ?????? > Sent: Wednesday, January 06, 2010 12:23 PM > To: solr-user > Subject: Re: replication --> missing field data file > > the index dir is in the name "index" others will be stored as > index<date-as-number> > > On Wed, Jan 6, 2010 at 10:31 PM, Giovanni Fernandez-Kincade > <gfernandez-kinc...@capitaliq.com> wrote: >> How can you differentiate between the backup and the normal index files? >> >> -----Original Message----- >> From: noble.p...@gmail.com [mailto:noble.p...@gmail.com] On Behalf Of Noble >> Paul ??????? ?????? >> Sent: Wednesday, January 06, 2010 11:52 AM >> To: solr-user >> Subject: Re: replication --> missing field data file >> >> On Wed, Jan 6, 2010 at 9:49 PM, Giovanni Fernandez-Kincade >> <gfernandez-kinc...@capitaliq.com> wrote: >>> I set up replication between 2 cores on one master and 2 cores on one >>> slave. Before doing this the master was working without issues, and I >>> stopped all indexing on the master. >>> >>> Now that replication has synced the index files, an .FDT field is suddenly >>> missing on both the master and the slave. Pretty much every operation (core >>> reload, commit, add document) fails with an error like the one posted below. >>> >>> How could this happen? How can one recover from such an error? Is there any >>> way to regenerate the FDT file without re-indexing everything? >>> >>> This brings me to a question about backups. If I run the >>> replication?command=backup command, where is this backup stored? I've tried >>> this a few times and get an OK response from the machine, but I don't see >>> the backup generated anywhere. >> The backup is done asynchronously. So it always gives an OK response >> immedietly. >> The backup is created in the data dir itself >>> >>> Thanks, >>> Gio. >>> >>> org.apache.solr.common.SolrException: Error handling 'reload' action >>> at >>> org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:412) >>> at >>> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:142) >>> at >>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) >>> at >>> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:298) >>> at >>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:174) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) >>> at >>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) >>> at >>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) >>> at >>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:172) >>> at >>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >>> at >>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) >>> at >>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) >>> at >>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174) >>> at >>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:875) >>> at >>> org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) >>> at >>> org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) >>> at >>> org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) >>> at >>> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) >>> at java.lang.Thread.run(Unknown Source) >>> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: >>> Y:\solrData\FilingsCore2\index\_a0r.fdt (The system cannot find the file >>> specified) >>> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1068) >>> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:579) >>> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:425) >>> at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:486) >>> at >>> org.apache.solr.handler.admin.CoreAdminHandler.handleReloadAction(CoreAdminHandler.java:409) >>> ... 18 more >>> Caused by: java.io.FileNotFoundException: >>> Y:\solrData\FilingsCore2\index\_a0r.fdt (The system cannot find the file >>> specified) >>> at java.io.RandomAccessFile.open(Native Method) >>> at java.io.RandomAccessFile.<init>(Unknown Source) >>> at >>> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:78) >>> at >>> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:108) >>> at >>> org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:65) >>> at >>> org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:104) >>> at >>> org.apache.lucene.index.SegmentReader$CoreReaders.openDocStores(SegmentReader.java:277) >>> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:640) >>> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:599) >>> at >>> org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:103) >>> at >>> org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:27) >>> at >>> org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:73) >>> at >>> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:704) >>> at >>> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:68) >>> at org.apache.lucene.index.IndexReader.open(IndexReader.java:476) >>> at org.apache.lucene.index.IndexReader.open(IndexReader.java:403) >>> at >>> org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:38) >>> at org.apache.solr.core.SolrCore.getSearcher(So >>> >>> >>> >>> >>> >>> >> >> >> >> -- >> ----------------------------------------------------- >> Noble Paul | Systems Architect| AOL | http://aol.com >> > > > > -- > ----------------------------------------------------- > Noble Paul | Systems Architect| AOL | http://aol.com > -- ----------------------------------------------------- Noble Paul | Systems Architect| AOL | http://aol.com
