Re: Pause and resume indexing on SolR 4 for backups

2013-01-09 Thread Upayavira
The point was as much about how to use a backup, as to how to make one in the first place. the replication handler can handle spitting out a backup, but there's no straightforward way to tell Solr to switch to another set of index files instead. You'd have to do clever stuff with the CoreAdminHandl

Re: Pause and resume indexing on SolR 4 for backups

2013-01-09 Thread Paul Jungwirth
Yes, I agree about making sure the backups actually work, whatever the approach. Thanks for your reply and all you've contributed to the Solr/Lucene community. The Lucene in Action book has been a huge help to me. Paul On Wed, Jan 9, 2013 at 12:16 PM, Otis Gospodnetic < otis.gospodne...@gmail.co

Re: Pause and resume indexing on SolR 4 for backups

2013-01-09 Thread Otis Gospodnetic
Hi Paul, Hot backup is OK. There was a thread on this topic yesterday and the day before. But you should always try running from backup regardless of what anyone says here, because if you have to do that one day you want to know you verified it :) Otis -- Solr & ElasticSearch Support http:/

Re: Pause and resume indexing on SolR 4 for backups

2013-01-09 Thread Paul Jungwirth
> Are you sure a commit didn't happen between? > Also, a background merge might have happened. > > As to using a backup, you are right, just stop solr, > put the snapshot into index/data, and restart. This was mentioned before but seems not to have gotten any attention: can't you use the Replicat

Re: Pause and resume indexing on SolR 4 for backups

2012-12-21 Thread Andy D'Arcy Jewell
On 20/12/12 20:19, alx...@aim.com wrote: Depending on your architecture, why not index the same data into two machines? One will be your prod another your backup? Because we're trying to keep costs and complexity low whilst in the development stage ;-) But more seriously, this will obviously

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread alxsss
Depending on your architecture, why not index the same data into two machines? One will be your prod another your backup? Thanks. Alex. -Original Message- From: Upayavira To: solr-user Sent: Thu, Dec 20, 2012 11:51 am Subject: Re: Pause and resume indexing on SolR 4 for

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Upayavira
You're saying that there's no chance to catch it in the middle of writing the segments file? Having said that, the segments file is pretty small, so the chance would be pretty slim. Upayavira On Thu, Dec 20, 2012, at 06:45 PM, Lance Norskog wrote: > To be clear: 1) is fine. Lucene index updates

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Lance Norskog
To be clear: 1) is fine. Lucene index updates are carefully sequenced so that the index is never in a bogus state. All data files are written and flushed to disk, then the segments.* files are written that match the data files. You can capture the files with a set of hard links to create a back

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Upayavira
Are you sure a commit didn't happen between? Also, a background merge might have happened. As to using a backup, you are right, just stop solr, put the snapshot into index/data, and restart. Upayavira On Thu, Dec 20, 2012, at 05:16 PM, Andy D'Arcy Jewell wrote: > On 20/12/12 13:38, Upayavira wro

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Andy D'Arcy Jewell
On 20/12/12 13:38, Upayavira wrote: The backup directory should just be a clone of the index files. I'm curious to know whether it is a cp -r or a cp -lr that the replication handler produces. You would prevent commits by telling your app not to commit. That is, Solr only commits when it is *tol

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Upayavira
> > From:Upayavira > > Sent: Thu 20-Dec-2012 14:45 > > To: solr-user@lucene.apache.org > > Subject: Re: Pause and resume indexing on SolR 4 for backups > > > > The backup directory should just be a clone of the index files. I'm > > curious to know

RE: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Markus Jelsma
You can use the postCommit event in updateHandler to execute a task. -Original message- > From:Upayavira > Sent: Thu 20-Dec-2012 14:45 > To: solr-user@lucene.apache.org > Subject: Re: Pause and resume indexing on SolR 4 for backups > > The backup directory should

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Upayavira
The backup directory should just be a clone of the index files. I'm curious to know whether it is a cp -r or a cp -lr that the replication handler produces. You would prevent commits by telling your app not to commit. That is, Solr only commits when it is *told* to. Unless you use autocommit, in

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Andy D'Arcy Jewell
On 20/12/12 11:58, Upayavira wrote: I've never used it, but the replication handler has an option: http://master_host:port/solr/replication?command=backup Which will take you a backup. I've looked at that this morning as suggested by Markus Jelsma. Looks good, but I'll have to work out how

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Upayavira
I've never used it, but the replication handler has an option: http://master_host:port/solr/replication?command=backup Which will take you a backup. Also something to note, if you don't want to use the above, and you are running on Unix, you can create fast 'hard link' clones of lucene indexe

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Gora Mohanty
On 20 December 2012 16:14, Andy D'Arcy Jewell wrote: [...] > It's attached to a web-app, which accepts uploads and will be available > 24/7, with a global audience, so "pausing" it may be rather difficult (tho I > may put this to the developer - it may for instance be possible if he has a > small

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Andy D'Arcy Jewell
On 20/12/12 10:24, Gora Mohanty wrote: Unless I am missing something, the index is only being written to when you are adding/updating the index. So, the question is how is this being done in your case, and could you pause indexing for the duration of the backup? Regards, Gora It's attached to a

RE: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Markus Jelsma
You can use the replication handler to fetch a complete snapshot of the index over HTTP. http://wiki.apache.org/solr/SolrReplication#HTTP_API -Original message- > From:Andy D'Arcy Jewell > Sent: Thu 20-Dec-2012 11:23 > To: solr-user@lucene.apache.org > Subject: Pause and resume indexi

Re: Pause and resume indexing on SolR 4 for backups

2012-12-20 Thread Gora Mohanty
On 20 December 2012 15:46, Andy D'Arcy Jewell wrote: > Hi all. > > Can anyone advise me of a way to pause and resume SolR 4 so I can perform a > backup? I need to be able to revert to a usable (though not necessarily > complete) index after a crash or other "disaster" more quickly than a > re-inde