Re: Commits on service after shutdown

2010-10-19 Thread Jan Høydahl / Cominvent
You never get full control of commits, as Solr will auto-commit anyway whenever the (configurable) input buffer is full. With the current architecture you cannot really trust adds or commits to 100% certainly be successful, because the server may have been restarted between an add and commit() w

Re: Commits on service after shutdown

2010-10-18 Thread Ezequiel Calderara
I'll see if i can resolve this adding an extra core with the same schema for holding this documents. So, Core0 will act as a "Queue" and the Core1 will be the real index. And the commit in the core0 will trigger an add to the core1 and its commit. That way i can be sure of not losing data. It surp

Re: Commits on service after shutdown

2010-10-18 Thread Ezequiel Calderara
But if something happens in between that hour, i will have lost or committed the documents to the index out of the schedule. How can i handle this scenario? I think that Solr (or Lucene) should make sure of the durabilityof the data even

Re: Commits on service after shutdown

2010-10-18 Thread Matthew Hall
No.. you would just turn autocommit off, and have the thread that is doing updates to your indexes commit every hour. I'd think that this would take care of the scenario that you are describing. Matt On 10/18/2010 3:50 PM, Ezequiel Calderara wrote: I understand, but i want to have control

Re: Commits on service after shutdown

2010-10-18 Thread Ezequiel Calderara
I understand, but i want to have control of what is commit or not. In our scenario, we want to add documents to the index, and maybe after an hour trigger the commit. If in the middle, we have a server shutdown or any process sending a Shutdown signal to the process. I don't want those documents b

Re: Commits on service after shutdown

2010-10-18 Thread Israel Ekpo
The documents should be implicitly committed when the Lucene index is closed. When you perform a graceful shutdown, the Lucene index gets closed and the documents get committed implicitly. When the shutdown is abrupt as in a KILL -9, then this does not happen and the updates are lost. You can us

Commits on service after shutdown

2010-10-18 Thread Ezequiel Calderara
Hi, i'm new in the mailing list. I'm implementing Solr in my actual job, and i'm having some problems. I was testing the consistency of the "commits". I found for example that if we add X documents to the index (without commiting) and then we restart the service, the documents are commited. They s