Re: Avoiding Transient state during a long running background process

2017-03-30 Thread Erick Erickson
Right, you're artificially forcing this loading/unloading, which is a good thing to stress! Every time the code accesses a core, the last access time should be updated. So somehow either you're accessing more than two cores in a round-robin fashion or you're somehow having other requests come in b

Re: Avoiding Transient state during a long running background process

2017-03-30 Thread Shashank Pedamallu
Hi, Thanks Erik and Shawn for so much info! 1) Yes, I have deliberately put transientCacheSize as very small(2) in my dev laptop for testing how Solr handles the switch and how it affects my backup process. 2) Yes, I'm not using Solr Cloud. Each individual Solr core is independent and we are n

Re: Avoiding Transient state during a long running background process

2017-03-30 Thread Erick Erickson
bq: I thought that LotsOfCores didn't coexist with Cloud very well. It doesn't, you're right, I got off on a tangent there. The OP mentioned "Cloud" and my brain cross-wired. On Thu, Mar 30, 2017 at 6:32 AM, Shawn Heisey wrote: > On 3/29/2017 8:09 PM, Erick Erickson wrote: >> bq: My guess is tha

Re: Avoiding Transient state during a long running background process

2017-03-30 Thread Shawn Heisey
On 3/29/2017 8:09 PM, Erick Erickson wrote: > bq: My guess is that it is decided by the load time, because this is > the option that would have the best performance. > > Not at all. The theory here is that this is to support the pattern > where some transient cores are used all the time and some co

Re: Avoiding Transient state during a long running background process

2017-03-29 Thread Erick Erickson
It's an LRU cache time. See the docs for LinkedHashmap, this form of the c'tor is used in SolrCores.allocateLazyCores transientCores = new LinkedHashMap(Math.min(cacheSize, 1000), 0.75f, true) { which is a special form of the c'tor that creates an access-ordered map. I had a terrible moment seein

Re: Avoiding Transient state during a long running background process

2017-03-29 Thread Shashank Pedamallu
Thanks again for the information Shawn. 1) The long running process I told earlier was about Backup. I have written a custom BackupHandler to backup the index files to a Cloud storage following the ReplicationHandler class. I’m just wondering how does switching between transient state affect s

Re: Avoiding Transient state during a long running background process

2017-03-29 Thread Shawn Heisey
On 3/29/2017 4:50 PM, Shashank Pedamallu wrote: > Thank you very much for the response. Is there no definite way of > ensuring that Solr does not switch transient states by an api? Like > solrCore.open() and solrCore.close()? I am not aware of any way to tell Solr to NOT unload a core when all of

Re: Avoiding Transient state during a long running background process

2017-03-29 Thread Shashank Pedamallu
Hi Shawn, Thank you very much for the response. Is there no definite way of ensuring that Solr does not switch transient states by an api? Like solrCore.open() and solrCore.close()? Thanks, Shashank Pedamallu MTS MBU vCOps Dev US-CA-Promontory E, E 1035 Email: spedama...@vmware.com Office: 650.

Re: Avoiding Transient state during a long running background process

2017-03-29 Thread Shawn Heisey
On 3/29/2017 11:17 AM, Shashank Pedamallu wrote: > I’m performing some long running operation on a Background thread on a > Core and I observed that since the core has the property “transient” > set to true, in between this operation completes, the core is being > CLOSED and OPENED by Solr (even th