Re: Solr7 org.apache.lucene.index.IndexUpgrader

2017-11-27 Thread Shawn Heisey
On 11/27/2017 2:58 AM, Leo Prince wrote: > Actually I have two major cores. One I have primary document store as MySQL > and I can populate and re-index data from MySQL. However the other core > with 40mil, is keeping as primary store (with stored=true), I get the fact > that it's not a good practi

Re: Solr7 org.apache.lucene.index.IndexUpgrader

2017-11-27 Thread Rick Leir
Leo Your low priority data could be accumulated in a Couchbase DB or just in JSONL. Then it would be easy to re-index. Cheers -- Rick -- Sorry for being brief. Alternate email is rickleir at yahoo dot com

Re: Solr7 org.apache.lucene.index.IndexUpgrader

2017-11-27 Thread Leo Prince
Hi Daniel, Thanks for the help. Actually I have two major cores. One I have primary document store as MySQL and I can populate and re-index data from MySQL. However the other core with 40mil, is keeping as primary store (with stored=true), I get the fact that it's not a good practice, however due

Re: Solr7 org.apache.lucene.index.IndexUpgrader

2017-11-27 Thread Daniel Collins
Leo, the general rule of thumb here is that the Solr index should *not* be your main document store. It is the index to your document store, but if it needs to be re-indexed, you should use your document store as the place to index from. Your index will not have the full source data (unless ALL y

Re: Solr7 org.apache.lucene.index.IndexUpgrader

2017-11-26 Thread Leo Prince
Hi Shawn, Thanks for the help. I hate to burst your bubble here ... but 4 million docs is pretty small for > a Solr index. I have one index that's a hundred times larger, and there > are people with *billions* of documents in SolrCloud. > Sorry I missed a "0" there. It's actually 40 Millions, s

Re: Solr7 org.apache.lucene.index.IndexUpgrader

2017-11-24 Thread Shawn Heisey
On 11/23/2017 11:31 PM, Leo Prince wrote: We were using bit older version Solr 4.10.2 and upgrading to Solr7. We have like 4mil records in one of the core which is of course pretty huge, hence re-sourcing the index is nearly impossible and re-querying from source Solr to Solr7 is also going to b