Re: How to re-index SOLR data

2016-08-10 Thread John Bickerstaff
Right... SOLR doesn't work quite that way... Keep in mind the value of the data import jar if you have the data from MySQL stored in a text file, although that would require a little programming to get the data into the proper format.. But once you get everything into a text file or similar, you

Re: How to re-index SOLR data

2016-08-10 Thread Bharath Kumar
Hi All, Thanks so much for your inputs. We have a MYSQL data source and i think we will try to re-index using the MYSQL data. I wanted something where i can export all my current data say to an excel file or some data source and then import it on another node with the same collection with empty d

Re: How to re-index SOLR data

2016-08-09 Thread Erick Erickson
Assuming you can re-index Consider "collection aliasing". Say your current collection is C1. Create C2 (using the same cluster, Zookeeper and the like). Go ahead and index to C2 (however you do that). NOTE: the physical machines may be _different_ than C1, or not. That's up to you. The critica

Re: How to re-index SOLR data

2016-08-09 Thread John Bickerstaff
In my case, I've done two things neither of them involved taking the data from SOLR to SOLR... although in my reading, I've seen that this is theoretically possible (I.E. sending data from one SOLR server to another SOLR server and having the second SOLR instance re-index...) I haven't used

Re: How to re-index SOLR data

2016-08-09 Thread Bharath Kumar
Hi John, Thanks so much for your inputs. We have time to build another system. So how did you index the same data on the main SOLR node to the new SOLR node? Did you use the re-index python script? The new data will be indexed correctly with the new rules, but what about the old data? Our SOLR da

Re: How to re-index SOLR data

2016-08-09 Thread John Bickerstaff
In case this helps... Assuming you have the resources to build a copy of your production environment and assuming you have the time, you don't need to take your production down - or even affect it's processing... What I've done (with admittedly smaller data sets) is build a separate environment (

Re: How to re-index SOLR data

2016-08-09 Thread Bharath Kumar
Hi Nick and Shawn, Thanks so much for the pointers. I will try that out. Thank you again! On Tue, Aug 9, 2016 at 9:40 AM, Nick Vasilyev wrote: > Hi, I work on a python Solr Client > library and there is a > reindexing helper module that you can use

Re: How to re-index SOLR data

2016-08-09 Thread Nick Vasilyev
Hi, I work on a python Solr Client library and there is a reindexing helper module that you can use if you are on Solr 4.9+. I use it all the time and I think it works pretty well. You can re-index all documents from a collection into another collection

Re: How to re-index SOLR data

2016-08-09 Thread Shawn Heisey
On 8/9/2016 1:48 AM, bharath.mvkumar wrote: > What would be the best way to re-index the data in the SOLR cloud? We > have around 65 million data and we are planning to change the schema > by changing the unique key type from long to string. How long does it > take to re-index 65 million documents