Re: Merge indexes in MapReduce

2015-04-17 Thread Erick Erickson
The core admin MERGEINDEXES will work for you I'm pretty sure. You copy the NRT index over to the all-the-time box. MERGEINDEXES just takes the path to the index you want to add to the existing core. Note the warnings in the reference guide about taking care that the indexes aren't changing and co

Re: Merge indexes in MapReduce

2015-04-17 Thread ariya bala
Hi Norgorn, I think there is no ready-made tool out of the box, but you have the spare parts in the MapreduceIndexerTool :-) With little effort you can decouple the index merging component from MRIndexerTool and use based on the needs. I did the same. On Fri, Apr 17, 2015 at 10:40 AM, Norgorn wr

Re: Merge indexes in MapReduce

2015-04-17 Thread Norgorn
Thank you for the reply. Out schema is: 1) Index real-time (on separate machine). 2) NRT index becomes large. 3) Copy NRT index on other machine. 3) Merge NRT-made indexes with large ("all-the-time") index. 4) Remove NRT index (until now it was available for searching). At the end we have big, opt

Re: Merge indexes in MapReduce

2015-04-16 Thread Erick Erickson
You're stating two things that are somewhat antithetical: 1: We have real-time search and 2: want to merge (and optimize) its indexes into one Needing to merge indexes implies (to me at least) that you're not really doing NRT processing as docs in the batch you're merging into your collection aren