Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-11 Thread Dmitry Kan
Thanks Shawn, perhaps the comment on the luceneMatchVersion in the example schema.xml could be changed to reflect / clarify this? this comment made me think that the parameter is affecting the index side of things too (aka index format version). I.e. I would appreciate seeing there things lik

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-11 Thread Shawn Heisey
On 4/11/2014 12:42 AM, Dmitry Kan wrote: > Thanks! So solr 4.7 does not seem to respect the luceneMatchVersion on the > binary (index) level. Or perhaps, I misunderstand the meaning of the > luceneMatchVersion. luceneMatchVersion does not dictate the index format. It is a way to signal things lik

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-10 Thread Dmitry Kan
Thanks! So solr 4.7 does not seem to respect the luceneMatchVersion on the binary (index) level. Or perhaps, I misunderstand the meaning of the luceneMatchVersion. This is what I see when loading index from hdfs via luke and launching the Index Checker tool: [clip] Segments file=segments_2 numSeg

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-10 Thread Wolfgang Hoschek
There’s no such other location in there. BTW, you can disable the mtree merge via --reducers=-2 (or --reducers=0 in old versions) . Wolfgang. On Apr 10, 2014, at 3:44 PM, Dmitry Kan wrote: > a correction: actually when I tested the above change I had so little data, > that it didn't trigger su

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-10 Thread Dmitry Kan
a correction: actually when I tested the above change I had so little data, that it didn't trigger sub-shard slicing and thus merging of the slices. Still, looks as if somewhere in the map-reduce contrib code there is a "link" to what lucene version to use. Wolfgang, do you happen to know where th

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-10 Thread Dmitry Kan
Thanks for responding, Wolfgang. Changing to LUCENE_43: IndexWriterConfig writerConfig = new IndexWriterConfig(Version.LUCENE_43, null); didn't affect on the index format version, because, I believe, if the format of the index to merge has been of higher version (4.1 in this case), it will merge

Re: MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-09 Thread Wolfgang Hoschek
There is a current limitation in that the code doesn’t actually look into solrconfig.xml for the version. We should fix this, indeed. See https://github.com/apache/lucene-solr/blob/trunk/solr/contrib/map-reduce/src/java/org/apache/solr/hadoop/TreeMergeOutputFormat.java#L100-101 Wolfgang. On Apr

MapReduceIndexerTool does not respect Lucene version in solrconfig Was: converting 4.7 index to 4.3.1

2014-04-08 Thread Dmitry Kan
Hello, When we instantiate the MapReduceIndexerTool with the collections' conf directory, we expect, that the Lucene version is respected and the index gets generated in a format compatible with the defined version. This does not seem to happen, however. Checking with luke: the expected Lucene

converting 4.7 index to 4.3.1

2014-04-07 Thread Dmitry Kan
Dear list, We have been generating solr indices with the solr-hadoop contrib module (SOLR-1301). Our current solr in use is of 4.3.1 version. Is there any tool that could do the backward conversion, i.e. 4.7->4.3.1? Or is the upgrade the only way to go? -- Dmitry Blog: http://dmitrykan.blogspot.