Re: Solr Lucene Index Version

2011-12-08 Thread Jamie Johnson
Thanks Robert. I'll watch them all. Any others that are good to keep track of? On Thu, Dec 8, 2011 at 1:25 PM, Robert Muir wrote: > On Thu, Dec 8, 2011 at 12:55 PM, Jamie Johnson wrote: >> Thanks Andrzej.  I'll continue to follow the portable format JIRA >> along with 3622, are there any other

Re: Solr Lucene Index Version

2011-12-08 Thread Robert Muir
On Thu, Dec 8, 2011 at 12:55 PM, Jamie Johnson wrote: > Thanks Andrzej.  I'll continue to follow the portable format JIRA > along with 3622, are there any others that you're aware of that are > blockers that would be useful to watch? > There is a lot to be done, particularly norms and deleted doc

Re: Solr Lucene Index Version

2011-12-08 Thread Jamie Johnson
Thanks Andrzej. I'll continue to follow the portable format JIRA along with 3622, are there any others that you're aware of that are blockers that would be useful to watch? On Thu, Dec 8, 2011 at 10:49 AM, Andrzej Bialecki wrote: > On 08/12/2011 14:50, Jamie Johnson wrote: >> >> Mark, >> >> Agre

Re: Solr Lucene Index Version

2011-12-08 Thread Andrzej Bialecki
On 08/12/2011 14:50, Jamie Johnson wrote: Mark, Agreed that Replication wouldn't help, I was dreaming that there was some intermediate format used in replication. Ideally you are right, I could just reindex the data and go on with life, but my case is not so simple. Currently we have some set

Re: Solr Lucene Index Version

2011-12-08 Thread Jamie Johnson
Thanks Robert. I'll continue to watch the Jira and try not to bother folks about this. Again greatly appreciate the insight. On Thu, Dec 8, 2011 at 11:31 AM, Robert Muir wrote: > On Thu, Dec 8, 2011 at 10:46 AM, Mark Miller wrote: >> >> On Dec 8, 2011, at 8:50 AM, Jamie Johnson wrote: >> >>> I

Re: Solr Lucene Index Version

2011-12-08 Thread Robert Muir
On Thu, Dec 8, 2011 at 10:46 AM, Mark Miller wrote: > > On Dec 8, 2011, at 8:50 AM, Jamie Johnson wrote: > >> Isn't the codec stuff merged with trunk now? > > Robert merged this recently AFAIK. > true but that issue only moved the majority of the rest of the index (stored fields, term vectors, fi

Re: Solr Lucene Index Version

2011-12-08 Thread Mark Miller
On Dec 8, 2011, at 8:50 AM, Jamie Johnson wrote: > Isn't the codec stuff merged with trunk now? Robert merged this recently AFAIK. - Mark Miller lucidimagination.com

Re: Solr Lucene Index Version

2011-12-08 Thread Jamie Johnson
Mark, Agreed that Replication wouldn't help, I was dreaming that there was some intermediate format used in replication. Ideally you are right, I could just reindex the data and go on with life, but my case is not so simple. Currently we have some set of processes which is run against the raw ar

Re: Solr Lucene Index Version

2011-12-08 Thread Andrzej Bialecki
On 08/12/2011 05:00, Mark Miller wrote: Replication just copies the index, so I'm not sure how this would help offhand? With SolrCloud this is a breeze - just fire up another replica for a shard and the current index will replicate to it. If you where willing to export the data to some portabl

Re: Solr Lucene Index Version

2011-12-07 Thread Mark Miller
Replication just copies the index, so I'm not sure how this would help offhand? With SolrCloud this is a breeze - just fire up another replica for a shard and the current index will replicate to it. If you where willing to export the data to some portable format and then pull it back in, why no

Re: Solr Lucene Index Version

2011-12-07 Thread Jamie Johnson
Yeah I was actually hoping that some how I could use the replication handler to do this, fire up 1 shard, set another as a slave and see if it would replicate the index to it but obviously I'm not sure that would work either. Something like this would be great too https://issues.apache.org/jira/br

Re: Solr Lucene Index Version

2011-12-07 Thread Mark Miller
Unfortunately, I think the the only silver bullet here, for pure Solr, is to build a system that makes it possible to reindex somehow. On Dec 7, 2011, at 1:38 PM, Erik Hatcher wrote: > > On Dec 7, 2011, at 13:20 , Shawn Heisey wrote: > >> On 12/6/2011 2:06 PM, Erik Hatcher wrote: >>> I think t

Re: Solr Lucene Index Version

2011-12-07 Thread Erik Hatcher
On Dec 7, 2011, at 13:20 , Shawn Heisey wrote: > On 12/6/2011 2:06 PM, Erik Hatcher wrote: >> I think the best thing that you could do here would be to lock in a version >> of Lucene (all the Lucene libraries) that you use with SolrCloud. Certainly >> not out of the realm of possibilities of s

Re: Solr Lucene Index Version

2011-12-07 Thread Shawn Heisey
On 12/6/2011 2:06 PM, Erik Hatcher wrote: I think the best thing that you could do here would be to lock in a version of Lucene (all the Lucene libraries) that you use with SolrCloud. Certainly not out of the realm of possibilities of some upcoming SolrCloud capability that requires some upgr

Re: Solr Lucene Index Version

2011-12-07 Thread Erik Hatcher
Jamie - The details would of course be entirely dependent on what changed, but with Lucene trunk/4.0 there is the flexible indexing API with codecs. I imagine with a compatibility codec layer one could provide some insulation to changes. You're at big scale, so the "just reindex everything" an

Re: Solr Lucene Index Version

2011-12-07 Thread Jamie Johnson
Erik, Do you have any details behind what would be required to write a tool to move from one index format to another? Any examples/suggestions would be appreciated. On Tue, Dec 6, 2011 at 5:19 PM, Jamie Johnson wrote: > What about modifying something like SolrIndexConfig.java to change the > lu

Re: Solr Lucene Index Version

2011-12-06 Thread Jamie Johnson
What about modifying something like SolrIndexConfig.java to change the lucene version that is used when creating the index? (may not be the right place, but is something like this possible?) On Tue, Dec 6, 2011 at 5:13 PM, Erik Hatcher wrote: > Right.  Not sure what to advise you.  We have worke

Re: Solr Lucene Index Version

2011-12-06 Thread Erik Hatcher
Right. Not sure what to advise you. We have worked on this problem with our LucidWorks platform and have some tools available to do this sort of thing, I think, but it's not generally something that you can do with Lucene going from a snapshot to a released version. Perhaps others with deeper

Re: Solr Lucene Index Version

2011-12-06 Thread Jamie Johnson
Problem is that really doesn't help me. We still have the same issue that when the 4.0 becomes final there is no migration utility from this pre 4.0 version to 4.0, right? On Tue, Dec 6, 2011 at 4:36 PM, Erik Hatcher wrote: > Oh geez... no... I didn't mean 3.x JARs... I meant the trunk/4.0 ones

Re: Solr Lucene Index Version

2011-12-06 Thread Erik Hatcher
Oh geez... no... I didn't mean 3.x JARs... I meant the trunk/4.0 ones that are there now. Erik On Dec 6, 2011, at 16:22 , Jamie Johnson wrote: > So if I wanted to used lucene index 3.5 with SolrCloud I "should" be > able to just move the 3.5 jars in and remove any of the snapshot jars >

Re: Solr Lucene Index Version

2011-12-06 Thread Jamie Johnson
So if I wanted to used lucene index 3.5 with SolrCloud I "should" be able to just move the 3.5 jars in and remove any of the snapshot jars that are present when I build locally? On Tue, Dec 6, 2011 at 4:06 PM, Erik Hatcher wrote: > Jamie - > > I think the best thing that you could do here would b

Re: Solr Lucene Index Version

2011-12-06 Thread Erik Hatcher
Jamie - I think the best thing that you could do here would be to lock in a version of Lucene (all the Lucene libraries) that you use with SolrCloud. Certainly not out of the realm of possibilities of some upcoming SolrCloud capability that requires some upgrading of Lucene though, but you may

Re: Solr Lucene Index Version

2011-12-06 Thread Jamie Johnson
Thanks, but I don't believe that will do it. From my understanding that does not control the index version written, it's used to control the behavior of some analyzers (taken from some googling). I'd love if someone told me otherwise though. On Tue, Dec 6, 2011 at 3:48 PM, Alireza Salimi wrote:

Re: Solr Lucene Index Version

2011-12-06 Thread Alireza Salimi
Hi, I'm not sure if it would help. in solrconfig.xml: LUCENE_34 On Tue, Dec 6, 2011 at 3:14 PM, Jamie Johnson wrote: > Is there a way to specify the index version solr uses? We're > currently using SolrCloud but with the index format changing I'd be > preferable to be able to specify a p

Solr Lucene Index Version

2011-12-06 Thread Jamie Johnson
Is there a way to specify the index version solr uses? We're currently using SolrCloud but with the index format changing I'd be preferable to be able to specify a particular index format to avoid having to do a complete reindex. Is this possible?