Thanks Toke. I’ve read some of your other helpful blog entries but I missed that one.
Did you find you had to do a full conversion all at once because simply turning on docvalues in the schema caused issues? I ask because my presupposition has been that we could turn it on without any harm as we incrementally converted our indexes. I have actually written a tool that will round robin between different indexes on the same server and update 10,000 items every two minutes by doing a simple atomic update (updating a date field). This actually works well in my testing. But this won’t work if enabling docvalues in the schema will lead to errors when fields don’t have docvalues actually populated. I.e.. the “IllegalStateException: unexpected docvalues type NONE for field 'id' (expected=SORTED)” error I see. I’m still trying to get to the bottom of whether that error means I cannot safely do an incremental conversion in-place. Your approach may come in handy if we cannot. -Ronald. On 8/24/16, 07:42, "Alessandro Benedetti" <[email protected]> wrote: Hi Toke ! Good stuff ! So basically using your tool you build a copy of the index ( similar to what optimize does) without affecting the main index right ? So the procedure would be : 1) Solr is running 2) Run the tool pointing to the Solr Index, this will be slow but will generate a new index copying from the stored content to docValues 3) Stop Solr 4) Change Solr schema enabling docValues 5) Point to the new converted index 6) Start Solr Am I right ? This is actually a useful tool when re-indexing could be extremely long. Cheers On Wed, Aug 24, 2016 at 12:05 PM, Toke Eskildsen <[email protected]> wrote: > On Tue, 2016-08-23 at 20:01 +0000, Ronald Wood wrote: > > In general, is there a way to migrate existing indexes (we have > > petabytes of data) by enabling docvalues and incrementally re- > > indexing? We expect the latter would take a month using an atomic > > update process. > > I did write a tool for that at some point, that works with Solr 4.x. > https://github.com/netarchivesuite/dvenabler > > It is not polished though, and the main problem is that it is slow to > convert. Some details at https://sbdevel.wordpress.com/2014/12/15/chang > ing-field-type-in-lucenesolr/ > > - Toke Eskildsen > > -- -------------------------- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
