On Oct 9, 2007, at 3:14 PM, Ryan McKinley wrote:
2. Figure out how to keep the base Tuple store in solr. I think
this will require finishing up SOLR-139. This would keep the the
core data in solr - so there is no good way to 'rebuild' the index.
With SOLR-139, cool stuff can be done to 'rebuild' an index
actually. Obviously if your store is Solr you'll be using stored
fields. So store the most basic stuff, and copyField things around.
With SOLR-139, to rebuild an index you simply reconfigure the
copyField settings and basically `touch` each document to reindex it.
I did this with Collex recently as I refactored all of my old Collex
tag architecture into SOLR-139. My tag design is nowhere near as
scalable as the one you're after, I don't think. Yonik has some
pretty prescient design ideas here:
<http://wiki.apache.org/solr/UserTagDesign>
Particularly interesting are the parts about leveraging intra Lucene
Field matching capability (Phrase/SpanQuery possibilities are pretty
neat) to reduce the number of fields.
3. something else? store input on disk?
*gasp* Inconceivable! :)
Erik
