Re: document storage

2011-05-16 Thread Mike Sokolov
On 05/15/2011 11:48 AM, Erick Erickson wrote: Where are the documents coming from? Because storing them ONLY in Solr risks losing them if your index is somehow hosed. In our case, we generally have source documents and can reproduce the index if need be, but that's a good point. Storing the

Re: document storage

2011-05-15 Thread Erick Erickson
Where are the documents coming from? Because storing them ONLY in Solr risks losing them if your index is somehow hosed. Storing them externally only has the advantage that your index will be much smaller, which helps when replicating as you scale. The downside here is that highlighting will be mo

Re: document storage

2011-05-13 Thread Rich Cariens
We've decided to store the original document in both Solr and external repositories. This is to support the following: 1. highlighting - We need to mark-up the entire document with hit-terms. However if this was the only reason to store the text I'd seriously consider calling out to the e