Re: document storage

2011-05-16 Thread Mike Sokolov
On 05/15/2011 11:48 AM, Erick Erickson wrote: Where are the documents coming from? Because storing them ONLY in Solr risks losing them if your index is somehow hosed. In our case, we generally have source documents and can reproduce the index if need be, but that's a good point. Storing the

Re: document storage

2011-05-15 Thread Erick Erickson
Where are the documents coming from? Because storing them ONLY in Solr risks losing them if your index is somehow hosed. Storing them externally only has the advantage that your index will be much smaller, which helps when replicating as you scale. The downside here is that highlighting will be mo

Re: document storage

2011-05-13 Thread Rich Cariens
We've decided to store the original document in both Solr and external repositories. This is to support the following: 1. highlighting - We need to mark-up the entire document with hit-terms. However if this was the only reason to store the text I'd seriously consider calling out to the e

document storage

2011-05-13 Thread Mike Sokolov
Would anyone care to comment on the merits of storing indexed full-text documents in Solr versus storing them externally? It seems there are three options for us: 1) store documents both in Solr and externally - this is what we are doing now, and gives us all sorts of flexibility, but doesn't