Re: Does Solr deduplicate stored values

2014-07-18 Thread Erick Erickson
The data will be stored 100 times in your example, independently for each document, albeit compressed. Hmmm, doing that would certainly reduce the disk space requirements, but it'd also complicate the document read process. Instead of a single contiguous read from disk per document, there'd be mul

Does Solr deduplicate stored values

2014-07-18 Thread Alexandre Rafalovitch
Hello, Say I have 100 documents with the same large field value. Stored and indexed. I know the indexed tokens are stored only once with posting lists. But what about original stored values? Do I get 100 copies of those? Or is Solr smarter that that? Regards, Alex