Shawn Heisey wrote > If the data for a field in the results comes from docValues instead of > stored fields, I don't think it is compressed, which hopefully means > that if a field is NOT requested, the corresponding docValues data is > never read.
I think we need to make a consideration here. DocValues is a data structure per field ( column style). And it is compressed on disk ( https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/codecs/lucene70/Lucene70DocValuesFormat.java ). I took a brief look to the code,I was wondering if it is possible that the index reader will read the entire segment content for all the docValues per type( like all the numeric doc values, all the binary ect) and then put it in a map in the Solr process heap only the docValues related the fields requested ? So the OS should memory map the whole content for the segment and then when Solr requests a specific field, it accesses the entry in the Map in the Heap ( really a brief look into Lucene70DocValuesProducer so I may be completely wrong). If this is correct, it means that we read the entire content from the segment for the docValues, even the portion related a field that is not requested. The only difference would be that a not requested doc Values field content, will not be stored in the Solr process heap. Is there any place to read more about this ? ( apart the source code) Cheers -- View this message in context: http://lucene.472066.n3.nabble.com/A-feature-idea-for-discussion-fields-that-can-only-be-explicitly-retrieved-tp4313890p4314288.html Sent from the Solr - User mailing list archive at Nabble.com.
