Re: Confusing DocValues documentation

2018-01-02 Thread Shawn Heisey
On 12/22/2017 12:45 PM, Tech Id wrote: > It seems that stored="false" docValues="true" is the default in Solr's > github and the recommended way to go. Like most things in Solr, there's no simple answer.  It depends. For the purposes of information retrieval (not facets, grouping, or sorting), wh

Re: Confusing DocValues documentation

2017-12-22 Thread Erick Erickson
About the docs. Recently we've changed the documents to be asciidoc format One of the ways to contribute is to raise a JIRA and submit a documentation patch. See: https://wiki.apache.org/solr/HowToContribute It's valuable to have people reading docs and trying to understand them help update them

Re: Confusing DocValues documentation

2017-12-22 Thread Tech Id
Thanks Emir, It seems that stored="false" docValues="true" is the default in Solr's github and the recommended way to go. grep "docValues=\"true\"" ./server/solr/configsets/_default/conf/managed-schema Point fields don't support FieldCache, so they must have docValues="

Re: Confusing DocValues documentation

2017-12-22 Thread Emir Arnautović
Your questions are already more or less answered: > 1) If the docValues are that good, can we git rid of the stored values > altogether? You can if you want - just configure your field with stored=“false” and docValues=“true”. Note that you can do that only if: * field is not analyzed (you cannot

Re: Confusing DocValues documentation

2017-12-22 Thread Tech Id
Very interesting discussion SG and Erick. I wish these details were part of the official Solr documentation as well. And yes, "columnar format" did not give any useful information to me either. "A good explanation increases contributions to the project as more people become empowered to improvise

Re: Confusing DocValues documentation

2017-12-21 Thread Erick Erickson
OK, last bit of the tutorial. bq: But that does not seem to be helping with sorting or faceting of any kind. This seems to be like a good way to speed up a stored field's retrieval. These are the same thing. I have two docs. I have to know how they sort. Therefore I need the value in the sort fie

Re: Confusing DocValues documentation

2017-12-21 Thread S G
Thank you Eric. I guess the biggest piece I was missing was the sort on a field other than the search field. Once you have filtered a list of documents and then you want to sort, the inverted index cannot be used for lookup. You just have doc-IDs which are values in inverted index, not the keys. H

Re: Confusing DocValues documentation

2017-12-21 Thread Erick Erickson
Here's where you're going off the rails: "I can just look at the map-for-field-A" As I said before, you're totally right, all the information you need is there. But you're thinking of this as though speed weren't a premium when you say. "I can just look". Consider that there are single replicas ou

Re: Confusing DocValues documentation

2017-12-21 Thread S G
Thanks a lot Erick and Emir. I am still a bit confused and an example will help me a lot. Here is a little bit modified version of the same to illustrate my point more clearly. Let us consider 3 documents - doc1, doc2 and doc3 Each contains upto 3 fields - A, B and C. And the values for these fie

Re: Confusing DocValues documentation

2017-12-21 Thread Erick Erickson
bq: I do not see why sorting or faceting on any field A, B or C would be a problem. All the values for a field are there in one data-structure and it should be easy to sort or group-by on that. This is totally true just totally incomplete: ;) for a given field: Inverted structure (leaving out po

Re: Confusing DocValues documentation

2017-12-21 Thread Emir Arnautović
Hi SG, It is all ok - it’s just that notation is different. Please see inline comments. Regards, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 21 Dec 2017, at 18:56, S G wrote: > > Hi, > > It se