Thanks Erik for reply, Since I was storing Id (its stored field) and after enabling docValues my guess is it will be stored in 2 places. also as per my understanding docValues are great when you have values which repeat. I am not sure how beneficial it would be for uniqueId field. I am looking at collection of few hundred billion documents , that is reason I really want to care about expense from design phase.
On Sun, Apr 24, 2016 at 7:24 PM, Erick Erickson <erickerick...@gmail.com> wrote: > In a word, "yes". > > DocValues aren't particularly expensive, or expensive at all. The idea > is that when you sort by a field or facet, the field has to be > "uninverted" which builds the entire structure in Java's JVM (this is > when the field is _not_ DocValues). > > DocValues essentially serialize this structure to disk. So your > on-disk index size is larger, but that size is MMaped rather than > stored on Java's heap. > > Really, the question I'd have to ask though is "why do you care about > the expense?". If you have a functional requirement that has to be > served by returning the id via the /export handler, you really have no > choice. > > Best, > Erick > > > On Sun, Apr 24, 2016 at 9:55 AM, sudsport s <sudssf2...@gmail.com> wrote: > > I was trying to use Streaming for reading basic tuple stream. I am using > > sort by id asc , > > I am getting following exception > > > > I am using export search handler as per > > https://cwiki.apache.org/confluence/display/solr/Exporting+Result+Sets > > > > null:java.io.IOException: id must have DocValues to use this feature. > > at > org.apache.solr.response.SortingResponseWriter.getFieldWriters(SortingResponseWriter.java:241) > > at > org.apache.solr.response.SortingResponseWriter.write(SortingResponseWriter.java:120) > > at > org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:53) > > at > org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:742) > > at > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:471) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214) > > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) > > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) > > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) > > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) > > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) > > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) > > at org.eclipse.jetty.server.session.SessionHandler.doScope( > > > > > > does it make sense to enable docValues for unique field? How expensive > is it? > > > > > > if I have existing collection can I update schema and optimize > > collection to get docvalues enabled for id? > > > > > > -- > > > > Thanks >
