>From a Lucene perspective, it's certainly possible to do lazy field loading. That is, when loading a document you can determine at run time what fields to load, even on a per-document basis. I'm not entirely sure how to accomplish this in Solr, but I'd give long odds that there's a way.....
I did a writeup on this on the Wiki, see: http://wiki.apache.org/lucene-java/FieldSelectorPerformance?highlight=%28fieldselectorperformance%29 The title is FieldSelectorPerformance if you need to search the Wiki... Best Erick On Dec 27, 2007 10:28 AM, Britske <[EMAIL PROTECTED]> wrote: > > > > Yonik Seeley wrote: > > > > On Dec 27, 2007 9:45 AM, Britske <[EMAIL PROTECTED]> wrote: > >> I am using SolrJ to communicate with SOLR. My Solr-queries perform > within > >> range (between 50 ms and 300 ms) by looking at the solr log as ouputted > >> on > >> my (windows) commandline. > >> > >> However I discovered that the following command at all times takes > >> significantly longer than the number outputted in the solr-log, > >> (sometimes > >> about 400% longer): > > > > It's probably due to stored field retrieval. > > The time in the response includes everything except the time to write > > the response (since it appears at the beginning). Writing the > > response involves reading the stored fields of documents (this was > > done to allow one to stream a large number of documents w/o having > > them all in memory). > > > > SolrJ's parsing of the response should be a relatively small constant > > cost. > > > > -Yonik > > > > > > Is it normal to see this much time taken in stored field retrieval? And > where would I start to make sure that it is indeed caused by stored field > retrieval? > > It seems quite much to me, although I have kind if an out of the ordinary > setup with between 2000-4000 stored fields per document. By far the > largest > part is taken by various 'product-variants' and their respective prices > (indexed field) and other characteristics (stored only). > However only about 10 stored fields per document are returned for any > possible query. > > Would the time taken still include iterating the non-returned fields (of > which there are many in my case), or are only the returned fields > retrieved > in a map-like implementation? > > Thanks, > Geert-Jan > -- > View this message in context: > http://www.nabble.com/big-perf-difference-between-solr-server-vs.--SOlrJ-req.process%28solrserver%29-tp14513964p14514441.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
