Thanks Hoss. 1. We currently use Solr 4.3.0. 2. I understand this architecture of LazyFields, but i did not understand why multiple LazyFields should be created for the multivalued field. You can't load a part of them. If you request the field, you will get ALL of its values. so 100 (or more) placeholders are not necessary in this case. Moreover, why should Solr KNOW how much values are in that unloaded field? 3. In our poor case, we might handle some concurrent queries, each one requests rows=2000.
What do you think about temporary disabling documentCache, for a specific query? On Thu, Aug 29, 2013 at 10:11 PM, Chris Hostetter <hossman_luc...@fucit.org>wrote: > > : The main issue we found was a lot of millions of LazyField instances, > : taking ~2GB of memory, even though queries request about 10 small fields > : only. > > which version of Solr are you using? there was a really bad bug with > lazyFieldLoading fixed in Solr 4.2.1 (SOLR-4589) > > : We've found that LazyDocument creates a LazyField object for every item > in > : a multivalued field, even if do not want this field. > > right, that's exactly how lazyFieldLoading is designed to work -- instead > of loading the full field values into ram, only a small LazyField object > is loaded in it's place and that LazyField only fetches the underlying > data if/when it's requested. > > If the LazyField instances weren't created as placeholders, subsequent > requests for the document that *might* request additional fields (beyond > the "10 small fields" that were requested the first time) would have no > way of knowing if/when those additional fields existed to be able to fetch > them from the index. > > : In our case, documentCache was configured to 32000. There are 2 cores per > : node, so 64000 LazyDocument instances are in memory. This is pretty big > : number, and we'll reduce it. > > FWIW: Even at 1/10 that size, that seems like a ridiculously large > documentCache to me. > > > -Hoss >
